DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicant
2. This communication is in response to the communication filed 1/13/2025. Claims 1-19 are currently pending.
Claim Rejections - 35 USC § 102
3. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
3.1. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
3.2. Claims 1, 6-7, 10, 15-16 and 19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Chukka et al. (US 2020/0342597).
CLAIM 1
Chukka teaches a system (Chukka: abstract) comprising:
at least one memory (Chukka: abstract; ¶¶ [0050]-[0052] “memory 201”; FIGS. 1-2); and
at least one processor (Chukka: abstract; ¶¶ [0050]-[0052] “processor”; FIGS. 1-2);
wherein the at least one processor configured to (Chukka: abstract; ¶¶ [0050]-[0052] “instructions for execution by the one or more processors”; FIGS. 1-2):
identify label information regarding a type of cell or tissue included in a pathological slide image by analyzing the pathological slide image using a machine learning model (Chukka: abstract; ¶¶ [0005] “automated image analysis of digitized tissue slides of biological specimens generally involves cell detection, cell segmentation, and feature extraction for each cell within an image (or region thereof), followed by cell classification, as well as region segmentation and classification”, [0008] “tissue region classification data and the final cell classification data is combined such that each pixel within the sample image is labeled with a single label denoting tissue type and or cell type”, [0113] “machine learning system”, [0118] “Deep learning (DL) is a machine learning”; FIGS. 1-8D),
generate a labeling image corresponding to the pathological slide image by setting a pixel value corresponding to the cell or tissue using a label color based on the label information (Granger: abstract; ¶¶ [0038] “color information associated with the internal structure of the target subject may be assigned to different pixel locations”, [0051] “the digital color image acquired by the imaging apparatus 12 is conventionally composed of elementary color pixels. Each colored pixel can be coded over three digital components, each comprising the same number of bits, each component corresponding to a primary color, generally red, green or blue, also denoted by the term “RGB” components”; FIGS. 1-8D), and
control a display apparatus to display the pathological slide image and the labeling image (Chukka: abstract; ¶¶ [0112-[0115]]; FIGS. 1-8D).
CLAIM 6
Chukka teaches the system of claim 1, wherein the label information includes information regarding whether or not a biomarker is expressed in a cell and whether a cell is at least one of an immune cell, a tumor cell, a lymphocyte, a macrophage, a stroma cell, a fibroblast, or an adipocyte (Chukka: abstract; ¶¶ [0007] “detect and classify different cell types (e.g. tumor cells, stromal cells, lymphocytes, etc.) and tissue regions (e.g. tumor regions, lymphocyte-rich regions, stromal regions, necrotic regions, etc.) within a sample image”; FIGS. 1-8D).
CLAIM 7
Chukka teaches the system of claim 1, wherein the label information includes information regarding whether the tissue is at least one of cancer, cancer stroma, necrosis, or background (Chukka: abstract; ¶¶ [0007] “detect and classify different cell types (e.g. tumor cells, stromal cells, lymphocytes, etc.) and tissue regions (e.g. tumor regions, lymphocyte-rich regions, stromal regions, necrotic regions, etc.) within a sample image”; FIGS. 1-8D).
CLAIMS 10, 15-16
Claims 10, and 15-16 repeat substantially the same limitations as those in claims 1, and 6-7. As such, claims 10, and 15-16 are rejected for substantially the same reasons given for claims 1, and 6-7 and are incorporated herein.
CLAIM 19
Claim 19 repeats substantially the same limitations as those in claims 1 and 10. As such, claim 19 is rejected for substantially the same reasons given for claims 1 and 10 and are incorporated herein.
Claim Rejections - 35 USC § 103
4. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4.1. Claims 2, 9, 11, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Chukka et al. (US 2020/0342597), in view of Granger (US 2009/0054756).
CLAIM 2
Chukka does not appear to explicitly teach the system of claim 1, wherein the labeling image has a lower resolution than a resolution of the pathological slide image.
Granger, however, teaches wherein the labeling image has a lower resolution than a resolution of the pathological slide image (Granger: abstract; ¶¶ [0025] “imaging system converts the captured digital information to luminance information and assigns this luminance information to a pixel location in a two dimensional bitmap for display on a computer screen”, [0032], [0038 “color information associated with the internal structure of the target subject may be assigned to different pixel locations in the two dimensional bitmap”, [0043]; FIGS. 1-6).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to include the method and system for identifying tissue pathology using different resolution images, as taught by Granger, with the deep-learning system and method for classifying biological images, as taught by Chukka, with the motivation of facilitating diagnosis of pathological tissue images (Granger: abstract; ¶¶ [0001]-[0014]).
CLAIM 9
Chukka does not appear to explicitly teach the system of claim 1, wherein the processor further configured to: generate a bitmap image based on the label information, wherein each pixel of the bitmap image has, as a pixel value, a label ID of a cell or tissue present at a corresponding location, convert the bitmap image to the labeling image by setting the pixel value corresponding to the cell or tissue within the bitmap image using the label color within the label information.
Granger, however, teaches wherein the processor further configured to: generate a bitmap image based on the label information, wherein each pixel of the bitmap image has, as a pixel value, a label ID of a cell or tissue present at a corresponding location, convert the bitmap image to the labeling image by setting the pixel value corresponding to the cell or tissue within the bitmap image using the label color within the label information (Granger: abstract; ¶¶ [0025] “imaging system converts the captured digital information to luminance information and assigns this luminance information to a pixel location in a two dimensional bitmap for display on a computer screen”, [0032], [0038 “color information associated with the internal structure of the target subject may be assigned to different pixel locations in the two dimensional bitmap”, [0043]; FIGS. 1-6).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to include the method and system for identifying tissue pathology using different resolution images, as taught by Granger, with the deep-learning system and method for classifying biological images, as taught by Chukka, with the motivation of facilitating diagnosis of pathological tissue images (Granger: abstract; ¶¶ [0001]-[0014]).
CLAIMS 11, 18
Claims 11 and 18 repeat substantially the same limitations as those in claims 2 and 9. As such, claims 11 and 18 are rejected for substantially the same reasons given for claims 2 and 9 and are incorporated herein.
4.2. Claims 3, 8, 12, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Chukka et al. (US 2020/0342597), in view of Fuchs et al. (US 2021/0295528).
CLAIM 3
Chukka does not appear to explicitly teach the system of claim 1, wherein the at least one processer further configured to: receive a user input for selecting or deselecting a check box displayed on the display apparatus, and control the display apparatus so that the labeling image indicating a location of the cell or tissue is displayed or removed on the pathological slide image based on the user input.
Fuchs, however, teaches wherein the at least one processer further configured to: receive a user input for selecting or deselecting a check box displayed on the display apparatus, and control the display apparatus so that the labeling image indicating a location of the cell or tissue is displayed or removed on the pathological slide image based on the user input (Fuchs: abstract; ¶¶ [0079]-[0080] User can select regions of interest, a portion, an area, etc. as part of an annotation., [0081]-[0082] “interface 514 may include an indicator 526 to select whether the sample image 518 is to be included in the retraining of the image segmentation model 504. The indicator 526 may be a selectable user interface element, such as a command button, a radio button, a checkbox (e.g., as depicted), and a prompt, among others”; FIGS. 1-8).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to include the deep interactive learning for image segmentation including check boxes for user selection of image display options, as taught by Fuchs, with the deep-learning system and method for classifying biological images, as taught by Chukka, with the motivation of facilitating user interaction (Fuchs: abstract; ¶¶ [0081]-[0082]).
CLAIM 8
Chukka does not appear to explicitly teach the system of claim 1, wherein the at least one processer further configured to: receive a user input for selecting or deselecting a check box displayed on the display apparatus, and control the display apparatus so that the labeling image including an IP map and/or a histological feature map is displayed or removed on the pathological slide image based on the user input.
Fuchs, however, teaches wherein the at least one processer further configured to: receive a user input for selecting or deselecting a check box displayed on the display apparatus, and control the display apparatus so that the labeling image including an IP map and/or a histological feature map is displayed or removed on the pathological slide image based on the user input (Fuchs: abstract; ¶¶ [0069]-[0070] “each annotation 522 may identify or indicate different conditions associated with the region of interest 520. For example, when the sample image 518 is a WSI of the sample tissue, the annotation 522 may identify one of the various histopathological characteristics, such as carcinoma tissue, benign epithelial tissue, stroma tissue, necrotic tissue, and adipose tissue, among others”, [0081]-[0082] “interface 514 may include an indicator 526 to select whether the sample image 518 is to be included in the retraining of the image segmentation model 504. The indicator 526 may be a selectable user interface element, such as a command button, a radio button, a checkbox (e.g., as depicted), and a prompt, among others”; FIGS. 1-8).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to include the deep interactive learning for image segmentation including check boxes for user selection of image display options, as taught by Fuchs, with the deep-learning system and method for classifying biological images, as taught by Chukka, with the motivation of facilitating user interaction (Fuchs: abstract; ¶¶ [0081]-[0082]).
CLAIMS 12, 17
Claims 12 and 17 repeat substantially the same limitations as those in claims 3 and 8. As such, claims 12 and 17 are rejected for substantially the same reasons given for claims 3 and 8 and are incorporated herein.
4.3. Claims 4-5 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Chukka et al. (US 2020/0342597), in view of Howe et al. (US 2012/0069049).
CLAIM 4
Chukka does not appear to explicitly teach the system of claim 1, wherein the at least one processer further configured to: receive a user input, change a portion of the pathological slide image output on the display apparatus based on the user input, and control the display apparatus to display a portion of the labeling image in correspondence to the portion of the pathological slide image.
Howe, however, teaches wherein the at least one processer further configured to: receive a user input, change a portion of the pathological slide image output on the display apparatus based on the user input, and control the display apparatus to display a portion of the labeling image in correspondence to the portion of the pathological slide image (Howe: abstract; ¶¶ [0051] “thumbnail images (e.g., selected slides 306, 308 of FIG. 3), may be selected”, [0075] “capability to zoom in and out of magnification levels in order to see any desired amount of image detail. For example, the user can select "zoom" by a pull-down or pop-up menu or may make a zoom selection and displace the scroll wheel of a mouse or the ball of a trackball device. The viewer module then zooms in or out according to the sensed displacement or based on the value received from a zoom query response”, [0091] “user can interface with (for example, enter commands and information) the computer 1310 through input devices”; FIGS. 1-13).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to include the digital pathology image manipulation system and method, as taught by Howe, with the deep-learning system and method for classifying biological images, as taught by Chukka, with the motivation of improving pathology image processing efficiency (Howe: abstract; ¶¶ [0001]-[0005]).
CLAIM 5
Chukka does not appear to explicitly teach the system of claim 1, wherein the at least one processer further configured to: control the display apparatus to display a thumbnail image representing a portion of the pathological slide image output on the display apparatus.
Howe, however, teaches wherein the at least one processer further configured to: control the display apparatus to display a thumbnail image representing a portion of the pathological slide image output on the display apparatus (Howe: abstract; ¶¶ [0051] “thumbnail images (e.g., selected slides 306, 308 of FIG. 3), may be selected”, [0091] “user can interface with (for example, enter commands and information) the computer 1310 through input devices”; FIGS. 1-13).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to include the digital pathology image manipulation system and method, as taught by Howe, with the deep-learning system and method for classifying biological images, as taught by Chukka, with the motivation of improving pathology image processing efficiency (Howe: abstract; ¶¶ [0001]-[0005]).
CLAIMS 13-14
Claims 13-14 repeat substantially the same limitations as those in claims 4-5. As such, claims 13-14 are rejected for substantially the same reasons given for claims 4-5 and are incorporated herein.
Conclusion
5. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael Tomaszewski whose telephone number is (313)446-4863. The examiner can normally be reached M-F 5:30 am - 2:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Peter H Choi can be reached at (469) 295-9171. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL TOMASZEWSKI/Primary Examiner, Art Unit 3681
Prosecution Insights