DETAILED ACTION
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim18 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims do not fall within at least one of the four categories of patent eligible subject matter because “computer readable medium” encompasses both transitory and non-transitory forms of media, thereby rendering the claim as a whole non-statutory (i.e., transitory media are non-statutory) for failing to be limited to one of the four statutory categories of invention.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 8-11, 18-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu ( US 20170206656) in view of West ( US 20190073510 )
Regarding claim 1, Liu teaches a method for analyzing a medical image performed by a user terminal including at least one processor, the method comprising:
determining a region of interest ( 102 in FIG. 1; REGION MODULE in FIG. 2) of a medical image based on a first user input on a user interface( 101 in Fig. 1 and Fig. 2);
distinguishing the cell present in the region of interest as positive or negative ( 224 and 226 in FIG. 2) based on a threshold which is adjustable according to a second user input on the user interface by using the position information and the staining information of the stained cell present in the region of interest[0031], the negative stain image 226 is compared to a negative nucleus threshold value to determine if the current pixel is a potential negative stained nucleus center); and
outputting analysis information for the cell distinguished as a positive or negative through the user interface( [0031], If the pixel value is greater than the negative nucleus threshold, the current pixel is identified as a potential negative stained nucleus center at output 409; otherwise, the pixel is excluded as a potential negative stained nucleus center).
Liu does not expressly teach
transmitting identification information and contour information of the region of interest to a server, and receiving position information and staining information of a stained cell present in the region of interest from the server;
However, West teaches
transmitting identification information and contour information of the region of interest to a server([0056], images and case data can be imported into the computing system 108, including, but not limited to, a local file upload, a remote uniform resource locator (URL) upload, a file transfer protocol (FTP), P2P, an image matching, or an upload via a third party storage provider), and receiving position information( [0092], given a position and a shape of such elements, such as objects, within a tissue image) and staining information of a stained cell present in the region of interest from the server( [0093], evaluated estimates or predictions of medical conditions or biological phenomena that include but are not limited to .. stain quality, or suggested regions of interest);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Liu and west, by implementing some processing steps in Liu on a server, and uploading the images to server as taught by West for further processing, with motivation that “more computation offloaded from the clients onto the servers leads to lighter clients, such as being more reliant on network sources and less reliant on local computing resources” ( West, [0044]) .
Regarding claim 2, Liu in view of West teaches the method of claim 1, further comprising: converting and storing a medical image to be analyzed for each resolution level( West, [0067], A high resolution source image can be smoothed and down-sampled, such as by a factor of two in both dimensions, yielding a lower-resolution, and thus a smaller resultant image. The same process can be applied to the smaller resultant image. This process can be iterated or repeated multiple times to create a series of images decreasing in resolution); and
outputting a medical image of a resolution level corresponding to a magnification level adjusted according to a third user input on the user interface through the user interface, in the stored medical image( West, [0068], the viewing application may render one of many lower resolution layers that have been generated).
Regarding claim 3, Liu in view of West teaches the method of claim 1, wherein the determining of the region of interest of the medical image based on the first user input on the user interface includes:
generating the region of interest of the medical image as a closed contour according to the first user input( Liu, [0028], extracts a user interest area from the input slide image).
Regarding claim 4, Liu in view of West teaches the method of claim 1, wherein the contour information of the region of interest includes a coordinate value of a point which becomes a reference of a contour of the region of interest( Liu, [0056], as input the potential negative stained nucleus centers and the binary image and applies distance masking to these inputs to find the minimum distance of each white pixel (value of 1) to any black pixel (value of 0)), and
wherein the point which becomes the reference of the contour of the region of interest is determined according to the first user input (Liu, [0057], For each potential negative stained nucleus center pixel identified in the blob identification module 762, the corresponding distance value in the same location is added onto the potential negative stained nucleus center pixel) .
Regarding claim 8, Liu in view of West teaches the method of claim 1, wherein the threshold adjustable according to the second user input on the user interface includes a first reference value indicating a staining expression intensity for distinguishing the cell present in the region of interest into positive or negative( [0024], The stain intensity of pixels of the positive stain image and the negative stain image are each analyzed to determine a pixel count for each stain image).
Regarding claim 9, Liu in view of West teaches the method of claim 8, wherein the threshold adjustable according to the second user input on the user interface further includes a second reference value for distinguishing a positive cell determined according to the first reference value according to a staining expression intensity(Liu, [0024], Positive stained nuclei in the positive stain image are identified using the size range parameter and the average intensity of the positive stain... A total number of identified nuclei is computed by summing the number of identified positive stained nuclei and the number of identified negative stained nuclei).
Regarding claim 10, Liu in view of West teaches the method of claim 1, wherein the distinguishing of the cell present in the region of interest as positive or negative includes: comparing the threshold and the staining information of the stained cell present in the region of interest and distinguishing the cell as positive or negative(Liu, [0024], Negative stained nuclei in the negative stain image are identified using the size range parameter) .
Regarding claim 11, Liu in view of West teaches the method of claim 10, wherein the comparing of the threshold and the staining information of the stained cell present in the region of interest and distinguishing of the cell as positive or negative includes:
comparing a first reference value included in the threshold and the staining information of the stained cell present in the region of interest, and distinguishing the cell as positive or negative( Liu, [0031], each pixel of the negative stain image 226 is compared to a negative nucleus threshold value to determine if the current pixel is a potential negative stained nucleus center), and
comparing a second reference value included in the threshold and staining information of a positive cell distinguished according to the first reference value, and distinguishing the positive cell according to a staining expression intensity (Liu, [0032], T1 is a threshold value that determines if a current pixel has strong enough negative stain to be qualified as a potential negative stained nuclei pixel).
Claims 18-19 recite medium and system for the method in claim 1. Since Liu also teaches medium and system (Fig. 7), those claims are also rejected.
Claim(s) 5-7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of West, further in view of BARN ( CA 2944831)
Regarding claim 5, Liu in view of West teaches the method of claim 1, wherein the position information of the stained cell present in the region of interest includes: a coordinate value of the stained cell present in the region of interest, which is calculated from an input image including the region of interest, and
wherein the staining information of the stained cell present in the region of interest includes: a staining ratio of the stained cell, which is calculated in a cell region of the region of interest corresponding to the coordinate value by the server(Liu, [0057], For each potential negative stained nucleus center pixel identified in the blob identification module 762, the corresponding distance value in the same location is added onto the potential negative stained nucleus center pixel).
Liu in view of West does not expressly teach
through a neural network model of the server;
wherein the staining information of the stained cell present in the region of interest includes: a staining ratio of the stained cell, which is calculated in a cell region of the region of interest corresponding to the coordinate value by the server.
However, BARNS teaches
through a neural network model of the server([0025], the convolutional neural network for determining a probability for the presence of a biological feature);
wherein the staining information of the stained cell present in the region of interest includes: a staining ratio of the stained cell, which is calculated in a cell region of the region of interest corresponding to the coordinate value by the server([0085], evaluate the ratio of the area of C and the area of A (or B). If the ratio is greater than a threshold… outside tumor regions are excluded).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Liu in view of West with that of BARNS, by substitute the distance calculation method in Liu and West with the neural network method taught by Barns, with motivation for “ identifying a location of interest within the field of view and then extracting the image patch that contains this location of interest”( BARNS, [0025])
Regarding claim 6, Liu in view of West and BARNS teaches the method of claim 5, wherein the staining ratio of the stained cell is a ratio of an area of a positive region of the cell expressed by staining in the cell region to a total area of the cell region( Liu, The stain intensity of pixels of the positive stain image and the negative stain image are each analyzed to determine a pixel count for each stain image, which pixel counts are used for obtaining a size range parameter An average intensity of the positive stain is determined using the positive stain image. Positive stained nuclei in the positive stain image are identified using the size range parameter and the average intensity of the positive stain ;[0024], BARNS, [0085], FOVs of selected markers (e.g. FP3 and 008) from an individual marker image, … We then evaluate the ratio of the area of C and the area of A (or B)).
Regarding claim 7, Liu in view of West and BARNS teaches the method of claim 5, wherein the staining ratio of the stained cell is calculated based on a binary image for the cell region generated based on a staining intensity of the region of interest by the server(BARNS, [0084], the FOV may be a border around a region of high intensity)
Claim(s) 12-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of West, further in view of Kamath ( US20150032671)
Regarding claim 12, Liu in view of West teaches the method of claim 1, wherein the distinguishing of the cell present in the region of interest as positive or negative includes:
Liu in view of West does not expressly teach
updating, when the threshold is adjusted by the second user input on the user interface, a result of distinguishing the cell as positive or negative based on the adjusted threshold.
However, Kamath teaches
updating, when the threshold is adjusted by the second user input on the user interface, a result of distinguishing the cell as positive or negative based on the adjusted threshold( [0181], selection component 3830 for selecting expression level criteria (for example, a threshold) for each of the selected biomarkers; [0214]-[0215], high and/or low thresholds to be used in an analysis of a possible correlation with a clinical outcome… The correlation analysis may be used to determine whether a positive correlation or a negative correlation exists between the selected clinical outcome and the selected biomarkers for the cohort).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Liu in view of West with that of Kamath, by using user interface as taught by Kamath, to adjust threshold in the teaching of Liu in view of West, with motivation “to isolate a population of biological particles in a biological sample for exclusion from further processing” ( Kamath, Abstract)
Regarding claim 13, Liu in view of West teaches the method of claim 1, wherein the outputting of the analysis information through the user interface includes:
Liu in view of West does not expressly teach
outputting a medical image in which a positive cell and a negative cell distinguished by using a first reference value included in the threshold are displayed in different colors to a first region of the user interface.
However Kamath teaches
outputting a medical image in which a positive cell and a negative cell distinguished by using a first reference value included in the threshold are displayed in different colors to a first region of the user interface( [0082],and one or more corresponding colors for the biomarkers).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Liu in view of West with that of Kamath, by using the medical image display method taught by Kamath in images in Liu in view of West, with motivation “to isolate a population of biological particles in a biological sample for exclusion from further processing” ( Kamath, Abstract)
Regarding claim 14, Liu in view of West teaches the method of claim 13.
Liu in view of West does not expressly teach
wherein positive cells distinguished by using the first reference value included in the threshold are displayed in different colors according to a staining expression intensity distinguished by using a second reference value included in the threshold.
However, Kamath teaches
wherein positive cells distinguished by using the first reference value included in the threshold are displayed in different colors according to a staining expression intensity distinguished by using a second reference value included in the threshold( [0126], if the expression level of a marker in a cell is above a pre-defined threshold, the corresponding grayscale value may be an "on" or "high" value (e.g., the color red). Conversely, if the expression level of a marker in a cell is below a pre-defined threshold, the corresponding grayscale value may be an "off" or "low" value (e.g., the color green)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Liu in view of West with that of Kamath, by using the medical image display method taught by Kamath in images in Liu in view of West, with motivation “to isolate a population of biological particles in a biological sample for exclusion from further processing” ( Kamath, Abstract).
Regarding claim 15, Liu in view of West teaches the method of claim 13, wherein the outputting of the medical image in which the positive cell and the negative cell distinguished by using the threshold are displayed in different colors to the first region of the user interface includes:
Liu in view of West does not expressly teach
storing a region of interest of the medical image in which the positive cell and the negative cell distinguished by using the threshold are displayed in different colors as a separate image, and
converting, when a magnification level for the region of interest is adjusted or the region of interest is moved from the first region according to a third user input on the user interface, a size or a coordinate of the region of interest stored as the separate image according to the third user input.
However Kamath teaches
storing a region of interest of the medical image in which the positive cell and the negative cell distinguished by using the threshold are displayed in different colors as a separate image( [0176]-[0178], The morphological feature selection component 3910 may further include a control 3918 for applying the overlay as well as a control 3920 for saving the selection settings, for example, as a .txt file;… selecting one of the four regions defined by the two thresholds illustrated in FIG. 17. The particles in the region of interest selected in the statistical representation 1920 may then be identified in the morphological representation 1910), and
converting, when a magnification level for the region of interest is adjusted or the region of interest is moved from the first region according to a third user input on the user interface, a size or a coordinate of the region of interest stored as the separate image according to the third user input (West, [0067], A high resolution source image can be smoothed and down-sampled, such as by a factor of two in both dimensions, yielding a lower-resolution, and thus a smaller resultant image. The same process can be applied to the smaller resultant image. This process can be iterated or repeated multiple times to create a series of images decreasing in resolution) .
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Liu in view of West with that of Kamath, by using the medical image display method taught by Kamath in images in Liu in view of West, with motivation “to isolate a population of biological particles in a biological sample for exclusion from further processing” ( Kamath, Abstract).
Regarding claim 16, Liu in view of West teaches the method of claim 1, wherein the outputting of the analysis information through the user interface includes:
Liu in view of West does not expressly teach
outputting, to a second region of the user interface, at least one of a number of negative cells distinguished by using the threshold, a number of positive cells distinguished by using the threshold, a first ratio based on the number of positive cells, or a second ratio based on an overall average of staining expression intensities of stained cells present in the region of interest.
However Kamath teaches
outputting, to a second region of the user interface, at least one of a number of negative cells distinguished by using the threshold ( [0126], the number of intensity levels or values is two. In this case, grayscale values may be assigned one of the two values based on one or more pre-defined criteria. For example, if the expression level of a marker in a cell is above a pre-defined threshold, the corresponding grayscale value may be an "on" or "high" value (e.g., the color red). Conversely, if the expression level of a marker in a cell is below a pre-defined threshold, the corresponding grayscale value may be an "off" or "low" value (e.g., the color green)), a number of positive cells distinguished by using the threshold, a first ratio based on the number of positive cells, or a second ratio based on an overall average of staining expression intensities of stained cells present in the region of interest.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Liu in view of West with that of Kamath, by using the medical image display method taught by Kamath in images in Liu in view of West, with motivation “to isolate a population of biological particles in a biological sample for exclusion from further processing” ( Kamath, Abstract).
Regarding claim 17, Liu in view of West teaches the method of claim 1, wherein the user interface includes:
Liu in view of West does not expressly teach
a first region receiving the first user input, and outputting a medical image for determining the region of interest, a medical image in which the region of interest is visualized, or a medical image in which a positive cell and a negative cell distinguished by using the threshold displayed in different colors,
a second region outputting at least one of a number of negative cells distinguished by using the threshold, a number of positive cells distinguished by using the threshold, a first ratio based on the number of positive cells, or a second ratio based on the overall average of staining expression intensities of stained cells present in the region of interest, and
a third region receiving the second user input, and outputting information on the threshold.
However, Kamath teaches
a first region receiving the first user input, and outputting a medical image for determining the region of interest, a medical image in which the region of interest is visualized( [0009], field-of-view selection component ), or a medical image in which a positive cell and a negative cell distinguished by using the threshold displayed in different colors.
a second region outputting at least one of a number of negative cells distinguished by using the threshold, a number of positive cells distinguished by using the threshold([0009], user to select a field-of-view from a data ; [0160], user to select expression level criteria for each of the selected biomarkers. For example, the criteria may be that the expression level is above a certain threshold value, below a certain threshold value, or between two threshold values ), a first ratio based on the number of positive cells, or a second ratio based on the overall average of staining expression intensities of stained cells present in the region of interest, and
a third region receiving the second user input, and outputting information on the threshold( [0300], rendered on a visual display device and including human-viewable inputs and outputs).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Liu in view of West with that of Kamath, by using the medical image display method taught by Kamath in images in Liu in view of West, with motivation “to isolate a population of biological particles in a biological sample for exclusion from further processing” ( Kamath, Abstract).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANGENG SUN whose telephone number is (571)272-3712. The examiner can normally be reached 8am to 5pm, EST, M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Randolph Vincent can be reached at 571 272 8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JIANGENG SUN
Examiner
Art Unit 2661
/Jiangeng Sun/Examiner, Art Unit 2671
Prosecution Insights