Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-6, 10-16, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Bauer et al. (US 20060041385 A1), (hereinafter Bauer).
Regarding claim 1, Bauer teaches a method for performing color calibration using calibration slides to automate detection, the method comprising:
receiving, by one or more processors coupled to a non-transitory memory, a plurality of calibration images from a scanner, each of the plurality of calibration images depicting a portion of a respective stained tissue sample having a known score value (Bauer, “Referring now to FIGS. 1 and 2, an apparatus for automated cell analysis of biological Samples is generally indicated by reference numeral 10 as shown in perspective view in FIG. 1 and in block diagram form in FIG. 2… A computer Subsystem comprises a computer 22 having at least one System processor 23, and a communications modem 29.”, pg. 3, paragraph 0032, lines 1-11, “FIG. 7 illustrates a flow diagram of a method 300 of establishing a calibration curve for accurately determining a target molecule in Sample 216 using image analysis… At box 310 a user, e.g., a pathologist or technician, selects and executes a Standard method to generate reference input data for a calibration curve. A calibration curve for CISH includes reference input data for the number of copies of a target gene… In box 320 a user prepares a sample on microscope Slide. The sample is, for example, a Second set of cultured cell lines such as those described in 310. The sample provides optical density data for a target molecule. The sample is stained for a target molecule using Standard CISH or IHC methods… At 330, an optical image of a sample on a microscope slide is generated using a computer-aided image analysis System such as apparatus 10 as described in reference to FIG. 1.”, pgs. 5 and 6, paragraphs 0048-0052, see Fig. 7, Initial control samples are collected for staining and imaging. These samples are associated with known scores, such as reference input data for gene copy number or protein expression levels.);
determining, by the one or more processors, a correction function for the scanner based on the plurality of calibration images (Bauer, “At 340 the mean intensity value of a selected color (e.g., blue or brownish-red) is determined from the pixels in the image… Referring now to FIG. 7, at 350 the mean intensity value for a Selected color is correlated to a calibration curve. The calibration curve is used to convert a reading in instrument units (i.e., mean intensity) to input data units (e.g., gene copy number).”, pg. 0053-0056, The imaged samples are then analyzed to determine a correction function by correlating mean intensity values of selected colors to these known scores. This process creates a calibration curve that converts scanner measurements to biological units, thus functioning as a correction function for the scanner.);
receiving, by the one or more processors, an unscored slide image depicting a portion of stained tissue, the unscored slide image captured by the scanner (Bauer, “FIG. 8 illustrates a flow diagram of a method 400 of accurately determining the Status of one or more target molecules in a biological sample. Method 400 uses the image-processing algorithm that transforms the image into a new color Space Such that the optical density of each Stain is determined in a different channel (i.e., the process performed in 340 of method 300). Method 400 is not limited to this approach, and other alternative image processing and image analysis algorithms can be used. Method 400 provides an accurate means of determining the number of target molecules, Such as the number of gene copies, without manual counting… At 420 an optical image of the sample on the microscope Slide is generated using a computer-aided image analysis System Such as apparatus 10 as described in reference to FIG. 1.”, pg. 6, paragraphs 0058 and 0060, see Fig. 8, Once the calibration curve is established, additional images of unscored tissue samples are obtained.); and
presenting, by the one or more processors in a graphical user interface of an application executing on the one or more processors, a score for the unscored slide image generated using the correction function for the scanner (Bauer, “At any time, however, a clinician may view and/or manipulate the digital image of any given Slide for the inspection and analysis of any given Specimen.”, pg. 5, paragraph 0042, lines 5-8, “At 440 the optical density of a selected color (e.g., blue or brown) is determined from the pixels in the image… At 450, the optical density data for each color dye is compared to the color-specific calibration curve generated in method 300 (see also FIG. 9). For example, optical density data for the brownish-red dye is compared to a calibration curve that converts the optical density data to numbers of gene copies. The optical density data for the brown dye is compared to a calibration curve that converts the optical density data to the amount of protein target present.”, pg. 6 and 7, paragraphs 0062-0063, see Fig. 2, computer 22 and monitor 27, These additional images are then analyzed to determine optical density values which are then converted to scores, such as number of gene copies, using the established calibration curve. The results are presented to users through the systems computer 22 and monitor 27 to reduce the need for manual counting.).
Regarding claim 2, Bauer teaches the method of claim 1, further comprising determining, by the one or more processors, the score for the unscored slide image based on the correction function (Bauer, “At 440 the optical density of a selected color (e.g., blue or brown) is determined from the pixels in the image… At 450, the optical density data for each color dye is compared to the color-specific calibration curve generated in method 300 (see also FIG. 9). For example, optical density data for the brownish-red dye is compared to a calibration curve that converts the optical density data to numbers of gene copies. The optical density data for the brown dye is compared to a calibration curve that converts the optical density data to the amount of protein target present.”, pg. 6 and 7, paragraphs 0062-0063, The score is determined via a conversion of optical density data using the calibration curve, this is determining the score based on the correction function.).
Regarding claim 3, Bauer teaches the method of claim 1, wherein each of the plurality of calibration images are selected to correspond to a respective score value (Bauer, “After the pixels outside of the selected region and outside the selected color threshold have been masked, the image including the remaining pixels is scored. The image is scored by measuring the color value of a pixel that is the complement of the selected color.”, pg. 6, paragraph 0054, lines 1-5).
Regarding claim 4, Bauer teaches the method of claim 1, wherein determining the correction function comprises performing a color calibration process (Bauer, “At 340 the mean intensity value of a selected color (e.g., blue or brownish-red) is determined from the pixels in the image. In one embodiment, the mean intensity value is determined using color Space transforms to build masks, as described in U.S. Pat. No. 6,697.509 (the disclosure of which is incorporated herein)… Referring now to FIG. 7, at 350 the mean intensity value for a Selected color is correlated to a calibration curve.”, see pg. 6, paragraphs 0053-0056, The image is processed by applying color thresholds to calculate mean intensity values, which are then used to make correlations with the calibration curve. This process can reasonably be considered a color calibration process. ).
Regarding claim 5, Bauer teaches the method of claim 4, wherein the color calibration process comprises calculating, by the one or more processors, a color correction matrix that is applied to the unscored slide image (Bauer, “During image acquisition and processing, an algorithm of the disclosure is used to quantify a color in a Sample comprising multiple colors (e.g., one color that stains proteins by IHC and another that stain polynucleotides by CISH). Typically a series of control slides (e.g., 2 or more control slides) comprising a single color (e.g., a single stain rendering a color precipitate) will be imaged by the System (see, FIG. 9 at 1000). A measure of a color channel value in a plurality of pixels comprising a Single color of interest is made (1000). This information defines a vector for each of the plurality of control Samples, wherein each vector comprises an average of each color channel value present in the control (FIG. 9 at 1200). This information is then used to define a control matrix comprising each of the averages for each of the color channels (1300). A conversion matrix is then generated comprising the inverse of the control matrix (1400). Once the control measurements are made, the system then measures color channel Values in an image of an experimental Sample comprising a plurality of colors of interest (1500), each of the pixels comprising a plurality of
color channels. The amount of a particular color in the experimental Sample can then be calculated by converting the channel values in the experimental Sample using the conversion matrix (1600).”, pgs. 7 and 8, paragraph 0071, lines 1-24).
Regarding claim 6, Bauer teaches the method of claim 1, wherein determining the correction function comprises determining, by the one or more processors, one or more color thresholds corresponding to the plurality of calibration images (Bauer, “At 340 the mean intensity value of a selected color (e.g., blue or brownish-red) is determined from the pixels in the image. In one embodiment, the mean intensity value is determined using color space transforms to build masks, as described in U.S. Pat. No. 6,697.509 (the disclosure of which is incorporated herein). In this method, a user Selects, e.g., with a pointing device such as a mouse, a region of the optical image of the Sample to process. The pixels outside the selected region are masked. A second mask is built for the selected color using predetermined color thresholds to further differentiate the selected color. Pixels that fall outside the color threshold corresponding to the Selected color are masked.”, pg. 6, paragraph 0053).
Regarding claim 10, Bauer teaches the method of claim 1, wherein the plurality of calibration images each correspond to a respective HER2 score (Bauer, “A breast tissue sample (either the same or different) may be stained for HER2/neu protein.”, pg. 5, paragraph 0044, lines 1-2, “At box 310 a user, e.g., a pathologist or technician, selects and executes a Standard method to generate reference input data for a calibration curve. A calibration curve for CISH includes reference input data for the number of copies of a target gene. In one example, reference input data for CISH is obtained using a plurality of cultured cell lines each containing a different copy number of the target gene (e.g., the HER2/neu gene).”, pg. 0049, lines 1-8).
Claim 11 corresponds to claim 1, additionally reciting a system comprising one or more processors coupled to a non-transitory memory to execute the steps according to claim 1. Bauer teaches the addition of a system comprising one or more processors coupled to a non-transitory memory (Bauer, “A computer Subsystem comprises a computer 22 having at least one System processor 23, and a communications modem 29. The computer subsystem further includes a computer/image monitor 27 and other external peripherals including storage device 21, a pointing device, Such as a track ball or mouse device 30, a user input device, such as a touch Screen, keyboard, or voice recognition unit 28 and color printer 35.”, pg. 3, paragraph 0032, lines 9-16) to execute the steps according to claim 1. As indicated in the analysis of claim 1, Bauer teaches all the limitations according to claim 1. Therefore, claim 11 is rejected for the same reason as claim 1.
Claims 12, 13, 14, 15, 16, and 20 correspond to claims 2, 3, 4, 5, 6, and 10, respectively, additionally reciting a system comprising one or more processors coupled to a non-transitory memory to execute the steps according to claims 2, 3, 4, 5, 6, and 10. Bauer teaches the addition of a system comprising one or more processors coupled to a non-transitory memory (see analysis of claim 11) to execute the steps according to claims 2, 3, 4, 5, 6, and 10. As indicated in the analysis of claims 2, 3, 4, 5, 6, and 10, Bauer teaches all the limitations according to claim 2, 3, 4, 5, 6, and 10, respectively. Therefore, claims 12, 13, 14, 15, 16, and 20 are rejected for the same reason as claims 2, 3, 4, 5, 6, and 10.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Bauer et al. (US 20060041385 A1) in view of Jain et al. (US 20210193323 A1), (hereinafter Jain).
Regarding claim 7, Bauer teaches the method of claim 1. Bauer does not teach further comprising annotating, by the one or more processors, the unscored slide image based on the correction function.
However, Jain teaches further comprising annotating, by the one or more processors, the unscored slide image based on the correction function (Jain, “In one embodiment, the method of further includes generating a biomarker slide based on the identified biomarkers; generating a patient image slide annotated with the regions of interest; and co-registering the biomarker slide and the annotated patient tissue image slide.”, pg. 2, paragraph 0015, “FIG. 11 illustrates an H&E image slide 1102 annotated or co-registered with patch-level morphological patterns, using the described embodiments. Biomarker slides can be developed for the annotated patch-level morphological patterns and for regions that have obtained a high patch-level score, as an example, indicating biomarkers predictive of patient response and/or outcome.”, pg. 12, paragraph 0107, lines 1-7, see Fig. 11).
Bauer teaches generating a score for unscored slide images using a calibration curve that converts optical density measurements to biological values (Bauer, pg. 6 and 7, paragraphs 0062-0063). Bauer does not teach annotating the images with these values. Jain teaches annotating slide images with identified morphological patterns (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Bauer to include image annotations as taught by Jain (Jain, pg. 12, paragraph 0107, lines 1-7, see Fig. 11). The motivation for doing so would have been to provide a visualization of qualitative and quantitative results, thereby improving diagnostic efficiency. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Bauer with Jain to obtain the invention according to claim 7.
Claim 17 corresponds to claim 7, additionally reciting a system comprising one or more processors coupled to a non-transitory memory to execute the steps according to claim 7. Bauer in view of Jain teaches the addition of a system comprising one or more processors coupled to a non-transitory memory (see analysis of claim 11) to execute the steps according to claim 7. As indicated in the analysis of claim 7, Bauer in view of Jain teaches all the limitations according to claim 7. Therefore, claim 17 is rejected for the same reasons of obviousness as claim 7.
Claims 8-9 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Bauer et al. (US 20060041385 A1) in view of Christiansen et al. (US 20080309929 A1), (hereinafter Christiansen).
Regarding claim 8, Bauer teaches the method of claim 1. Bauer does not teach wherein determining the correction function comprises determining one or more intensity values over a multiple day period.
However, Christiansen teaches wherein determining the correction function comprises determining one or more intensity values over a multiple day period (Christiansen, “Individual calibration cubes may have intrinsic variations due to material differences that are preferably accounted for in order to normalize quantitative results obtained across instruments.”, pg. 7, paragraph 0071, lines 1-4, “To determine temporal effects, light intensity measurements were measured for each cube using the same instrument over a two-day period. A minimum of ten measurements were taken through each cube on each day (sum of pixel intensities in the captured image). The results are shown graphically in FIG.5 as an average total intensity on a scale of 0 to 120,000.”, pg. 7, paragraph 0073, lines 1-7).
Bauer teaches determining mean intensity values for select colors of stains from slide images to generate a calibration curve (Bauer, paragraphs 0053-0055). Bauer does not teach determining intensity values over a multiple day period. Christiansen teaches monitoring light source intensity over multiple days to establish calibration parameters (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the calibration curve generation of Bauer to include determining intensity values over a multiple day period as taught by Christiansen (Christiansen, pg. 7, paragraph 0073, lines 1-7). The motivation for doing so would have been to account for light intensity fluctuations over time, thereby improving the accuracy of the calibration curve generation. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Bauer with Christiansen to obtain the invention according to claim 8.
Regarding claim 9, Bauer teaches the method of claim 8. Claim 8 corresponds to claim 1, but differs in that the steps for scoring an unscored slide image are applied to a second scanner using a second unscored slide image during a second time period and applying a second correction function to the second scanner. Bauer does not teach applying this process to a second scanner.
However, Christiansen teaches applying similar calibration processes to two scanners to standardize output data. Specifically, Christiansen teaches determining separate correction factors for each microscope instrument and applying a standardization for the factors to normalize results across systems (Christiansen, “The system also includes a calibration device configured to redirect a standardized sample of the illumination Source to the detector. In at least some embodiments a system processor is configured to determine a correction factor for a given microscope… Alternatively or in addition, the system processor is configured with instructions for using the correction factor to correct detected images.”, pgs. 2 and 3, paragraph 0030, lines 1-14, “An equation provided below (EQ. 1) was used to standardize results obtained for each system 100. The calibration cube correction factor (CC) and machine intrinsic factors (OP) were determined for each system and stored for later use in manipulating test results. Next, standard quantitative results (specimen quantitative score raw) were obtained for a particular specimen, or sample under test.”, pg. 10, paragraph 0104, lines 1-7, see Tables 2 and 3). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the scanner calibration process of Bauer to be applied to a second scanner as taught by Christiansen (Christiansen, pg. 10, paragraphs 0103-0106). The motivation for doing so would have been to remove variability in quantitative results across different microscope scanning systems (as suggested by Christiansen, “Such calibration is useful to remove from any quantitative results, variability in intensity of the illumination Source within the same microscope system, as may occur over time, and between quantitative results obtained using different microscope systems and/or different illumination sources.”, pg. 3, paragraph 0030, lines 14-19). The combination of Bauer in view of Christiansen would apply scoring for unscored slide images by calculating a calibration curve independently for each scanner. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Bauer with Christiansen to obtain the invention according to claim 9.
Claims 18 and 19 correspond to claims 8 and 9, respectively, additionally reciting a system comprising one or more processors coupled to a non-transitory memory to execute the steps according to claims 8 and 9. Bauer teaches the addition of a system comprising one or more processors coupled to a non-transitory memory (see analysis of claim 11) to execute the steps according to claims 8 and 9, respectively. As indicated in the analysis of claims 8 and 9, Bauer teaches all the limitations according to claim 8 and 9, respectively. Therefore, claims 18 and 19 are rejected for the same reasons of obviousness as claims 8 and 9.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CONNOR LEVI HANSEN whose telephone number is (703)756-5533. The examiner can normally be reached Monday-Friday 9:00-5:00 (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CONNOR L HANSEN/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights