DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . The Amendment filed 23 February 2026 (hereinafter “the Amendment”) has been entered and considered. Claims 1, 7-8, 11, 15, 17, and 18 have been amended. Claims 2, 4, 12, and 14 have been canceled. Claims 1, 3, 5-11, 13, and 15-20, all the claims pending in the application, are rejected. All new grounds of rejection set forth in the present action were necessitated by Applicant’s claim amendments; accordingly, this action is made final.
Response to Amendment
Claim Objections
In view of amendments to claims 7, 15, and 17-18, the claim objections are withdrawn.
Claim Interpretation – 35 USC § 112(f)
Independent claim 1 has been amended to modify the functional language which previously invoked 35 USC § 112(f) with a structural modifier. Accordingly, the claims are no longer interpreted as invoking 35 USC § 112(f).
Claim Rejections - 35 USC § 101
Independent claim 1 has been amended to include the limitations of claim 4 (including intervening claim 2) which was not rejected under 101 for the reasons discussed on pages 10-11 of the previous action and reproduced by Applicant on pages 6-7 of the Amendment. Independent claim 11 has similarly been amended to include the limitations of claim 14 (including intervening claim 12) which was not rejected under 101 for the same reasons. Accordingly, the claims recite eligible subject matter, and the rejections under 35 USC 101 are withdrawn.
Prior Art Rejections
Independent claim 1 has been amended to include the limitations of claim 4 (including intervening claim 2) , and independent claim 11 has similarly been amended to include the limitations of claim 14 (including intervening claim 12). Each of independent claims 1 and 11 have been further amended to require “the labeled training images selected based on usability for training”. On pages 8-9, Applicant contends that Schmolze, Elliott Range, and Stumpe fail to teach or suggest the newly added limitation of the independent claims. The Examiner respectfully disagrees and submits that Elliott Range teaches the limitation in question.
Elliott Range is directed to evaluating “thyroid fine-needle aspiration biopsy (FNAB)” cytopathology images using a screening machine-learning algorithm MLA based on a “convolutional neural network” used “to identify follicular groups” in images (See “Materials and Methods” section and Fig. 1). Elliott Range discloses a plurality of “exclusion criteria” for the “final cohort of 908 WSIs from 659 patients” that is split “into a training set of 799 WSIs and a test set of 109 consecutive FNABs” (“Results” section). For example, final pathology results with “uncertain malignant potential were excluded because cases with this diagnosis cannot be placed in a benign or malignant category” (“Materials and Methods” Section). Also, whole slide images that exhibited “poor focus or incomplete scanning with noticeable portions of the slide missing” were excluded from the cohort of images used for training (“Results” section).
Here, Elliott Range expressly discloses that certain images were excluded from the annotated training dataset as unusable for training because of the uncertainty of the ground truth label (malignant or benign), the lack of focus in the image, or missing portions of image data, for example. That is, Elliott Range does indeed teach that “the labeled training images [are] selected based on usability for training”, contrary to Applicant’s assertions.
In view of the foregoing, the prior art rejections of the independent claims based on the teachings of Schmolze, Elliott Range, and Stumpe are maintained.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3 and 15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 3 is dependent on now-canceled claim 2, thus rendering the scope of the claim unclear. In view of the previous dependency chain of claim 3, the claim will be interpreted as being dependent on claim 1. The Examiner recommends amending the claim accordingly.
Claim 15 is dependent on now-canceled claim 12, thus rendering the scope of the claim unclear. In view of the previous dependency chain of claim 15, the claim will be interpreted as being dependent on claim 11. The Examiner recommends amending the claim accordingly.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3, 5, 9-11, 13, 15 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over “An Automatable Method for Determining Adequacy of Thyroid Fine-Needle Aspiration Samples” by Schmolze et al. (hereinafter “Schmolze”) in view of “Application of Machine Learning Algorithm to Predict Malignancy in Thyroid Cytopathology” by Elliott Range et al. (hereinafter “Elliott Range”) and further in view of U.S. Patent Application Publication No. 2021/0064845 to Stumpe et al. (hereinafter “Stumpe”).
As to independent claim 1, Schmolze discloses a system for evaluating a cell sample (Abstract discloses that Schmolze is directed to “assessing for adequacy” an “entire aspirated sample” from a “cytologic evaluation via fine-needle aspiration” of “thyroid nodules”), comprising: a specimen slide scanner configured to acquire images of an entire surface of a microscope slide upon which the cell sample is mounted (the section entitled “Design” discloses that the sample is “placed on a standard-sized microscope slide” and “imaged at x10 magnification using a noninverted Nikon Eclipse E800 epifluorescence microscope equipped with a Spot RT3 camera”; the same section discloses that “overlapping images were acquired and then digitally stitched to yield a single image” in order to “capture the entire 4-mm sample”); a computer-based image analyzer configured to identify one or more follicular clusters within each of the images acquired by the specimen slide scanner (the section entitled “Design” discloses “automated computer scoring” of the samples by an “image analysis algorithm…developed using the R programming language” which identifies “groups of closely clustered nuclei” of “thyroid follicular cells”, wherein the groups are then “enumerated”; see Figs. 1A-G); and a computer-based evaluation subsystem configured to (i) compare a number of follicular clusters identified by the computer-based image analyzer to an adequacy threshold, and (ii) present an adequacy notification to a user when the number of the follicular clusters identified by the computer-based image analyzer exceeds the adequacy threshold (the section entitled “Design” in Schmolze discloses that “a sample was deemed adequate if there were at least 6 groups” enumerated by the image analysis algorithm of the “automated computer scoring” system; the Table set forth in the “Results and Conclusions” shows a comparison of “the adequacy assessments rendered on the 11 test cases by the cytopathologist and the automated algorithm”, wherein the adequacy notification must have been displayed to the person that aggregated these results; notably, the computer which performs the “automated computer scoring” necessarily includes a monitor for displaying such results).
Schmolze does not expressly disclose that the computer-based image analyzer comprises a neural network that is trained using training images labeled, the labeled training images selected based on usability for training.
Elliott Range, like Schmolze, is directed to evaluating “thyroid fine-needle aspiration biopsy (FNAB)” cytopathology images using computer-based techniques (Abstract). In particular, Elliott Range discloses a screening machine-learning algorithm MLA based on a “convolutional neural network” used “to identify follicular groups” in images, wherein the screening CNN is trained using a “training set” with “annotated ROIs containing follicular cells” (See “Materials and Methods” section and Fig. 1). Elliott Range discloses a plurality of “exclusion criteria” for the “final cohort of 908 WSIs from 659 patients” that is split “into a training set of 799 WSIs and a test set of 109 consecutive FNABs” (“Results” section). For example, final pathology results with “uncertain malignant potential were excluded because cases with this diagnosis cannot be placed in a benign or malignant category” (“Materials and Methods” Section). Also, whole slide images that exhibited “poor focus or incomplete scanning with noticeable portions of the slide missing” were excluded from the cohort of images used for training (“Results” section).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to replace the aspects of Schmolze’s “the image analysis algorithm” which identifies the follicular groups with Elliott Range’s “convolutional neural network” which is used “to identify follicular groups” in images and is trained using images that passed certain exclusion/usability criteria, to arrive at the claimed invention discussed above. Such a modification is the result of simple substitution of one known element for another producing a predictable result. More specifically, Schmolze’s image analysis techniques and Elliott Range’s CNN perform the same general and predictable function, the predictable function being identifying follicular groups in thyroid cytology images. Since each individual element and its function are shown in the prior art, albeit shown in separate references, the difference between the claimed subject matter and the prior art rests not on any individual element or function but in the very combination itself - that is in the substitution of Schmolze’s image analysis techniques by replacing them with Elliott Range’s CNN. Thus, the simple substitution of one known element for another producing a predictable result renders the claim obvious. It is predictable that a CNN would identify objects of interest more accurately than classic image processing techniques. It is also predictable that requiring the training images to meet certain exclusion criteria would similarly contribute to accurate prediction by the CNN.
Schmolze as modified by Elliott Range does not expressly disclose that the cell sample is an unstained cell sample or that the training images are labeled based on corresponding stained images.
Stumpe, like Elliott Range, is directed to a trained “pattern recognizer” comprising a “convolutional neural network” trained to “identify regions of interest” in cell images (Abstract and [0050]). Specifically, Stumpe discloses that the CNN identifies the ROIs in unstained images, wherein the CNN is trained using an “annotation” on a corresponding stained image (Abstract and Fig. 2).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the Elliott Range’s CNN to be trained on images annotated based on corresponding stained images such that the CNN identifies ROIs in unstained images, as taught by Stumpe, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have saved time and effort that might have otherwise been required if the CNN required the images to be stained.
As to claim 3, Schmolze as modified above further teaches that the neural network is a convolutional neural network (“Materials and Methods” section and Fig. 1 of Elliott Range discloses a screening machine-learning algorithm MLA based on a “convolutional neural network” used “to identify follicular groups” in images; the reasons for combining the references are the same as those discussed above in conjunction with claim 1).
As to claim 5, Schmolze as modified above further teaches that the cell sample is a thyroid fine needle aspiration (FNA) specimen (Abstract discloses that Schmolze is directed to “assessing for adequacy” an “entire aspirated sample” from a “cytologic evaluation via fine-needle aspiration” of “thyroid nodules”).
As to claim 9, Schmolze as modified above further teaches that the adequacy threshold is six follicular clusters, such that at least six follicular clusters are required to determine that the cell sample is diagnosable (the section entitled “Design” in Schmolze discloses that “a sample was deemed adequate if there were at least 6 groups” of “closely clustered nuclei” of “thyroid follicular cells” enumerated by the image analysis algorithm).
As to claim 10, Schmolze does not expressly disclose a post-processor configured to distinguish between a sample image that is suitable for training purposes and a sample image that is not suitable for training purposes.
Elliott Range, like Schmolze, is directed to evaluating “thyroid fine-needle aspiration biopsy (FNAB)” cytopathology images using computer-based techniques (Abstract). In particular, Elliott Range discloses a screening machine-learning algorithm MLA based on a “convolutional neural network” used “to identify follicular groups” in images (See “Materials and Methods” section and Fig. 1). Elliott Range further discloses that whole slide images that exhibited “poor focus or incomplete scanning with noticeable portions of the slide missing” were excluded from the cohort of images used for training (“Results” section).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Schmolze to distinguish suitable slide images by excluding slide images that exhibit poor focus or incomplete portions from the training set of images, as taught by Elliott Range, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have increased the accuracy of recognition by virtue of training the CNN only on high quality images.
Independent claim 11 recites a method comprising steps performed by the system recited in independent claim 1. Accordingly, claim 11 is rejected for reasons analogous to those discussed above in conjunction with claim 1.
Claims 13, 15, and 19-20 recite features nearly identical to those recited in claims 3, 5, and 9-10, respectively. Accordingly, claims 13, 15, and 19-20 are rejected for reasons analogous to those discussed above in conjunction with claims 3, 5, and 9-10, respectively.
Claims 6-8 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Schmolze in view of the Elliott Range and Stumpe and further in view of U.S. Patent Application Publication No. 2020/0209600 to Chen et al. (hereinafter “Chen”).
As to claim 6, Schmolze as modified above further teaches that the specimen slide scanner further comprises a microscope (the section entitled “Design” of Schmolze discloses that the sample is “placed on a standard-sized microscope slide” and “imaged at x10 magnification using a noninverted Nikon Eclipse E800 epifluorescence microscope”). Although an ordinarily skilled artisan would recognize that the E800 has a mechanical stage, Schmolze does not expressly disclose that the specimen slide scanner further comprises a mechanical stage configured to be movable in at least two dimensions with respect to the microscope, and a motor controller configured to drive motors coupled to the mechanical stage to move the mechanical stage.
Chen, like Schmolze, is directed to a scanning system for capturing microscopy images of glass slides (Abstract and [0057-0077]). Chen discloses that the scanning system includes a “movable stage 580” which is a “linear-motor-based X-Y stage”, wherein “the relative positions of the stage 580 and the objective lens 600 in X, Y, and/or Z axes are coordinated and controlled in a closed-loop manner using motion controller 570 under the control of the processor 555” ([0068-0070]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the proposed combination of Schmolze, Elliott Range, and Stumpe to include with the scanning system a mechanical stage which moves in the X-Y directions relative to the lens of the microscope and a processor that coordinates the motors to move the movable stage, as taught by Chen, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. It is predictable that the proposed modification would have reduced the effort required by the technician capturing the images while also increasing the precision of the relative motion between the slide and the microscope.
As to claim 7, Schmolze as modified above further teaches that the specimen slide scanner conveys images to a recording device configured to receive the images and store the images in storage media (“Design” Section of Schmolze discloses that the sample is “placed on a standard-sized microscope slide” and “imaged at x10 magnification using a noninverted Nikon Eclipse E800 epifluorescence microscope equipped with a Spot RT3 camera”; the same section discloses that “overlapping images were acquired and then digitally stitched to yield a single image” in order to “capture the entire 4-mm sample”, wherein the images must be stored in some media in order to perform such stitching; furthermore, [0073] of Chen discloses images captured by the microscopy camera are stored in memory 565).
As to claim 8, Schmolze as modified by Chen further teaches that a controller coupled to the recording device and the motor controller facilitates moving the mechanical stage with respect to the specimen slide scanner ([0068-0070] of Chen discloses that the scanning system includes a “movable stage 580” which is a “linear-motor-based X-Y stage”, wherein “the relative positions of the stage 580 and the objective lens 600 in X, Y, and/or Z axes are coordinated and controlled in a closed-loop manner using motion controller 570 under the control of the processor 555”), and storing images of a specimen slide mounted to the mechanical stage as the mechanical slide steps through multiple locations of the slide in a field of view of the specimen slide scanner (“Design” Section of Schmolze discloses that the sample is “placed on a standard-sized microscope slide” and “imaged at x10 magnification using a noninverted Nikon Eclipse E800 epifluorescence microscope equipped with a Spot RT3 camera”; the same section discloses that “overlapping images were acquired and then digitally stitched to yield a single image” in order to “capture the entire 4-mm sample”, wherein the images must be stored in some media in order to perform such stitching; furthermore, [0073] of Chen discloses images captured by the microscopy camera are stored in memory 565; the reasons for combining the references are the same as those discussed above in conjunction with claim 6).
Claims 16-18 recite features nearly identical to those recited in claims 6-8, respectively. Accordingly, claims 16-18 are rejected for reasons analogous to those discussed above in conjunction with claims 6-8, respectively.
Pertinent Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Iwase (U.S. Patent Application Publication No. 2021/0390696) and Lee (U.S. Patent Application Publication No. 2020/0175420) each contemplate removing images that lack the quality that might make them suitable for image diagnosis from the training dataset.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEAN M CONNER whose telephone number is (571)272-1486. The examiner can normally be reached 10 AM - 6 PM Monday through Friday, and some Saturday afternoons.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Greg Morse can be reached at (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEAN M CONNER/Primary Examiner, Art Unit 2663