DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character “132” has been used to designate both Whole Slide Image (WSI) and GUI in Figure 1.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description:
Reference Numeral “1112” shown in Figure 11. However, Specimen 1112 should be Specimen 1124.
Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 4-6, 8-11, 14-16, and 18-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Wirch et al. (U.S. Pub. No. 2021/0407076).
Re claims 1 and 11: Wirch et al. disclose an apparatus/method for visualizing digitized slides (i.e., “WSI processing system”, Paragraph [0092]), the apparatus comprising:
at least a processor (i.e., “Central processor unit 84”, Paragraph [0098]); and
a memory (i.e., “Memory 90”, Paragraph [0096]) communicatively connected to the at least a processor, the memory containing instructions configuring the at least a processor to:
retrieve/retrieving, using at least a processor (i.e., “Central processor unit 84”, Paragraph [0098]), a digitized slide comprising a reference serial section and one or more remaining serial sections (See for example, “At least two whole slide images (WSIs or WSI data) (101) are input into the system”, Paragraph [0032]; and “the WSI is divided by a “grid” of areas”, Paragraph [0078]);
determine/determining, using the at least a processor, one or more visualization components of the digitized slide (i.e., “one or more annotations are specified, through an external system (i.e., importing annotations from a third-party), through an automated process (i.e., automated tissue segmentation algorithm), or by an expert user. The annotations may take one of two forms: WSI-based or region-based”, Paragraph [0053]; and “keypoints are identified using a feature detection algorithm”, Paragraph [0076]);
generate/generating, using the at least a processor, a virtual slide corresponding to the digitized slide based on the one or more visualization components (i.e., “The viewports each display a view of a differently stained, registered WSI … The results may also be visibly presented to the user as an annotation overlayed on the slides themselves”, Paragraph [0089]);
compute/computing, using the at least a processor, one or more registration transforms for each of the one or more remaining serial sections relative to the reference serial section (i.e., “The Affine Transform Matrices (ATMs) provide a simple mathematical way to correlate any point on the first image with a point on the second image”, Paragraph [0073]; and “progressively finer ATMs can be calculated for smaller subsections of the images to improve precision, such that the WSI is divided by a “grid” of areas, each with a separate ATM”, Paragraph [0078]); and
display/displaying, using a user display (i.e., “I/O device interface 82 for connecting various input and output devices (e.g., keyboard, mouse, wheels, buttons, touch screens, displays, printers, speakers, voice controls, etc.)”, Paragraph [0096]), a visualization of the virtual slide (i.e., “display results of the above steps to the user, such as in Step 180 of FIGS. 1 & 7, using an interface”, Paragraph [0089]).
Re claims 4 and 14: Wirch et al. disclose wherein the one or more visualization components comprise one or more of a tissue section and an annotation (i.e., “one or more annotations are specified, through an external system (i.e., importing annotations from a third-party), through an automated process (i.e., automated tissue segmentation algorithm), or by an expert user. The annotations may take one of two forms: WSI-based or region-based”, Paragraph [0053]).
Re claims 5 and 15: Wirch et al. disclose wherein the at least a processor is configured to generate a plurality of aligned serial sections by: classifying the reference serial section and the one or more remaining serial sections (i.e., “normalized and otherwise preconditioned image patches may be supplied to the model. The pipeline selected for the classifier accepts patches of a predetermined size as input and results in a classification and scoring of each patch based on the classes identified during training”, Paragraph [0070]); and
aligning the one or more remaining serial sections to the reference serial section (i.e., “Using the information from the coarse ATM about the general translation and rotation of the WSIs, progressively finer ATMs can be calculated for smaller subsections of the images to improve precision, such that the WSI is divided by a “grid” of areas, each with a separate ATM. Fine ATMs provide better alignment precision than the coarse ATM, but they are only valid within a smaller subsection of the whole image”, Paragraph [0078]).
Re claims 6 and 16: Wirch et al. disclose wherein the one or more registration transforms comprises one or more transformation matrices (i.e., “Affine Transform Matrices (ATMs)”, Paragraph [0073]).
Re claims 8 and 18: Wirch et al. disclose wherein displaying the visualization of the virtual slide comprises displaying/further configured to display the visualization using a whole slide image viewer (i.e., “the user is presented with a software application for reviewing the WSIs”, Paragraph [0089]).
Re claim 9: Wirch et al. disclose wherein the whole slide image viewer may display one or more of the virtual slide and at least a user-configurable option (i.e., “if a user disagrees with the classification of a result generated by a model, the user can correct the classification by clicking a button next to the result thumbnail (e.g., false positive, incorrect classification)”, Paragraph [0091]).
Re claims 10 and 20: Wirch et al. disclose wherein the at least a processor is further configured to:
generate at least a visual indicator associated with the one or more registration transformation (i.e., “The viewports each display a view of a differently stained, registered WSI (that is, the viewports display the similar location on each WSI). The sidebar can provide various tools, but during the review of results, the sidebar presents a series of thumbnails (321) representing the detected and classified ROIs. Each result may also contain further details (beyond location) as metadata (e.g., score, classification) (323). The results may also be visibly presented to the user as an annotation overlayed on the slides themselves. In one embodiment, the results may be colored or labeled differently based on result properties such as classification score, ML algorithm with the strongest classification, or classification category (i.e., output label) in the case of n-class classification”, Paragraph [0089]); and
display, using the user display, the visual indicators (i.e., “the results may be colored or labeled differently”, Paragraph [0089]).
Re claim 19: Wirch et al. disclose wherein displaying the visualization of the virtual slide comprises displaying the visualization using a whole slide image viewer (i.e., “the user is presented with a software application for reviewing the WSIs”, Paragraph [0089]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 3, 12, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Wirch et al. in view of Remiszewski et al. (U.S. Pub. No. 2016/0110584). The teachings of Wirch et al. have been discussed above.
As to claims 2 and 12, Wirch et al. does not explicitly disclose wherein the at least a processor is further configured to determine a membership of the digitalized slide by cross referencing the digitalized slide with a central database comprising membership data associated with the digitalized slide.
Remiszewski et al. teaches a processor that is configured to (i.e., “processor 1904”, Paragraph [0308]) determine a membership of the digitalized slide by cross referencing the digitalized slide with a central database (i.e., “receiving clinical information during the annotation process 110. In an aspect, the system may receive clinical information from a medical practitioner, an electronic medical record of a patient, or other data source, such as a data repository that may include clinical data”, Paragraph [0111]) comprising membership data associated with the digitalized slide (i.e., “receiving annotation information for the IR image 108. Annotation information may include, but is not limited to, any suitable clinical data regarding the selected annotation region, such as data that may be relevant to a diagnosis, including, for example, biochemical signatures as correlated to a feature of a type of cells and/or tissues that are likely present in the sample”, Paragraph [0112]; and “visual image of the sample”, Paragraph [0116]).
Wirch et al. and Remiszewski et al. are analogous art because they are from the field of digital image processing for biological imaging.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Wirch et al. by incorporating the determination of membership of the digitalized slide by cross referencing the digitalized slide with a central database comprising membership data associated with the digitalized slide, as taught by Remiszewski et al.
The suggestion/motivation for doing so would have been to improve the classification of tissue samples by providing more information about the sample to be evaluated.
Therefore, it would have been obvious to combine Remiszewski et al. with Wirch et al. to obtain the invention as specified in claims 2 and 12.
As to claims 3 and 13, Remiszewski et al. teaches wherein the membership data is associated with at least one of a patient case or a tissue block (i.e., “receiving annotation information for the IR image 108. Annotation information may include, but is not limited to, any suitable clinical data regarding the selected annotation region, such as data that may be relevant to a diagnosis, including, for example, biochemical signatures as correlated to a feature of a type of cells and/or tissues that are likely present in the sample”, Paragraph [0112]; and “visual image of the sample”, Paragraph [0116]).
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Wirch et al. in view of El-Baz et al. (U.S. Pub. No. 2022/0406049). The teachings of Wirch et al. have been discussed above.
As to claims 7 and 17, Wirch et al. teaches wherein: a particular constituent visualization component is placed within the one or more transformation matrices (i.e., “The generation of an ATM can be achieved via identifying at least 3 matching keypoint pairs and then solving a linear system of equations using identified keypoint pairs”, Paragraph [0075]).
However, Wirch et al. does not explicitly disclose the at least a processor is configured to convert the one or more transformation matrices into a resulting matrix that takes into account an orientation angle.
El-Baz et al. teaches a processor that is configured to (i.e., Paragraph [0042]) convert the one or more transformation matrices into a resulting matrix that takes into account an orientation angle (i.e., “transformation matrix h”, Paragraph [0046]; and “Also in local registration, a constraint is added on the slope angle measured for the line connecting the matched key points in the two images. If this angle is greater than the constraint, the matched key points are removed from the set used for estimating matrix h to overcome incidents of misalignment”, Paragraph [0047]).
Wirch et al. and El-Baz et al. are analogous art because they are from the field of digital image processing for biological imaging.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Wirch et al. by incorporating the converting of the one or more transformation matrices into a resulting matrix that takes into account an orientation angle, as taught by El-Baz et al.
The suggestion/motivation for doing so would have been to overcome incidents of misalignment when registering the serial sections.
Therefore, it would have been obvious to combine El-Baz et al. with Wirch et al. to obtain the invention as specified in claims 7 and 17.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSE M TORRES whose telephone number is (571)270-1356. The examiner can normally be reached Monday thru Friday; 10:00 AM to 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSE M TORRES/Examiner, Art Unit 2664 09/29/2026
Prosecution Insights