Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9 and 12-20 are rejected under 35 U.S.C. 103 as being unpatentable over Krishnan et al. (US 20120172700).
Regarding claims 1, 19 and 20 disclose a computer implemented method/a system a computer program product tangibly embodied in a non-transitory machine-readable storage medium including instruction (Krishnan, ABSTRACT, paragraphs 0029, Figs. 1-4 and 5a-5b and note: Krishnan claims 1, 32 and 38 ) comprising:
accessing, from one or more databases, a first medical image, accessing, from the one or more databases, a second medical image; (Krishnan, paragraph 0046 discloses that "medical images may be received from, for example, a storage device, a database system or an archiving system, such as a picture archiving and communication (PACS) system"),
receiving, via a graphical user interface (GUI), a selection input corresponding to selection of a first region of interest in the first medical image (Krishnan paragraph 0060 discloses that "a first user input selecting a region of interest is received);
determining a second region of interest in the second medical image based the first region of interest and the second region of interest corresponding to a same tissue; (Krishnan paragraph 0061 discloses that "once the user selection is made, the images 404a-b may be automatically created to display the selected region of interest" and Krishnan paragraph 0063 discloses that "when the user selects a particular anatomical landmark 406b in image 404b, the corresponding points (406a and c) in the other images (404a and c) may be highlighted or otherwise visually distinguished". A region of interest can also be interpreted as a "landmark" or a set of landmarks. This obviously corresponds to determining a second region of interest in the second medical image based the first region of interest and the second region of interest corresponding to a same tissue);
storing correspondence information that associates the first region of interest with the second region of interest (Krishnan paragraph 0064 discloses "linking (or registering) the medical images 404a-c and sub-images 412 based on the pre-identified anatomical landmarks", which "refers to establishing a landmark-by-landmark (or point-by-point) correspondence between multiple images or views (e.g., sagittal, transverse and coronal views. In the system of Krishnan it would be obvious to storing correspondence information that associates the first region of interest with the second region of interest i.e. registration or link)
displaying, in a first viewport of the GUI, the first medical image and a first indication of the first region of interest in the first medical image , displaying, in a second viewport of the GUI, the second medical image and a second indication of the second region of interest in the second medical image (Krishnan paragraph 0066 discloses that the "user may be navigated to the selected region of interest by presenting a reformatted view of the region of interest in the main display window, or by highlighting the region of interest within the medical images 404a-c".);
receiving a display adjustment input via to adjust the displaying of one of the first region of interest or the second region of interest in one of the first viewport or the second viewport (Krishnan paragraphs 0063 disclose the first user input is received when a user selects an anatomical landmark within the region of interest in the medical image directly. Referring to FIG. 7a, for example, when the user selects a particular anatomical landmark 406b in image 404b, the corresponding points (406a and c) in the other images (404a and c) may be highlighted or otherwise visually distinguished. In addition, the corresponding transverse (or axial) view 404c of the selected vertebra (e.g., C3) may also be displayed automatically and 0064 discloses that by "linking the different images, synchronized scrolling or navigation may be achieved". This obviously corresponds to receiving a display adjustment input via to adjust the displaying of one of the first region of interest or the second region of interest in one of the first viewport or the second viewport ); and
synchronizing, based on the display adjustment input and the correspondence information, an adjustment of the displaying of the first region of interest in the first viewport and an adjustment of the displaying of the second region of interest in the second viewport (Krishnan paragraphs 0063 disclose the first user input is received when a user selects an anatomical landmark within the region of interest in the medical image directly. Referring to FIG. 7a, for example, when the user selects a particular anatomical landmark 406b in image 404b, the corresponding points (406a and c) in the other images (404a and c) may be highlighted or otherwise visually distinguished. In addition, the corresponding transverse (or axial) view 404c of the selected vertebra (e.g., C3) may also be displayed automatically and 0064 discloses that by "linking the different images, synchronized scrolling or navigation may be achieved" and paragraph 0065 discloses that "when the user clicks on one label in one image, the regions containing the same label in the other images may be highlighted or displayed". This obviously corresponds to synchronizing, based on the display adjustment input and the correspondence information, an adjustment of the displaying of the first region of interest in the first viewport and an adjustment of the displaying of the second region of interest in the second viewport).
Therefore it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to access from one or more databases, a first medical image and second images, receive via a graphical user interface (GUI), a selection input corresponding to selection of a first region of interest in the first medical image, determine a second region of interest in the second medical image based the first region of interest and the second region of interest corresponding to a same tissue, store correspondence information that associates the first region of interest with the second region of interest, display in a first viewport of the GUI, the first medical image and a first indication of the first region of interest in the first medical image , display in a second viewport of the GUI, the second medical image and a second indication of the second region of interest in the second medical image, receive a display adjustment input via to adjust the displaying of one of the first region of interest or the second region of interest in one of the first viewport or the second viewport and synchronize based on the display adjustment input and the correspondence information, an adjustment of the displaying of the first region of interest in the first viewport and an adjustment of the displaying of the second region of interest in the second viewport as shown by DAS because such a system provides automated medical imaging system for evaluation and analysis of diagnosis of disease of patient as stated in ABSTRACT and paragraph 0003 of Krishnan.
Regarding claim 2 Krishnan disclose the first and second images comprise at least one of digital radiology or digital path ology images (Krishnan paragraph 0025 discloses that "data from any type of imaging modality including but not limited to X-Ray radiographs, MRI, CT, PET (Krishnan paragraph 0025 disclose positron emission tomography), PET-CT, SPECT, SPECT-CT, MR-PET, 3D ultrasound images or the like may also be used", paragraph 0027 also include electron microscopic and paragraph 0038 include tissue image and paragraph 0064 disclose registering images. Therefore it is obvious that Krishnan system include the first and second images comprise at least one of digital radiology or digital pathology).
Regarding claims 3-4 Krishnan disclose first medical image comprises one or more positron emission tomography (PET) images; and the second medical image comprises a digital pathology image of a specimen slide and the digital pathology image captures the specimen slide processed based on one of: Hematoxylin and Eosin (H&E) staining or Immunohistochemistry (IHC) staining ((Krishnan paragraph 0025 disclose positron emission tomography), PET-CT, SPECT, SPECT-CT, MR-PET, 3D ultrasound images or the like may also be used", paragraph 0027 also include electron microscopic and paragraph 0038 include tissue image and paragraph 0064 disclose registering images. In the system Krishnan it would be obvious to use first image positron emission tomography (PET) images; and the second medical image comprises a digital pathology image of a specimen slide and the digital pathology image captures the specimen slide processed based on one of: Hematoxylin and Eosin (H&E) staining or Immunohistochemistry (IHC) staining because these image are conventional in medical images).
Regarding claim 5, Krishnan disclose the digital pathology image is obtained based on at least one of: a fluorescent illumination, or a bright-field illumination (Krishnan paragraph 0025 disclose positron emission tomography), PET-CT, SPECT, SPECT-CT, MR-PET, 3D ultrasound images or the like may also be used", paragraph 0027 also include electron microscopic and paragraph 0038 include tissue image and paragraph 0064 disclose registering images. Krishnan discloses digital pathology and fluorescent illumination, or a bright-field illumination are conventional and well known it would be obvious to a fluorescent illumination, or a bright-field illumination ).
Regarding claim 6 Krishnan disclose the selection input comprises a selection of a plurality of first landmarks in the first viewport; and further comprises: determining first location information of the first region of interest in the first medical image based on locations of the plurality of first landmarks in the first viewport and a display scaling factor of the first medical image in the first viewport; and storing the first location information in the correspondence information (Krishnan paragraph 0060 discloses that "a first user input selecting a region of interest is received i.e. landmarks paragraph 0064 discloses that "the visualization module 202 links the images by generating a mapping table that maps the coordinates of each anatomical landmark represented in one image to the coordinates of the same anatomical landmark represented in another image" and . Krishnan in paragraph 0066 navigating selected regions of images and zoom in-out, panning and rotation To allow a (potentially manual) selection of landmarks in the viewports and paragraph 006 disclose the detection of a selection of an option would be obvious and it be obvious to display scale factor`).
Regarding claim 7 Krishnan disclose receiving a selection of a plurality of second landmarks in the second viewport; determining second location information of the second region of interest in the second medical image based on locations of the plurality of second landmarks in the second viewport and a display scaling factor of the second medical image in the second viewport; and storing the second location information in the correspondence information (Krishnan paragraph 0060 discloses that "a first user input selecting a region of interest is received i.e. landmarks paragraph 0064 discloses that "the visualization module 202 links the images by generating a mapping table that maps the coordinates of each anatomical landmark represented in one image to the coordinates of the same anatomical landmark represented in another image", Krishnan paragraph 0061 discloses that "once the user selection is made, the images 404a-b may be automatically created to display the selected region of interest" and Krishnan paragraph 0063 discloses that "when the user selects a particular anatomical landmark 406b in image 404b, the corresponding points (406a and c) in the other images (404a and c) may be highlighted or otherwise visually distinguished". A region of interest can also be interpreted as a "landmark" or a set of landmarks, Krishnan in paragraph 0066 navigating selected regions of images and zoom in-out, panning and rotation. It would be obvious receiving a selection of a plurality of second landmarks in the second viewport; determining second location information of the second region of interest in the second medical image based on locations of the plurality of second landmarks in the second viewport and a display scaling factor of the second medical image in the second viewport; and storing the second location information in the correspondence information and display scaling factor i.e. zooming factor).
Regarding claim 8, Krishnan disclose detecting, by the GUI, a selection of an option to select landmarks of corresponding regions of interest in the first viewport and the second viewport, wherein the determination that the first region of interest and the second region of interest correspond to the same tissue is based on detecting the selection of the option (Krishnan in Figs. 1-2 and 4, 5a-5b, 6 illustrates GUI, Krishnan paragraph 0060 discloses that "a first user input selecting a region of interest is received i.e. landmarks paragraph 0064 discloses that "the visualization module 202 links the images by generating a mapping table that maps the coordinates of each anatomical landmark represented in one image to the coordinates of the same anatomical landmark represented in another image", Krishnan paragraph 0061 discloses that "once the user selection is made, the images 404a-b may be automatically created to display the selected region of interest" and Krishnan paragraph 0063 discloses that "when the user selects a particular anatomical landmark 406b in image 404b, the corresponding points (406a and c) in the other images (404a and c) may be highlighted or otherwise visually distinguished". A region of interest can also be interpreted as a "landmark" or a set of landmarks. It is obvious in the system of Krishnan in Fig, 1-2, 4 and 5a-5b and disclose paragraph 0060-0064 and 0066 detecting, by the GUI, a selection of an option to select landmarks of corresponding regions of interest in the first viewport and the second viewport, wherein the determination that the first region of interest and the second region of interest correspond to the same tissue is based on detecting the selection of the option and paragraph 0064-0065 liking and registration of two regions of interest of two images)
Regarding claim 9 Krishnan disclose performing a first image processing operation on the first medical image to determine the first region of interest; performing a second image processing operation on the second medical image to determine the second region of interest; receiving the selection input as a confirmation that the first region of interest determined by the first image processing operation and the second region of interest determined by the second image processing operation correspond to the same tissue; and storing the correspondence information based on the confirmation
(Krishnan paragraphs 0060-0061, Krishnan 0060 discloses that "a first user input selecting a region of interest is received and paragraph 0061 and Krishnan paragraph 0061 discloses that "once the user selection is made, the images 404a-b may be automatically created to display the selected region of interest" and Krishnan paragraph 0063 discloses that "when the user selects a particular anatomical landmark 406b in image 404b, the corresponding points (406a and c) in the other images (404a and c) may be highlighted or otherwise visually distinguished". A region of interest can also be interpreted as a "landmark" or a set of landmarks. In the system of Krishnan it would be obvious to receiving the selection input as a confirmation that the first region of interest determined by the first image processing operation and the second region of interest determined by the second image processing operation correspond to the same tissue because when same regions of interest and areas and shape are selected in both images and by registration of both images ).
Regarding claim 12 Krishnan disclose the second image processing operation is performed by a convolutional neural network (Krishnan Fig. 2 intelligent system, paragraphs 0060, 0061 discloses that "once the user selection is made, the images 404a-b may be automatically created to display the selected region of interest" and Krishnan paragraph 0063 discloses that "when the user selects a particular anatomical landmark 406b in image 404b, the corresponding points (406a and c) in the other images (404a and c) may be highlighted or otherwise visually distinguished". A region of interest can also be interpreted as a "landmark" or a set of landmarks. Krishan paragraph 0051 and Krishnan claim 10 disclose machine learning and CNN which is machine model is conventional in the art therefore it would be to train the system of Krishnan illustrated in Figs. 1-4 and 5a-5b use the machine learning model to train such as CNN to perform second image processing operation).
Regarding claim 13 Krishnan disclose generating using a learning system, a prediction that the first region of interest and the second region of interest correspond to the same tissue; displaying in the GUI the prediction and receiving the selection input as a confirmation of the prediction (Krishnan Fig. 2 intelligent system, paragraphs 0060-0061, Krishnan 0060 discloses that "a first user input selecting a region of interest is received and paragraph 0061 and Krishnan paragraph 0061 discloses that "once the user selection is made, the images 404a-b may be automatically created to display the selected region of interest" and Krishnan paragraph 0063 discloses that "when the user selects a particular anatomical landmark 406b in image 404b, the corresponding points (406a and c) in the other images (404a and c) may be highlighted or otherwise visually distinguished". A region of interest can also be interpreted as a "landmark" or a set of landmarks. In the system of Krishnan it would be obvious to receiving the selection input as a confirmation that the first region of interest determined by the first image processing operation and the second region of interest determined by the second image processing operation correspond to the same tissue because when same regions of interest and areas and shape are selected in both images and by registration of both images. Krishnan paragraph 0051 and Krishnan claim 10 disclose machine learning and it would be obvious to train and learn using machine learning model to prediction that the first region of interest and the second region of interest correspond to the same tissue; displaying in the GUI the prediction and receiving the selection input as a confirmation of the prediction which can be based on regions of interest and landmark in two images).
Regarding claim 14 Krishnan disclose adjustment input comprises at least one of a zoom-in input, a zoom-out input, a panning input, or a rotation input (Krishnan 0064 discloses that by "linking the different images, synchronized scrolling or navigation may be achieved" and paragraph 0065 discloses that "when the user clicks on one label in one image, the regions containing the same label in the other images may be highlighted or displayed". Krishnan in paragraph 0066 navigating selected regions of images and zoom in-out, panning and rotation therefore it would be obvious to adjustment input comprises at least one of a zoom-in input, a zoom-out input, a panning input, or a rotation input).
Regarding claim 15 Krishnan synchronizing an adjustment of the displaying of the first region of interest in the first viewport and an adjustment of the displaying of the second region of interest in the second viewport is performed such that the first region of interest and the second region of interest are displayed by a same degree of magnification in, respectively, the first viewport and the second viewport (Krishnan 0064 discloses that by "linking the different images, synchronized scrolling or navigation may be achieved" and paragraph 0065 discloses that "when the user clicks on one label in one image, the regions containing the same label in the other images may be highlighted or displayed". Krishnan in paragraph 0066 navigating selected regions of images and zoom in-out, panning and rotation therefore it would be obvious to adjustment input comprises at least one of a zoom-in a zoom-out input (magnification), a panning input, or a rotation input. Furthermore it would be obvious programmed the system of Krishnan shown in Figs 1-4 and 5a-5b as highlighting of two labels and regions of interest).
Regarding claims 16 Krishnan disclose synchronizing an adjustment of the displaying of the first region of interest in the first viewport and an adjustment of the displaying of the second region of interest in the second viewport is performed such that a same portion of the first region of interest and second region of interest are displayed in, respectively, the first viewport and the second viewport (Krishnan 0064 discloses that by "linking the different images, synchronized scrolling or navigation may be achieved" and paragraph 0065 discloses that "when the user clicks on one label in one image, the regions containing the same label in the other images may be highlighted or displayed". Therefore it would be obvious in the system Krishnan synchronizing an adjustment of the displaying of the first region of interest in the first viewport and an adjustment of the displaying of the second region of interest in the second viewport is performed such that a same portion of the first region of interest and second region of interest are displayed in, respectively, the first viewport and the second viewport).
Regarding claim 17 Krishnan disclose synchronizing an adjustment of the displaying of the first region of interest in the first viewport and an adjustment of the displaying of the second region of interest in the second viewport is performed such that the first region of interest and second region of interest are rotated by a same degree in, respectively, the first viewport and the second viewport (Krishnan 0064 discloses that by "linking the different images, synchronized scrolling or navigation may be achieved" and paragraph 0065 discloses that "when the user clicks on one label in one image, the regions containing the same label in the other images may be highlighted or displayed". Krishnan in paragraph 0066 navigating selected regions of images and zoom in-out, panning and rotation therefore it would be obvious to adjustment input comprises at least one of a zoom-in input, a zoom-out input, a panning input, or a rotation input. Furthermore it would be obvious to programmed the system of Krishnan shown in Figs 1-4 and 5a-5b as highlighting of two labels and regions of interest).
Regarding 18 Krishnan disclose generating an analytics output based on the first region of interest and the second region of interest corresponding to same tissue and the analytics output comprises at least one of: a prediction of a cancer diagnosis, a verification of a prior surgical procedure, a classification of the same tissue captured in the first region of interest and in the second region of interest, or a research operation regarding a treatment (Krishnan paragraph 0061 discloses that "once the user selection is made, the images 404a-b may be automatically created to display the selected region of interest" and Krishnan paragraph 0063 discloses that "when the user selects a particular anatomical landmark 406b in image 404b, the corresponding points (406a and c) in the other images (404a and c) may be highlighted or otherwise visually distinguished". A region of interest can also be interpreted as a "landmark" or a set of landmarks and 0064 discloses that by "linking/registering the different images, synchronized scrolling or navigation may be achieved" and paragraph 0065 discloses that "when the user clicks on one label in one image, the regions containing the same label in the other images may be highlighted or displayed. In the system of Krishnan based on linking and registering two region of two images it would be obvious to generate analytic output based on the first region of interest and the second region of interest corresponding to same tissue and the analytics output comprises at least one of: a prediction of a cancer diagnosis by the expert and a classification of the same tissue captured in the first region of interest and in the second region of interest).
Claims objected
Claims 10-11 are objected as being dependent on rejected base but would be allowable over the prior art of record if written in the independent form including the limitations of the base claim and any intervening claims.
Communication
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISHRAT I SHERALI whose telephone number is (571)272-7398. The examiner can normally be reached Monday-Friday 8:00AM -5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached on 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ISHRAT I SHERALI/Primary Examiner, Art Unit 2667
ISHRAT I. SHERALI
Examiner
Art Unit 2667
Prosecution Insights