DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-16 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claim(s) 1-4 and 6-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ofuji et al (US20090086891) in view of Gustafson (US20110125526).
Regarding claims 1 and 15-16, Ofuji teaches an image processing apparatus comprising: at least one processor, wherein the processor:
(Ofuji, Fig. 1)
acquires a plurality of breast images;
(Ofuji, Fig. 3; “The image storage means 31 sends a browse request to the image management server 4 and obtains mammograms P that are necessary to perform image-reading. The image storage means 31 obtains and stores a plurality of mammograms P”, [0060]; acquiring a plurality of breast images (mammograms P))
displays, according to any one mammary gland volume type among a plurality of mammary gland volume types assigned to each of the plurality of acquired breast images based on a mammary gland volume of a breast, a plurality of selection candidate images corresponding to each of the plurality of breast images for each mammary gland volume type; and
(Ofuji, Fig. 3; “a classification means for classifying the plurality of mammograms into a plurality of breast types based on the mammary-gland content rates thereof, obtained by the mammary-gland content rate calculation means; and a second breast image display means for displaying the plurality of mammograms separately based on their breast types”, [0024-0025]; “the plurality of breast types are a fatty type, a mammary-gland-scattered type, a heterogeneously-dense type and a dense type”, [0032]; “It is possible to classify the mammograms P into the four breast types by applying three threshold values for classification”, [0073]; Fig. 6, “The classification means 36 classifies the plurality of mammograms P into a plurality of breast types based on the mammary-gland content rates of the mammograms ... The second breast image display means 37 displays the mammograms P separately on the display of the display device 35 based on their breast types, into which the mammograms P have been classified”, [0078]; “Users can select the sort order of the image list from the ascending order based on mammary-gland content rates and the descending order based on mammary-gland content rates”, [0063]; assigning a mammary gland volume type (breast type) based on mammary gland volume (content rate) and displaying the images (selection candidate images) according to these types; Fig. 8, displaying the mammograms images in an order based on mammary-gland content rates thereof by switching the mammograms; the sorted mammograms images in Fig. 8 => “selection candidate images”)
Ofuji does not expressly disclose but Gustafson teaches:
extracts a breast image selected for each mammary gland volume type from the plurality of breast images based on the plurality of selection candidate images displayed for each of the mammary gland volume types.
(Gustafson, “storing a selected region of the first mammographic image as a second image”, [0012]; “a clipping tool with which a portion of the mammographic image ... can be selected as a second image, the second image displayable on at least one of the plurality of electronic displays as a subset of the mammographic image”, [0013]; “The system stores lower resolution clippings, or thumbnail images, for pathological images, reports, and abnormalities found and optionally categorized, by radiologists or CAD products at a facility that have been entered into a mammography information system”, [0030]; “displaying thumbnail images of at least one of the first mammographic image”, [claim 14]; extract thumbnail images from mammographic images; the thumbnail images may be displayed also; Gustafson teaches a system for selecting (clipping) and storing (extracting) specific breast images or regions from a displayed gallery (plurality of candidate images) for reporting or database storage)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the image selection and clipping mechanisms of Gustafson into the density-based classification and display of Ofuji in order to allow a user to select and extract specific representative breast images (e.g., thumbnail images) from the density-sorted groups for reports or certification. The combination of Ofuji and Gustafson also teaches other enhanced capabilities.
Regarding claim 2, the combination of Ofuji and Gustafson teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 1, wherein the selection candidate image is a thumbnail image of a corresponding breast image.
(Gustafson, “The system stores lower resolution clippings, or thumbnail images, for pathological images, reports, and abnormalities found and optionally categorized, by radiologists or CAD products at a facility that have been entered into a mammography information system”, [0030]; “displaying thumbnail images of at least one of the first mammographic image”, [claim 14])
Regarding claim 3, the combination of Ofuji and Gustafson teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 1, wherein the processor:
extracts one or more breast images from a breast image group to which the same mammary gland volume type is assigned, and
(Ofuji, “displays the mammograms P separately for each of the classified breast types... mammograms P that belong to the same breast type t are displayed at the same time on the same display”, [0078]; Gustafson, “The system stores lower resolution clippings, or thumbnail images, for pathological images, reports, and abnormalities found and optionally categorized, by radiologists or CAD products at a facility that have been entered into a mammography information system”, [0030]; “displaying thumbnail images of at least one of the first mammographic image”, [claim 14]; extract thumbnail images from mammographic images; these thumbnail image will represent the same breast types as that of the original mammographic images)
displays, side by side, breast images for comparison corresponding to each of the one or more breast images.
(Ofuji, “a plurality of mammograms are arranged and displayed on a high-resolution display device”, [0006]; “mammograms P that belong to the same breast type t are displayed at the same time on the same display”, [0078]; arranging and displaying multiple images of the same type simultaneously for the purpose of diagnosis/comparison)
Regarding claim 4, the combination of Ofuji and Gustafson teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 3, wherein the processor:
assigns an evaluation result regarding the selection to the breast image for comparison,
(Gustafson, “The detailing window 400 provides an interface for a radiologist to enter or view the detailed attributes that describe an abnormality in a selected ROI... the “5 Highly suggestive” 414 attribute indicates that a follow-up examination of the patient is necessary”, [0041]; “All of this information can be stored in a database configured to correlate all of a patent's ROI data and images”, [0042]; “When an image is associated to an abnormality the title bar 520 changes color indicating a direct association. Tapping the “+” 524 provides a mechanism to attach image to abnormality 510”, [0049]; assigning attributes (evaluation results like "Highly suggestive" or the binary status of being "associated/attached") to an image; the act of "attaching" an image to an abnormality record constitutes assigning an evaluation result regarding its selection as a relevant image)
displays the evaluation result, and
(Gustafson, “The detailing window 400 provides an interface for a radiologist to enter or view the detailed attributes”, [0041]; “displays information that can be stored as BI-RADS compatible data points”, [0042]; “When an image is associated to an abnormality the title bar 520 changes color”, [0049]; displaying the evaluation attributes (e.g., in the detailing window) and displaying the association status (via the color change in the title bar))
stores a comparison result in association with the breast image corresponding to the breast image for comparison to which the evaluation result is assigned.
(Gustafson, “processing engine configured to link the second image to the mammographic image... and to associate the second image with a corresponding region of interest”, [0013]; “All of this information can be stored in a database configured to correlate all of a patent's ROI data and images”, [0042]; storing the links, associations, and attributes (the results of the user's comparison and evaluation) in a database associated with the image)
Regarding claim 6, the combination of Ofuji and Gustafson teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 4, wherein the processor:
acquires a certification result for the breast image extracted for each mammary gland volume type, and
(Ofuji, “After the mammogram is checked by the test technologist, the mammogram is transferred to the image management server 4 through the network 5”, [0055]; Gustafson, “track positive mammography findings and correlate such findings with biopsy results”, [0008]; “As more patients are definitively diagnosed and their pathology records updated in the system, the larger the collection of abnormality images depicting a previously diagnosed and imaged condition that become available in the system”, [0030]; “allowing a comparison of additional images from a larger database or final pathology results”, [0044]; as established in the analysis of Claim 11, Ofuji's QA check and Gustafson's "pathology records" or "biopsy results" serve as certification results confirming the quality or diagnostic validity (certification) of extracted images)
performs an evaluation regarding the selection on the breast image for comparison based on the breast image associated with the certification result.
(Gustafson, “The features provided by the system can also be combined with any one of several available computer aided diagnostic (CAD) products to validate, improve, and allow simplified characterization of images... The CAD product can pre-select the ROI classifications”, [0043]; “comparing an analysis of a computer aided diagnostic tool on the plurality of tissue images with the result of the diagnosis of the tissue sample of the abnormality”, [claim 21]; “a database of thumbnail or clipped images can provide a source of investigational data that may assist a radiologist in categorizing an abnormality ... for use as a training tool”, [0048]; using a database of certified (pathology-confirmed) images as a "training tool" for CAD or for "comparison"; the processor (via CAD or comparison logic) "performs an evaluation" (e.g., pre-selecting classifications or analyzing) on the current breast image based on the data derived from these certified images)
Regarding claim 7, the combination of Ofuji and Gustafson teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 4, wherein the processor:
receives selection of the selection candidate image for each mammary gland volume type, extracts the breast image according to the received selection, and
(Gustafson, “storing a selected region of the first mammographic image as a second image”, [0014]; “a clipping tool with which a portion of the mammographic image including the at least one region of interest and displayed on at least one of the plurality of electronic displays can be selected as a second image”, [0013]; receiving a selection and extracting (clipping/storing) the image)
performs an evaluation regarding the selection on the breast image for comparison based on feature information indicating a feature of the breast image extracted in the past.
(Gustafson, “select the “Clone Prey” button 280 to review and import data from a previous examination”, [0039]; “allows a radiologist to select images based on the ... abnormality descriptors, allowing a comparison”, [0044]; “The CAD product can pre-select the ROI classifications for each abnormality detected”, [0043]; importing "data from a previous examination" (feature information from a breast image extracted in the past) using the "Clone Prey" feature or comparing against stored descriptors; the processor uses this past feature information to perform an evaluation (e.g., pre-selecting classifications or facilitating comparison) on the current breast image)
Regarding claim 8, the combination of Ofuji and Gustafson teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 3, wherein the breast image for comparison is an image larger than the selection candidate image.
(Gustafson, “The system stores lower resolution clippings, or thumbnail images”, [0030]; “The system also provides a link from the compressed image clippings 510 to the full-sized high-resolution image for the situations, such as making a diagnostic assessment, that require a radiologist to view the high-resolution image”, [0047]; “magnification button 528 brings up an individual ROI viewer 550 to allow a large view”, [0049]; using "thumbnail images" as the selection candidates (in the gallery) and "full-sized high-resolution" or "large view" images for comparison/assessment, which are larger than the thumbnails)
Regarding claim 9, the combination of Ofuji and Gustafson teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 3, wherein the processor displays the selection candidate image and the breast image for comparison on the same screen.
(Ofuji, “displaying a plurality of mammograms at the same time in the same display”, [0020]; “mammograms P that belong to the same breast type t are displayed at the same time on the same display”, [0078]; Ofuji teaches the simultaneous display of multiple images. It would be obvious to a person skilled in the art, in view of Gustafson's teaching of thumbnails and large views, to configure the "same display" of Ofuji to show the list of candidates (thumbnails) alongside the currently selected breast image for comparison (large view) to facilitate efficient workflow)
Regarding claim 10, the combination of Ofuji and Gustafson teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 3, wherein the processor displays, side by side, the breast images for comparison in which at least one of a technique or an imaging direction is the same among the breast images for comparison corresponding to each of the one or more breast images.
(Ofuji, “diagnosis is performed by comparing the displayed mammograms with each other”, [0005]; “displaying a plurality of mammograms at the same time in the same display, arranging them in ascending order or in descending order based on the mammary-gland content rates”, [0020]; displaying images simultaneously for the specific purpose of "comparing"; Gustafson, “includes both a craniocaudal (CC) view 250 and a mediolateral/oblique (ML) view 260 of both the left and right breasts of a patient”, [0034]; “description of the view is display from the image it was obtained from for example RCC (RightCranioCaudal) image”, [0050]; identifying the imaging direction (e.g., CC, ML); it would be obvious to a person skilled in the art to arrange these comparison images "side by side" based on the same technique or imaging direction (e.g., Left CC next to Right CC, or Current ML next to Prior ML) as identified in Gustafson, as this is a standard radiological hanging protocol to enable the "comparing" required by Ofuji)
Regarding claim 11, the combination of Ofuji and Gustafson teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 1, wherein the processor:
acquires a certification result for the breast image extracted for each mammary gland volume type, and
(Ofuji, “the QA-WS 2 displays the processed mammogram and the content of the attribute information on the display and prompts the test technologist to check the mammogram. After the mammogram is checked by the test technologist, the mammogram is transferred”, [0055]; Ofuji teaches a Quality Assurance Workstation (QA-WS) where a "check" is performed on the image; this "check" constitutes a certification result confirming the image is acceptable for transfer and downstream use; Gustafson also teaches associating "pathology records" or "biopsy results" with images, which acts as a certification of the diagnosis)
specifies and presents a selection target breast image based on the breast image associated with the certification result.
(Gustafson, “allows a radiologist to select images based on... abnormality descriptors, allowing a comparison of additional images from a larger database”, [0044]; “database of thumbnail or clipped images can provide a source of investigational data... or for use as a training tool”, [0048]; “the capability of importing any ROI from a patient's previous examination”, [0039]; accessing and presenting stored, validated (certified) images to assist in the analysis or selection of current images (functioning as a "training tool" or for "comparison"); it would be obvious to modify the display system of Ofuji to utilize the "checked" (certified) status of images (as taught by Ofuji) in conjunction with the search and comparison logic of Gustafson to automatically specify and present new selection candidates that match the quality or characteristics of those previously certified images, thereby assisting the user in selecting consistent images.)
Regarding claim 12, the combination of Ofuji and Gustafson teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 1, wherein the processor:
receives selection of the selection candidate image for each mammary gland volume type,
(Ofuji, “displaying the plurality of mammograms separately based on their breast types”, [0025]; the context of displaying candidate images is organized by mammary gland volume type; Gustafson, “The activation of the “Roi Gallery” button 290, shown in FIG. 3, causes the ROI Gallery 500 to be presented to the user. The image clippings 510 can be selected”, [0045]; Gustafson teaches a user interface ("ROI Gallery") where a user can select specific candidate images ("image clippings") from a presented group; it would be obvious to apply the selection interface of Gustafson to the type-sorted display of Ofuji)
extracts the breast image according to the received selection, and
(Gustafson, "storing a selected region of the first mammographic image as a second image ", [0014]; " a clipping tool with which a portion of the mammographic image ... can be selected as a second image ... a processing engine configured to ... store the second image in an image database "; Gustafson teaches "extracting" the image by using a "clipping tool" to select a portion of the image and storing that selection as a distinct second image in the database)
specifies and presents a selection target breast image based on feature information indicating a feature of the breast image extracted in the past.
(Gustafson, "importing any ROI from a patient's previous examination... elect the “Clone Prey” button 280 to review and import data from a previous examination ", [0039]; "database of thumbnail or clipped images can provide a source of investigational data... or for use as a training tool ", [0048]; "select images based on the... abnormality descriptors, allowing a comparison of additional images from a larger database", [0044]; "The ROI is characterized as... 'High density' 410", [0041]; Gustafson teaches storing extracted (clipped) images from the "past" (previous examinations) along with "feature information" (abnormality descriptors/BI-RADS data like "High density"); Gustafson further teaches using this past data to assist the current workflow by importing it ("Clone Prey") or using the database for "comparison"; it would be obvious to automate this comparison by configuring the processor to specify and present a current "selection target" (a new image) that matches the feature information of the previously extracted images, thereby ensuring consistency with past diagnoses or training sets)
Regarding claim 13, the combination of Ofuji and Gustafson teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 1, wherein the plurality of breast images include breast images of different examinees.
(Gustafson, “allowing a comparison of additional images from a larger database”, [0044]; “a database of thumbnail or clipped images can provide a source of investigational data ... or for use as a training tool”, [0048]; accessing a larger database for comparison or training, which implies the inclusion of images from different examinees/patients; ““As more patients are definitively diagnosed and their pathology records updated in the system, the larger the collection of abnormality images depicting a previously diagnosed and imaged condition that become available in the system”, [0030]; Ofuji also identifies images by "patient ID" (“a patient ID for distinguishing subjects (subjects of radiography) from each other”, [0057]; “doctors (radiologists or the like) who diagnose patients by reading images (mammograms) can be conscious of the breast types to which the breasts in the mammograms have been classified”, [0038]), allowing for multiple subjects)
Regarding claim 14, the combination of Ofuji and Gustafson teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 1, wherein the plurality of selection candidate images corresponding to each of the plurality of mammary gland volume types are displayed on the same screen.
(Ofuji, “displaying a plurality of mammograms at the same time in the same display, arranging them in ascending order or in descending order based on the mammary-gland content rates”, [0020]; displaying multiple images sorted by mammary gland content on the same screen. Since the sorted list spans the range of content rates, images corresponding to different volume types (e.g., fatty and dense) are displayed on the same screen)
Allowable Subject Matter
Claim(s) 5 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening Claim(s).
The following is a statement of reasons for the indication of allowable subject matter:
Claim(s) 5 recite(s) limitation(s) related to switching comparison images via user input; displaying evaluation and switching buttons superimposed on images. There are no explicit teachings to the above limitation(s) found in the prior art cited in this office action and from the prior art search.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANXUN YANG whose telephone number is (571)272-9874. The examiner can normally be reached on MON-FRI: 8AM-5PM Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JIANXUN YANG/
Primary Examiner, Art Unit 2662 1/14/2026