DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Abdulsahib et. al (“Comprehensive review of retinal blood vessel segmentation and classification techniques: intelligent solutions for green computing in medical images, current challenges, open issues, and knowledge gaps in fundus medical images”).
Regarding Claim 1, Abdulsahib teaches a method (100) for generating an accurate classification of veins and arteries in a blood vessel (BV) annotation map comprising steps of:
Abstract: “Recently, there has been an advancement in the development of innovative computer-aided techniques for the segmentation and classification of retinal vessels, the application of which is predominant in clinical applications…Most of the techniques used for the classification of the retinal vessels are based on geometric and visual characteristics that set the veins apart from the arteries.”
Receiving (110) BV annotation map, veins annotation map, arteries annotation map, and disc annotation map;
Introduction: “In addition, several applications have used the segmented vascular tree, some of which include the synthesis of retinal mosaic image, biometric identification, optic disc identification temporal or multimodal image registration, and fovea localization. Examples of retinal fundus images are shown in Fig. 1, which also shows the blood vessel structures that have been manually segmented.”
Explanation: The reference teaches receiving or obtaining retinal images and annotated vascular structures used for segmentation/classification tasks. It teaches obtaining vascular maps and optic disk information which correspond to vessel and disc annotation maps.
Preprocessing (120) said annotation maps configured to generate an accurate classification of veins and arteries in a BV annotation map;
Abstract: “Then, an introduction to the pre-processing operations and advanced methods of identifying retinal vessels is deliberated.”
Retinal vessel segmentation techniques: “However, all the techniques of retinal segmentation share the same stages, which are pre-processing, processing and post-processing tasks.”
Overlaying (160) said BV map on top of a vein, artery, or both maps to determine if a given segment on said BV annotation map is a vein or an artery;
Abstract: “Most of the techniques used for the classification of the retinal vessels are based on geometric and visual characteristics that set the veins apart from the arteries.”
Retinal quantification measures: “The quantitative measurement of the retina involves measuring the structure of the retinal blood vessels, as well as angles at bifurcations, junctional exponents, measures of vascular tortuosity, length, fractal dimensions, diameter ratios, and AVR… The AVR is basically made up of the central retinal artery equivalent known as (CRAE) and the central retinal vein equivalent known as (CRVE), as well as the estimates of the arteriolar or venular diameter during the entrance of the vessels into the retina via the OD.”
Explanation: The reference teaches combining vessel feature information and structural analysis to differentiate vessel types. Using combined structural and feature information corresponds to overlaying maps to determine vessel type.
wherein said preprocessing step (120) comprises
Normalizing (130) said annotation maps;
Retinal fundus medical images: “Due to several transmutation and measurement apparatus that are designed to work on grayscale, color images must be subjected to transformation… the green channel is used in the analysis of fundus image since the difference between the vessel features and background is larger than that of other channels.”
Explanation: The reference teaches intensity normalization and contrast enhancement used in preprocessing. Transformation and channel selection correspond to normalization steps improving vessel detection.
Skeletonizing (140) said BV annotation map, said arteries annotation map or said veins annotation map, including any combination thereof; and
4.2 Vessel tracing/tracking methods: “The segmentation stage involves the extraction of the main skeleton of the RVS, whereas, in the second stage, which is the tracing stage, the digraph representation is constructed, thereby enabling the tracing process to perform as digraph depending on the label propagation by means of the Matrix-forest theorem method.”
Fragmenting (150) said BV annotation map, said arteries annotation map or said veins annotation map, including any combination thereof.
Introduction: “A crucial step in the quantitative analysis of retinal fundus images is the segmentation of blood vessels, which involves the extraction of clinically relevant features like length, tortuosity, blood vessel density, etc. from the segmented vascular tree.”
Explanation: The reference teaches segmentation and decomposition of vascular structures into analyzable components. Segmenting vascular tree into components corresponds to fragmenting vessel maps.
Regarding Claim 2, Abdulsahib teaches the method of claim 1, wherein said method corrects classification errors of veins, arteries, or both in said BV map.
Abstract: “In addition, a discussion on the validation stage and assessment of the outcomes of retinal vessels segmentation is presented.”
Retinal vessel segmentation techniques: “There are several metrics available for the assessment of retinal segmentation algorithm in terms of its ability to efficiently extract the RVS. In this regard, the commonly used metrics include average precision, average accuracy, average true-positive rate (TPR), average sensitivity (recall, TPR), and average false-positive rate (FPR). Of all the aforementioned metrics, the most commonly used ones in the area of medical research are specificity and sensitivity. When higher values of specificity and sensitivity are achieved, better diagnosis can also be achieved.”
Explanation: The reference teaches improving vessel classification accuracy using validation and evaluation metrics and correction through algorithm optimization.
Regarding Claim 3, Abdulsahib teaches the method of claim 1, wherein said annotation maps are produced from a retinal image, by algorithms configured to generate said BV annotation map, said veins annotation map, said arteries annotation map, and said disc annotation map, including any combination thereof (Abstract (shown above) and Introduction (shown above)).
Regarding Claim 4, Abdulsahib teaches the method of claim 1, wherein said normalizing (130) comprises an image resizing step (134).
4.4 Multi-scale-based methods: “The central notion of the manifold scale (multi-scale) method is to identify the retinal image at stages of multiple scales by integrating a datum that include a specific image with a set of one-factor set at multiple scales of derived images… Subsequent to the extraction of the green channel of raw images of the retina, a Gaussian pyramid of purpose hierarchy is produced. It’s clear that there are three levels in the hierarchy, including, level 0, 1, and 2. The green channel that represents the original retinal image at level 0 for the highest resolution must be obtained, the height and width of the image start to decrease as the next level is approached.”
Regarding Claim 5, Abdulsahib teaches the method of claim 4, wherein said normalizing step (130) further comprises a removal step of disc area (136) from said BV map using a disc annotation map.
4.1 Kernel-based methods: “Meanwhile, the identification of optic disc is made in two stages including (1) the detection of maximum local variance to facilitate the identification of optic disc center, and (2) the use of snake active contour for the identification of optic disc boundary.”
Regarding Claim 6, Abdulsahib teaches the method of claim 1, wherein said fragmenting step (150) comprises a graph node blackening step (154).
4.2 Vessel tracing/tracking methods: “In their work, the mathematical graph theory is used, and the building of the technique is made by establishing a relation between the tracing issue and the digraph matrix-forest theorem in the algebraic theory subject… in the second stage, which is the tracing stage, the digraph representation is constructed, thereby enabling the tracing process to perform as digraph depending on the label propagation by means of the Matrix-forest theorem method.”
Explanation: The reference teaches graph-based vascular modeling and node-based representation of vessels. Graph-node modeling corresponds to node-based fragmenting operations.
Regarding Claim 7, Abdulsahib teaches the method of claim 6, wherein said fragmenting step (156) further comprises a removal of small object step.
4.1 Kernel-based methods: “In addition, the post-processing step, which involves lengthy filtering for the elimination of isolated pixel…”
Regarding Claim 8, Abdulsahib teaches the method of claim 1, wherein said preprocessing step (120) further comprises a graph creation and a cleaning step (142).
4.2 Vessel tracing/tracking methods: “The segmentation stage involves the extraction of the main skeleton of the RVS, whereas, in the second stage, which is the tracing stage, the digraph representation is constructed, thereby enabling the tracing process to perform as digraph depending on the label propagation by means of the Matrix-forest theorem method.”
Regarding Claim 9, Abdulsahib teaches the method of claim 1, wherein said graph creation and cleaning step (142) comprises a removal of self-loops step (144), a removal of small objects step (146), or both.
4.1 Kernel-based methods: “In addition, the post-processing step, which involves lengthy filtering for the elimination of isolated pixel…”
4.2 Vessel tracing/tracking methods: “To improve the accuracy and tracking efficiency of the method, the geometric properties of the vessel are combined with the grey level.”
Explanation: Graph-based vascular modeling removes redundant graph structures and noise. Removal of loops/noise corresponds to cleaning graph structures.
Regarding Claim 10, Abdulsahib teaches the method of claim 1, wherein said method further comprises a re-adding step (164) configured to add removed areas around said nodes.
Retinal vessel segmentation techniques: “However, these problems can be solved by combining a variety of basic techniques and transformations to achieve high-performance hybrid. It is for these reasons that the use of hybrid techniques in solving problems associated with retinal vessels localization becomes essential…”
Explanation: The reference teaches combining segmentation and morphological reconstruction to restore vessel continuity. Hybrid reconstruction teaches re-adding vessel segments.
Regarding Claim 11, Abdulsahib teaches the method of claim 1, wherein said method further comprises removal of small objects step (166) from resulted BV map.
4.1 Kernel-based methods: “In addition, the post-processing step, which involves lengthy filtering for the elimination of isolated pixel…”
Regarding Claim 12, Abdulsahib teaches the method of claim 1, wherein said determination is done by segment pixel count.
Retinal vessel segmentation techniques: “Sensitivity is indicative of the algorithm’s ability to detect vessels’ pixels, while specificity indicates an algorithm’s ability to detect non-vessel pixels.”
Regarding Claim 13, Abdulsahib teaches a system for generating an accurate classification of veins and arteries in a blood vessel (BV) annotation map comprising:
Computer-implemented method (100) for accurately classifying veins and arteries in a blood vessel (BV) annotation map;
Abstract: “Recently, there has been an advancement in the development of innovative computer-aided techniques for the segmentation and classification of retinal vessels, the application of which is predominant in clinical applications.”
A computer or hardware platform for running said software;
Retinal fundus medical images: “Additionally, in the medical images are regularly encourage assessed by automated computer program to supporting the diagnostics process.”
A graphic processing unit having high computational power and resources to execute algorithms; and
Abstract: “In this study, different major contributions are summarized as review studies that adopted deep learning approaches and machine learning techniques to address each of the limitations and problems in retinal blood vessel segmentation and classification techniques.”
Explanation: Such methods require high-performance computinfg hardware including GPUs.
Display monitor for visualizing said generated BV annotation map.
Retinal fundus medical images: “By means of the fundus retinal images, the major vessel features as well as the vascular structure can be studied with several tools and applications…The design of the high-powered lenses is made in a manner that allows the ophthalmologist to visualize the eye’s rear by making light more focused through the cornea, pupil and lens.”
Regarding Claim 14, Abdulsahib teaches the system of claim 13, and additional limitations are met as in the consideration of claim 1 above.
Regarding Claim 15, Abdulsahib teaches the system of claim 13, and additional limitations are met as in the consideration of claim 2 above.
Regarding Claim 16, Abdulsahib teaches the system of claim 13, and additional limitations are met as in the consideration of claims 4 and 5 above.
Regarding Claim 17, Abdulsahib teaches the system of claim 13, and additional limitations are met as in the consideration of claims 6 and 7 above.
Regarding Claim 18, Abdulsahib teaches the system of claim 13, and additional limitations are met as in the consideration of claims 8 and 9 above.
Regarding Claim 19, Abdulsahib teaches the system of claim 13, and additional limitations are met as in the consideration of claims 10 and 11 above.
Regarding Claim 20, Abdulsahib teaches the system of claim 13, and additional limitations are met as in the consideration of claim 12 above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Dashtbozorg et. al (“An Automatic Graph-Based Approach for Artery/Vein Classification in Retinal Images”) teaches an automated artery/vein classification in retinal images using graph-based vessel analysis and centerline extraction.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM ADU-JAMFI whose telephone number is (571) 272-9298. The examiner can normally be reached M-T 8:00-6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571) 270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WILLIAM ADU-JAMFI/Examiner, Art Unit 2677
/ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677