DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Note: It is recommended to contact the Examiner to discuss a potential amendment regarding figures 14 and 15.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 8/10/2026 has been entered.
Response to Arguments
Applicant's arguments filed 8/10/2026 have been fully considered but they are not persuasive. The arguments state that the applied references do not perform the features of “which is an area other than the fundus and is not based on reflected light from the examined eye; and by performing background processing to replace a first pixel value of a first pixel configuring the background area with a second pixel value different from the first pixel value”. The Examiner respectfully disagrees with this assertion and will explain why below.
Regarding the Talwar reference, the invention discloses determining a different between foreground and background areas within in image in ¶ [25]-[27]. In particular, the system discloses using KNN based alpha matting and performing an Otsu segmentation process to the output of the alpha matting. In figure 7A-7C images of a fundus area is shown to go through the KNN-Matting to create a tri-map, which is explained in ¶ [52]-[56]. The alpha matte is then filtered by a first order derivative of Gaussian (FDOG) filter to create an image that appears in figure 8A. This creates a clearer image in the fundus area while removing the background area of black around the vessels and disc of the eye. This is explained in ¶ [57]-[61]. This makes the area outside of the vessels and disc a different pixel value or white (i.e. figures 8A-8C) than previous black around the area (i.e. see figure 7C). Thus, this performs the feature of “by performing background processing to replace a first pixel value of a first pixel configuring the background area with a second pixel value different from the first pixel value”. However, the rest of the contended feature is taught by the reference of Fujita.
Regarding the reference of Fujita, similar to the primary reference, the secondary reference comprises a dark region around the vessels and disc with some white area in between the vessels. This is reflected in figure 2.
PNG
media_image1.png
294
406
media_image1.png
Greyscale
Although the dark hashed or shaded portion is not called the background within Fujita, this is considered equivalent to the background stated in Applicant’s spec. The black shaded portion in figure 2 is considered outside of the fundus camera range, and thus considered not based on the reflected light from the examined eye. The shaded portion is a representation of an area outside of the fundus camera light. This area changes in figure 3 based on processing the entire image including the shaded region.
PNG
media_image2.png
290
366
media_image2.png
Greyscale
Since this shaded region is changed to a different color based on processing the overall image, this is considered to also change the outside pixel value to another value. Therefore, based on the above, the rejection of the claims is maintained.
Thus, based on the above, the features of the claims are disclosed below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 4-6, 8-11 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Talwar (US Pub 2016/0163041) in view of Fujita (JP Pub 2007-097634 (Pub Date: 4/19/2007)).
Re claim 1: Talwar discloses an image processing method comprising:
a processor acquiring a first fundus image of the examined eye including a foreground area, which is an area of a fundus that includes blood vessels, and a background area other than the foreground area, which is an area other than the fundus (e.g. an image of a fundus image is acquired, which is taught in ¶ [18], [29] and [30]. The image is separated into foreground and background, which is taught in ¶ [53]. A processor is taught in ¶ [62]. Figures 5, 7 and 8 show the vessels within the foreground area and the background area outside the area where the vessels are present.); and
[0018] A technique for blood vessel extraction in fundus color images of the eye using an alpha-matting technique is described. In one embodiment, the alpha-matting technique is a K-nearest neighbors (KNN) based alpha-matting and is used to separate vessel and non-vessel regions in an image. In one embodiment, larger blood vessels are used to generate the tri-map needed for matting. Therefore, in one embodiment, no input is required from the user to generate the tri-map. A multi-dimensional feature set is constructed and fed into the matting framework. The affinities among the pixels are used to obtain the matting Laplacian, whose eigen decomposition leads to segregation of the vessel and non-vessel regions in the image having retinal vessels.
[0029] FIG. 1 is a flow diagram of one embodiment of a process for creating a multi-dimensional feature space. The process is performed by processing logic that may comprise hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), firmware, or a combination of the three.
[0030] Referring to FIG. 1, the process begins by performing preprocessing on an original image 100 (processing block 110). Color fundus images invariably show important intensity variations, poor contrast and noise. In one embodiment, the preprocessing includes filtering and other operations performed on the input image. For example, in one embodiment, the preprocessing includes noise filtering to remove granular noise. In one embodiment, the noise filtering is accomplished by using Wiener filtering. In one embodiment, a Weiner filter is applied over a 3×3 window to remove granular and speckle noise from the image.
KNN-Matting
[0053] Alpha matting refers to the process of separating an image into foreground and background components and obtaining a corresponding opacity mask. The opacity mask is commonly called the alpha matte and its value is obtained for each individual pixel in the image. Typically, for a grayscale image
I(i,j)=αF(i,j)+(1−α)B(i,j) (6)
where I(i,j) is the image intensity at pixel (i,j), F is the foreground and B is the background image. At each pixel, the value of three unknowns {α, F, B} is calculated from one equation. Consequently, the alpha matting problem is highly under constrained. A user could provide a tri-map or scribbles which can be used to identify known foregrounds and backgrounds. However, additional constraints are still needed to solve the problem. KNN matting uses the multi-dimensional feature space and works on non-local principle for finding neighbors. It is advantageous because it does not need large kernels to search for similar pixels in the neighborhood and provides a closed form solution.
[0062] Referring to FIG. 9, vessel extraction system 910 includes a bus 912 to interconnect subsystems of vessel extraction system 910, such as a processor 914, a system memory 917 (e.g., RAM, ROM, etc.), an input/output controller 918, an external device, such as a display screen 924 via display adapter 926, serial ports 928 and 930, a keyboard 932 (interfaced with a keyboard controller 933), a storage interface 934, a floppy disk drive 937 operative to receive a floppy disk 938, a host bus adapter (HBA) interface card 935A operative to connect with a Fibre Channel network 990, a host bus adapter (HBA) interface card 935B operative to connect to a SCSI bus 939, and an optical disk drive 940. Also included are a mouse 946 (or other point-and-click device, coupled to bus 912 via serial port 928), a modem 947 (coupled to bus 912 via serial port 930), and a network interface 948 (coupled directly to bus 912).
the processor generating a second fundus image by performing background processing to replace a first pixel value of a first pixel configuring the background area with a second pixel value different from the first pixel value, (e.g. the system discloses using KNN based alpha matting and performing an Otsu segmentation process to the output of the alpha matting. In figure 7A-7C images of a fundus area is shown to go through the KNN-Matting to create a tri-map, which is explained in ¶ [52]-[56]. The alpha matte is then filtered by a first order derivative of Gaussian (FDOG) filter to create an image that appears in figure 8A. This creates a clearer image in the fundus area while removing the background area of black around the vessels and disc of the eye. This is explained in ¶ [57]-[61]. This makes the area outside of the vessels and disc a different pixel value or white (i.e. figures 8A-8C) than previous black around the area (i.e. see figure 7C).).
[0050] Referring to FIG. 6, processing logic receives feature images 601 of the feature space and applies a matched filter with first-order derivative of the Gaussian (MF-FDOG) to the images (processing block 602). Processing logic also applies the MF-FDOG filter to the original (preprocessed) image 610 (processing block 611).
[0051] Processing logic then performs alpha matting to the image data of the images (processing block 604). In one embodiment, the alpha matting is K-nearest neighbor (KNN)-alpha matting.
[0052] Using the alpha matte 604 produced by performing alpha matting and the filter image from processing block 611, processing logic performs Otsu segmentation (processing block 605) and produces output image 606, which is an image of the delineated block vessels.
Tri-Map Generation
[0055] As mentioned above, any matting problem needs user inputs in the form of sparse mark-ups. In one embodiment, the dominant vessels in the retina images are used to generate the tri-map automatically. The normalized high pass image consists of only dominant vessels and is used to create a skeleton for the vasculature. This skeleton acts as user scribbles for foreground area (vessels). In one embodiment, prior to using the normalized high pass image as a tri-map, optic cup removal is applied to the normalized high pass image.
[0056] Morphological thinning operation and area thresholding are performed before using the high pass image as a tri-map. The known background areas are also determined from the same image. It is assumed that the normalized image covers most of the vasculature of the retina images. This image is dilated and the inverted image can then be used as the background mask. FIGS. 7A-C illustrate a normalized image, an example of a foreground map generated by the procedure and a background map generated by this procedure, respectively.
Segmentation
[0057] The feature images in multi-dimensional feature set contain traces of boundary of optic disc and eye due to their vessel like appearance. In one embodiment, a first order derivative of Gaussian (FDOG) filter is used to suppress these. Blood vessels have a gaussian like profile across their cross-section whereas the optic disk and other thick bright regions have intensity profile similar to a step function. The FDOG curve is anti-symmetric around its center value. Thus, the response to applying an FDOG filter is symmetric for bright optic disk and anti-symmetric for blood vessels. The local average of the response (equivalent to low pass filtering) typically returns a high value at the boundary of optic disk and minimal value for blood vessels. The inverted image of this response is used as mask, multiplied to the feature images in the multi-dimensional feature set.
[0058] For choosing a threshold value, the Otsu segmentation technique is used along with the locally averaged response output from the FDOG filter. The Otsu segmentation on alpha matte returns a threshold value which is added to the locally averaged response image output from the FDOG filter. The resulting image generated by adding the threshold value to the locally averaged response image output from the FDOG filter is then used as a reference threshold image, Every pixel in the resultant alpha matte is compared with the corresponding pixel in the reference image and a decision (binarize) is made based on whether the value in the reference image is higher or lower than a current pixel. This yields a relatively higher threshold value for false positives regions whereas other regions are compared against the normal value of Otsu segmentation technique.
[0059] FIGS. 8A-C show an alpha matte, a thresholded image, and ground truth, respectively. As shown in FIGS. 8A-C, the alpha matte contains a prominent optic disk boundary which can be greatly reduced by the FDOG based thresholding technique.
[0060] Thus, a novel scheme of alpha matting to separate foreground from background for purposes of vessel extraction has been disclosed. The tri-map generation to generate the tri-map as part of the alpha matting is automated from the generated features. This serves skeleton for sure vessel region whereas background region is constructed from it. This is a huge step in the automatic extraction of blood vessel from retina color images, thus reducing the human involvement.
An Example of a Vessel Extraction System
[0061] FIG. 9 depicts a block diagram of a vessel extraction system. The vessel extraction system comprising a memory to store one or more images, such as an original image that is to undergo retinal vessel extraction and/or feature set images. The system also includes a processing unit that acts as a retinal vessel extractor to perform alpha matting on a multi-dimensional feature set derived from image data of a first image and perform retinal vessel extraction, including performing segmentation on the image by separating foreground and background image data from the first image using an output from the alpha matting. In one embodiment, the alpha matting comprises K-nearest neighbor (KNN) based alpha matting and the processing unit performs an Otsu segmentation process on the output of the alpha matting.
However, Talwar fails to specifically teach the features of a processor controlling an emission of light from a light source to emit light towards an examined eye, and is not based on reflected light from the examined eye.
However, this is well known in the art as evidenced by Fujita. Similar to the primary reference, Fujita discloses evaluating a fundus (same field of endeavor or reasonably pertinent to the problem).
Fujita discloses a processor controlling an emission of light from a light source to emit light towards an examined eye (e.g. it is conventional to have an eyeball irradiated with light controlled by a controller in a device in order to capture the state of the fundus. The state of the fundus is then stored as a medical record, which is taught in ¶ [02].),
[0002] In addition, the state of the fundus is observed by irradiating the eyeball with light from the outside, or the state of the fundus is photographed as a fundus photograph (fundus image) by an optical device such as a camera, and is recorded as a medical record.
[0032] Next, the flow of processing of the image analysis computer 2 in the image analysis system 1 of the present embodiment will be described mainly based on the flowcharts of FIGS. 6 and 7. Here, Steps S1 to S20 in FIGS. 6 and 7 correspond to the image analysis program of the present invention.
[0033] First, the fundus of a subject (patient or examinee) is photographed using a fundus camera 9 having a digital camera function, and image data 15 (see FIG. 2) related to the photographed fundus image 3 is image data reading means 16. Is read into the image analysis computer 2 (step S1). Here, the fundus image 3 that is the source of the image data 15 may be either a color image or a monochrome image. In the present embodiment, the image data 15 is directly read from the fundus camera 9. For example, the fundus image (fundus photo) printed on the photographic paper is read using an optical reading device such as a scanner. The digitized image data 15 may be acquired by scanning or scanning a fundus photographic negative film using a film scanner. Then, the read image data 15 is stored in the storage means 17 (step S2).
[0034] Thereafter, the read image data 15 is analyzed, and binarization processing is performed according to a predetermined threshold value using a difference in pixel value of each pixel 21 constituting the image data 15, and the processed binary image is processed. Two regions of the blood vessel region 4 and the background region 19 are extracted from the data 18 (step S3: see FIG. 3). In the photographed fundus image 3, a portion (part) corresponding to the blood vessel region 4 is shown in black or dark color, and a background region 19 other than the blood vessel region 4 (for example, a retinal region or an optic disc) is generally used. It is often shown in white or lighter than the blood vessel region 4. Therefore, the blood vessel region 4 and the background region 19 can be clearly extracted by using the difference in color (pixel value) and binarizing the above-described image data 15 with the prescribed threshold as a boundary. it can. Here, the extracted blood vessel region 4 and the background region 19 are configured by a plurality of square pixels 21 (pixels) closely arranged vertically and horizontally, as schematically shown in FIG. That is, the boundary portion 24 between the blood vessel region 4 and the background region 19 is in a state in which the vertical or horizontal sides of each pixel 21 are in contact with each other, and is configured by a combination of straight lines and right angles. Then, one target pixel 5 is selected from the plurality of pixels 21 (step S4). The selection of the target pixel 5 is performed on all the pixels constituting the blood vessel region 4 as will be described later, so that the target pixel 5 is selected in order from the end of the blood vessel region 4 in advance. It is selected according to established rules. Simultaneously with the selection, the value of the inclination θ with respect to the virtual perpendicular V passing through the pixel center C of the selected target pixel 5 is set to “θ = 0 °” (step S5).
PNG
media_image3.png
322
370
media_image3.png
Greyscale
PNG
media_image4.png
298
372
media_image4.png
Greyscale
and is not based on reflected light from the examined eye (e.g. although the dark hashed or shaded portion is not called the background within Fujita, this is considered equivalent to the background stated in Applicant’s spec. The black shaded portion in figure 2 is considered outside of the fundus camera range and is considered not based on the reflected light from the examined eye. The shaded portion is a representation of an area outside of the fundus camera light. This area is changed in figure 3 based on processing the entire image including the shaded region. The figures are explained in ¶ [25]-[27].).
[0025]First, the fundus of a subject (patient or examinee) is photographed using the retinal camera 8 having a digital camera function, and the image datum 9 (see FIG. 2) relating to the photographed fundus image 3 is read into the image analysis computer 2 via the image datum reading means 18 (step S1). Here, the fundus oculi image 3 that is the source of the image data 9 may be either a color image or a monochrome image. Although the image data 9 is directly read from the fundus camera 8 in this embodiment, the digitized image data 9 may be acquired by, for example, scanning a fundus image (fundus photograph) printed on photographic paper using an optical reading device such as a scanner or scanning a negative film of a fundus photograph using a film scanner. Then, the read image datum 9 is stored in the storage means 37 (step S2).[0026]Thereafter, the read image datum 9 is analyzed, binarization processing is performed in accordance with predetermined thresholds using differences in pixel values of respective pixels 38 constituting the image datum 9, and two regions of the vascular region 4 and the background region 19 are respectively extracted from the processed binarized image datum 39 (step S3: see FIG. 3). In the captured fundus image 3, a portion (site) corresponding to the blood vessel region 4 is shown in a relatively dark color, and a background region 19 (for example, the retina region 10, the optic disc portion 32, or the like) other than the blood vessel region 4 is generally shown in a color brighter than the blood vessel region 4 in many cases. Therefore, the blood vessel region 4, the background region 19, and the like other than the blood vessel region 4 can be clearly extracted by binarizing the above-described image data 9 using the difference in color (pixel value) and using the specified threshold value as a boundary. Information on the extracted region is stored in the storage unit 37 as the region data 27.[0027]Thereafter, the portion corresponding to the extracted vascular region 4 is erased from the image 9 (step S4: see FIG. 4). As a result, the portion of the blood vessel region 4 of the image data 9 is displayed as a blank. Then, an erased portion 22 where the vascular region 4 has been erased is estimated from the pixel values of the background region 19 in the vicinity of the pixel values, and interpolation processing is performed (step S5).
Therefore, in view of Fujita, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of a processor controlling an emission of light from a light source to emit light towards an examined eye, and is not based on reflected light from the examined eye, incorporated in the device of Talwar, in order to acquire an amount of information about the eye, which can aid in acquiring useful information without a physical burden on a patient (as stated in Fujita ¶ [02]).
Re claim 4: Talwar discloses the image processing method of claim 1, wherein the background area is a single color area (e.g. the background is a single color in the image shown in figure 7A, which is described in ¶ [55] and [56] above.).
Re claim 5: Talwar discloses the image processing method of claim 1, further comprising the processor generating a third fundus image by binarizing pixel values of pixels of the foreground area in the second fundus image or in an image resulting from emphasizing blood vessels in the second fundus image, by binarization with respect to a threshold value determined based on pixel values of peripheral pixels to the pixels of the foreground area (e.g. Figure 7C shows a binarization of figure 7B or an inversion of the values in order to create a third image, which is taught in ¶ [55] and [56] above and illustrated in Figures 7A-7C.).
Re claim 6: Telwar discloses the image processing method of claim 1, further comprising the processor executing processing to analyze blood vessels of a fundus of the examined eye (e.g. the invention teaches analyzing the blood vessels of the image, which is taught in ¶ [18] above.).
Re claim 8: Talwar discloses the image processing method of claim 1, further comprising the processor replacing, with respect to the second fundus image, a pixel value of a pixel of the background area with a third pixel value different from the second pixel value (e.g. the pixels in the image are again compared to a threshold in order to determine whether to binarize the pixel. This would turn the pixel to a different value based on being higher or lower than a current pixel, which is taught in ¶ [57]-[60]. In addition, the background area is turned to white, which is seen in figure 8A-8C.).
Segmentation
[0057] The feature images in multi-dimensional feature set contain traces of boundary of optic disc and eye due to their vessel like appearance. In one embodiment, a first order derivative of Gaussian (FDOG) filter is used to suppress these. Blood vessels have a gaussian like profile across their cross-section whereas the optic disk and other thick bright regions have intensity profile similar to a step function. The FDOG curve is anti-symmetric around its center value. Thus, the response to applying an FDOG filter is symmetric for bright optic disk and anti-symmetric for blood vessels. The local average of the response (equivalent to low pass filtering) typically returns a high value at the boundary of optic disk and minimal value for blood vessels. The inverted image of this response is used as mask, multiplied to the feature images in the multi-dimensional feature set.
[0058] For choosing a threshold value, the Otsu segmentation technique is used along with the locally averaged response output from the FDOG filter. The Otsu segmentation on alpha matte returns a threshold value which is added to the locally averaged response image output from the FDOG filter. The resulting image generated by adding the threshold value to the locally averaged response image output from the FDOG filter is then used as a reference threshold image, Every pixel in the resultant alpha matte is compared with the corresponding pixel in the reference image and a decision (binarize) is made based on whether the value in the reference image is higher or lower than a current pixel. This yields a relatively higher threshold value for false positives regions whereas other regions are compared against the normal value of Otsu segmentation technique.
[0059] FIGS. 8A-C show an alpha matte, a thresholded image, and ground truth, respectively. As shown in FIGS. 8A-C, the alpha matte contains a prominent optic disk boundary which can be greatly reduced by the FDOG based thresholding technique.
[0060] Thus, a novel scheme of alpha matting to separate foreground from background for purposes of vessel extraction has been disclosed. The tri-map generation to generate the tri-map as part of the alpha matting is automated from the generated features. This serves skeleton for sure vessel region whereas background region is constructed from it. This is a huge step in the automatic extraction of blood vessel from retina color images, thus reducing the human involvement.
Re claim 9: Talwar discloses the image processing method of claim 8, wherein the first pixel value is the same as the third pixel value (e.g. the pixels in the background area in figure 8A are white, which is taught in ¶ [57]-[60] above and seen in figure 8A-8C.).
Re claim 10: Talwar discloses the image processing method of claim 1, wherein the background processing is performed on at least pixels adjacent to pixels of the foreground area, among pixels configuring the background area (e.g. background pixel processing occurs on pixels that are adjacent to foreground pixels, which is taught in ¶ [55] and [56] above.).
Re claim 11: Talwar discloses the image processing method of claim 1, wherein the second pixel value is a value in a range of possible values for pixel values of pixels in the foreground area (e.g. the background is turned to a value that was the value of the foreground area before inversion, which is taught in ¶ [57]-[60] above.).
Re claim 13: Talwar discloses an image processing device comprising:
a memory, and a processor coupled to the memory (e.g. the invention contains a processor and memory coupled thereto, which is taught in ¶ [62].),
[0062] Referring to FIG. 9, vessel extraction system 910 includes a bus 912 to interconnect subsystems of vessel extraction system 910, such as a processor 914, a system memory 917 (e.g., RAM, ROM, etc.), an input/output controller 918, an external device, such as a display screen 924 via display adapter 926, serial ports 928 and 930, a keyboard 932 (interfaced with a keyboard controller 933), a storage interface 934, a floppy disk drive 937 operative to receive a floppy disk 938, a host bus adapter (HBA) interface card 935A operative to connect with a Fibre Channel network 990, a host bus adapter (HBA) interface card 935B operative to connect to a SCSI bus 939, and an optical disk drive 940. Also included are a mouse 946 (or other point-and-click device, coupled to bus 912 via serial port 928), a modem 947 (coupled to bus 912 via serial port 930), and a network interface 948 (coupled directly to bus 912).
wherein the processor:
acquires a first fundus image of an examined eye including a foreground area, which is an area of a fundus that includes blood vessels, and a background area other than the foreground area, which is an area other than the fundus (e.g. an image of a fundus image is acquired, which is taught in ¶ [18], [29] and [30] above. The image is separated into foreground and background, which is taught in ¶ [53] above. A processor is taught in ¶ [62] above. Figures 5, 7 and 8 show the vessels within the foreground area and the background area outside the area where the vessels are present.); and
generates a second fundus image by performing background processing to replace a first pixel value of a first pixel configuring the background area with a second pixel value different from the first pixel value (e.g. the system discloses using KNN based alpha matting and performing an Otsu segmentation process to the output of the alpha matting. In figure 7A-7C images of a fundus area is shown to go through the KNN-Matting to create a tri-map, which is explained in ¶ [52]-[56] above. The alpha matte is then filtered by a first order derivative of Gaussian (FDOG) filter to create an image that appears in figure 8A. This creates a clearer image in the fundus area while removing the background area of black around the vessels and disc of the eye. This is explained in ¶ [57]-[61] above. This makes the area outside of the vessels and disc a different pixel value or white (i.e. figures 8A-8C) than previous black around the area (i.e. see figure 7C).).
However, Talwar fails to specifically teach the features of an examined eye, and is not based on reflected light from the examined eye.
However, this is well known in the art as evidenced by Fujita. Similar to the primary reference, Fujita discloses evaluating a fundus (same field of endeavor or reasonably pertinent to the problem).
Fujita discloses an examined eye (e.g. it is conventional to have an eyeball irradiated with light controlled by a controller in a device in order to capture the state of the fundus. The state of the fundus is then stored as a medical record, which is taught in ¶ [02] above.),
and is not based on reflected light from the examined eye (e.g. although the dark hashed or shaded portion is not called the background within Fujita, this is considered equivalent to the background stated in Applicant’s spec. The black shaded portion in figure 2 is considered outside of the fundus camera range and is considered not based on the reflected light from the examined eye. This area is modified in figure 3 based on processing the entire image including the shaded region. The figures are explained in ¶ [25]-[27] above.).
Therefore, in view of Fujita, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of an examined eye, and is not based on reflected light from the examined eye, incorporated in the device of Talwar, in order to acquire an amount of information about the eye, which can aid in acquiring useful information without a physical burden on a patient (as stated in Fujita ¶ [02]).
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Talwar in view of Fujita and Yasuno (US Pub 2016/0135683).
Re claim 14: Talwar discloses a non-transitory storage medium storing a program that causes a computer to execute processing comprising:
acquiring a first fundus image of an examined eye including a foreground area, which is an area of a fundus that includes blood vessels, and a background area other than the foreground area, which is an area other than the fundus (e.g. an image of a fundus image is acquired, which is taught in ¶ [18], [29] and [30] above. The image is separated into foreground and background, which is taught in ¶ [53] above. A processor is taught in ¶ [62] above. Figures 5, 7 and 8 show the vessels within the foreground area and the background area outside the area where the vessels are present.); and
generating a second fundus image by performing background processing to replace a first pixel value of a first pixel configuring the background area with a second pixel value different from the first pixel value (e.g. the system discloses using KNN based alpha matting and performing an Otsu segmentation process to the output of the alpha matting. In figure 7A-7C images of a fundus area is shown to go through the KNN-Matting to create a tri-map, which is explained in ¶ [52]-[56] above. The alpha matte is then filtered by a first order derivative of Gaussian (FDOG) filter to create an image that appears in figure 8A. This creates a clearer image in the fundus area while removing the background area of black around the vessels and disc of the eye. This is explained in ¶ [57]-[61] above. This makes the area outside of the vessels and disc a different pixel value or white (i.e. figures 8A-8C) than previous black around the area (i.e. see figure 7C).).
However, Talwar fails to specifically teach the features of an examined eye, wherein the first fundus image is acquired using reflected light from the examined eye in response to an emission of light from a light source.
However, this is well known in the art as evidenced by Fujita. Similar to the primary reference, Fujita discloses evaluating a fundus (same field of endeavor or reasonably pertinent to the problem).
Fujita discloses controlling an examined eye, wherein the first fundus image is acquired using reflected light from the examined eye in response to an emission of light from a light source (e.g. it is conventional to have an eyeball irradiated with light controlled by a controller in a device in order to capture the state of the fundus. The state of the fundus is then stored as a medical record, which is taught in ¶ [02] above.),
and is not based on reflected light from the examined eye (e.g. although the dark hashed or shaded portion is not called the background within Fujita, this is considered equivalent to the background stated in Applicant’s spec. The black shaded portion in figure 2 is considered outside of the fundus camera range and is considered not based on the reflected light from the examined eye. This area is modified in figure 3 based on processing the entire image including the shaded region. The figures are explained in ¶ [25]-[27] above.).
Therefore, in view of Fujita, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of an examined eye, wherein the first fundus image is acquired using reflected light from the examined eye in response to an emission of light from a light source, and is not based on reflected light from the examined eye, incorporated in the device of Talwar, in order to acquire an amount of information about the eye, which can aid in acquiring useful information without a physical burden on a patient (as stated in Fujita ¶ [02]).
However, the combination above fails to specifically teach the features of comprises a vascular image in which retinal blood vessels are removed and choroidal blood vessels are visible.
However, this is well known in the art as evidenced by Yasuno. Similar to the primary reference, Yasuno discloses emphasizing choroidal vessels from scanned data (same field of endeavor or reasonably pertinent to the problem).
Yasuno discloses comprises a vascular image in which retinal blood vessels are removed and choroidal blood vessels are visible (e.g. the invention discloses separating the choroidal vessels from the rest of the vascular network image. This involves removing the shadows of the retinal vessel image and while the choroidal vessel images are emphasized or shown with the removal of the retinal vessel image. This is taught in the ¶ [38]-[41], [114] and [119]-[122].).
[0038] To achieve the aforementioned object, the present invention provides an optical coherence tomography apparatus for selectively visualizing and analyzing the vascular network in the choroidal layer comprising an optical coherence tomography, and a computer that obtains three-dimensional OCT tomographic images based on OCT-measured data acquired by the optical coherence tomography and processes the three-dimensional OCT tomographic images, wherein such optical coherence tomography apparatus for selectively visualizing and analyzing the vascular network in the choroidal layer is characterized in that: the computer functions as a means for selectively separating out only the images of the choroidal vessels from the three-dimensional OCT tomographic images to obtain image data of the choroidal vessels, and also as a means for obtaining the data to be used in the quantitative evaluation of the shape of the choroidal vessels based on the image data of the choroidal vessels; and the means for acquiring image data of the choroidal vessels is constituted in such a way that tomographic image data of the choroidal layer is extracted from OCT-measured data, the tomographic image data of the choroidal layer is sliced at equally pitched positions in the depth direction of the choroidal layer and data of image slices is extracted, after which image data of the choroidal vessels is obtained from the data of image slices.
[0039] Desirably the means for acquiring image data of the choroidal vessels is constituted in such a way that, for each of multiple windows of different sizes, each pixel in the image slice is binarized according to whether or not the pixel color density is equal to or higher than the pre-determined specified threshold in order to obtain an estimated vessel parts-extracted binary image, and in this estimated vessel parts-extracted binary image, those regions where the ratio of pixels with different binary data to each pixel in the applicable window is equal to or greater than the pre-determined specified value are deleted as pseudo vessels, while the estimated vessel parts whose diameter is smaller than the pre-determined specified diameter with respect to the dimension of the applicable window are also deleted as noise and non-vessels, in order to obtain binary image data of the vascular network in the choroidal layer.
[0040] Desirably the means for acquiring image data of the choroidal vessels is constituted in such a way that the depth-direction slope of the anterior region of Bruch's membrane at the back of the eye is detected from the three-dimensional OCT tomographic images to obtain data at the positions of the inner segment/outer segment junctions with the photoreceptor cells at the back of the eye, so that the data at the positions of the inner segment/outer segment junctions and Bruch's membrane is used to extract highly reflective structures around the retinal pigment epithelium, after which the optical intensities at the highly reflective structures are averaged to obtain image data specifying the shadows created by the retinal vessels at the highly reflective structures, which is then followed by flange-filtering the image data to emphasize the lines and binarizing the obtained image data, in order to obtain binary image data of the shadows of the retinal vessels.
[0041] Desirably the means for acquiring image data of the choroidal vessels is constituted in such a way that, for all pixels corresponding to the vessels, classified data of medium and small vessels and large vessels that have been classified by magnitude of optical intensity is created based on the tomographic image data of the choroidal layer, and based on this classified data, the binary image data of the vascular network in the choroidal layer as obtained for each of the multiple windows of different sizes is selectively combined, while at the same time binary image data of the choroidal network vessels separating out only the choroidal vessels is formed based on the binary image data of the shadows of the retinal vessels.
[0114] On the OCT tomographic image of the choroidal layer, shadows of the retinal vessels in front are projected and mistaken as the choroidal vessels. This mistake can inhibit accurate quantitative evaluation of the thickness of the choroidal layer and thickness of the choroidal vessels. Accordingly, these shadows must be removed.
(7) Means and Method for Separating Out the Shadows Created by the Retinal Vessels ((7) in FIG. 3)
[0119] As explained in (6) above, the retinal vessels enter the image data of the choroidal layer as shadows and are mistaken as the choroidal vessels in the data of image slices. This mistake can inhibit accurate quantitative evaluation of the thickness of the choroidal layer and thickness of the choroidal vessels.
[0120] Accordingly, the images at the highly reflective structures obtained in (6) above must be used to separate out the shadows created by the retinal vessels, from the choroidal vessels. The image-processing program pertaining to the present invention causes the computer to function as a means for separating out the shadows created by the retinal vessels and to perform the following operations.
[0121] The image data at the highly reflective structures is flange-filtered to emphasize the lines, and as a result of non-linear binarization with appropriate threshold (flange filtering, etc.), image data is obtained which consists of “1” representing the vessel parts and “0” representing the remainder, as shown in FIG. 6 (b).
[0122] Thereafter, morphological closing (the regions having the value “1” are eroded and then dilated) is performed to ensure continuity of the vessels, which effectively removes the shadows of the retinal vessel image that have been extracted slightly thicker than they actually are, from the choroidal vessel image. Since the retinal vessel image is completely removed, the binary image data shown in FIG. 6 (c) can be obtained.
Therefore, in view of Yasuno, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of comprises a vascular image in which retinal blood vessels are removed and choroidal blood vessels are visible, incorporated in the device of Talwar, as modified by Fujita, in order to remove the retina vessel images to view a choroidal vessel image, which clears an image to view the appropriate vessels within the image (as stated in Yasuno ¶ [34]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Dimter discloses replacing background pixels with foreground pixels when discussing figure 9.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHAD S DICKERSON whose telephone number is (571)270-1351. The examiner can normally be reached Monday-Friday 10AM-6PM EST..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abderrahim Merouan can be reached on 571-270-5254. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHAD DICKERSON/ Primary Examiner, Art Unit 2681
Prosecution Insights