DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-20 are pending.
Response to Arguments
Applicant’s arguments, see p. 9, filed June 30, 2026, with respect to the 35 USC 112(a) rejections have been fully considered and are persuasive. The amendments to the claims have overcome the previous rejection and it has been withdrawn.
Applicant’s arguments, see p. 9-11, filed June 30, 2026, with respect to the 35 USC 103 rejection of claim 1 have been fully considered, however, as referenced in the updated rejection below, amended claim 1 is now rejected under the Tosa reference, which was previously presented in the rejection of claims 13-17. Tosa teaches categorizing an image into an eyelash occlusion class and a non-eyelash occlusion class based on a predetermined threshold and training a model using images from the non-eyelash occlusion class (see Tosa, Paras. [0082] and [0084]). Additionally, the amended limitation of “generating a set of final generated images by inputting fundus images belonging to the eyelash occlusion class and the corresponding inflated segmentation masks into the image restoration model” is taught by the Sha reference, which teaches obtaining the final image by inputting the image with eyelashes (i.e., fundus images belonging to the eyelash occlusion class) and the mask image (i.e., segmentation mask) into generators to produce outputs to add to the input image to obtain the final image (see Sha, Para. [0097]). Although the Sha reference does not explicitly teach the inflated segmentation mask is input, Examiner asserts that one having ordinary skill in the art would have been motivated to combine the Sha reference with the Wang reference, which teaches inflating a segmentation mask (see Wang, pg. 3780), to teach an inflated segmentation mask to be input into the image restoration model.
Applicant further argues that the specific classification-based workflow of the present claims is not shown by the cited references and that the limitations should not be regarded as inherent in the cited references. Examiner asserts that, as shown in the 35 USC 103 rejections below, the rejections are not based on inherency but rather, each and every limitation is taught by a prior art reference, and as such, the workflow of the claims is taught. Although the Sha reference does not teach a specific threshold based classification into eyelash and non-eyelash images, the Tosa reference teaches determining heavy eyelid/eyelash occlusion based on a measure of coverage of the iris (i.e., a threshold) (see Tosa, Para. [0082]). The limitations based on the type of occlusion class do not rely on inherency but are instead taught by the prior art reference in a classification based way. For example, “training the image restoration model for fine-tuning training of random dynamic mask restoration using images from the non-eyelash occlusion class” is taught by Tosa in Para. [0084], which states generated training histograms from images believed to be free of occlusion. Additionally, “generating a set of final generated images by inputting fundus images belonging to the eyelash occlusion class and the corresponding inflated segmentation masks into the image restoration model” is taught by the Sha reference, which teaches obtaining the final image by inputting the image with eyelashes (i.e., fundus images belonging to the eyelash occlusion class) and the mask image (i.e., segmentation mask) into generators to produce outputs to add to the input image to obtain the final image (see Sha, Para. [0097]). Thus, Examiner asserts the claimed workflow is taught by the prior art references without relying upon inherency, and the 35 USC 103 rejection of claim 1 is maintained.
Applicant’s arguments, see p. 11, filed June 30, 2026, with respect to the 35 USC 103 rejection of claim 2 have been fully considered but they are not persuasive. Applicant argues that amended claim 2 is not taught by the Wang reference. Examiner respectfully disagrees. As presented in the 35 USC 103 rejections below, the Wang reference teaches the high-resolution features of the SSSR stream are further enhanced by the fine-grained structural representation from the SISR stream with Feature Affinity (FA) module. Moreover, these two streams share the same feature extractor (see Wang, pg. 3775). Thus, the 35 USC 103 rejection of claim 2 is maintained.
Applicant’s arguments, see p. 12, filed June 30, 2026, with respect to the 35 USC 103 rejection of claim 7 have been fully considered but they are not persuasive. Applicant argues that amended claim 7 is not taught by the prior art of record. Examiner respectfully disagrees. As presented in the 35 USC 103 rejections below, the Sha reference teaches generating segmentation masks for fundus images with eyelash occlusions (see Sha, Para. [0097]). Although the Sha reference does not explicitly teach the segmentation mask is inflated, Examiner asserts that one having ordinary skill in the art would have been motivated to combine the Sha reference with the Wang reference, which teaches inflating a segmentation mask (see Wang, pg. 3780), to teach generating inflated segmentation masks for fundus images categorized into the eyelash occlusion class. Thus, the 35 USC 103 rejection of the claims is maintained, and consequently, THIS ACTION IS FINAL.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3, 13-15 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Sha et al. (CN 116934609 A, machine translation used herein for mapping purposes) in view of Tosa et al. (US 2008/0253622 A1) further in view of Wang et al. (“Dual Super-Resolution Learning for Semantic Segmentation”) and Bermano et al. (US 2017/0024907 A1).
Regarding claim 1, Sha teaches a method of virtual eyelash removal for ultra-wide-field fundus images comprising the steps of:
obtaining a set of original ultra-wide-field fundus images (Sha, Para. [0002], ultra-wide-angle fundus cameras use the principle of ellipsoidal lenses to create images, resulting in a very large depth of focus);
determining if any of the set of original ultra-wide-field fundus images has one or more occluding eyelashes (Sha, Para. [0079], determine the location of the eyelash root in the original ultra-wide-angle fundus image (i.e., determine presence of occluding eyelash)),
and if so, marking the one or more occluding eyebrows to identify the one or more occluding eyebrows for later processing and adding a corresponding member of the set of original ultra-wide-field fundus images to a set of occluded images (Sha, Para. [0089], a first region containing the location of the eyelash root is cropped from the above eyelash mask image, and an eyelid region and an eyelash-free non-eyelid region as an image without eyelash masking are cropped from the original ultra-wide-angle fundus image respectively (i.e., marking the one or more occluding eyelash and eyelid regions for later processing));
creating a set of segmentation masks, each member of the set of segmentation masks corresponding to the occluding eyebrows in one member of the set of occluded images (Sha, Para. [0085], traverse all points on the center line of the eyelash to be generated, construct Gaussian heatmaps with these points as the center points in turn and superimpose them, and then perform Gaussian blur to obtain the eyelash mask image (i.e., segmentation mask corresponding to occluding eyelash));
inputting the set of occluded images into an image restoration model (Sha, Para. [0097], the first generator takes a three-channel image with eyelashes as input (i.e., inputting the set of occluded images into an image restoration model), and the second generator takes a concatenated image without eyelashes and an eyelash mask image as input, for a total of 4 channels. Both generators produce a three-channel residual image, which is then added to the input image to obtain the final image with or without eyelashes);
generating a set of final generated images by inputting fundus images belonging to the eyelash occlusion class and the corresponding (Sha, Para. [0097], the first generator takes a three-channel image with eyelashes as input (i.e., fundus images belonging to the eyelash occlusion class), and the second generator takes a concatenated image without eyelashes and an eyelash mask image (i.e., inflated segmentation masks) as input, for a total of 4 channels. Both generators produce a three-channel residual image, which is then added to the input image to obtain the final image with or without eyelashes).
Although Sha teaches determining occluding eyelashes (Sha, Para. [0079], Sha does not explicitly teach “obtaining eyelash segmentation results for the set of original ultra-wide-field fundus images and categorizing the original ultra-wide-field fundus images into an eyelash occlusion class and a non-eyelash occlusion class based on a predetermined threshold” and “training the image restoration model for fine-tuning training of random dynamic mask restoration using images from the non-eyelash occlusion class”. However, in an analogous field of endeavor, Tosa teaches for eyelid/eyelash segmentation, some embodiments may employ a two-stage technique that first applies coarse detection in a first stage and a fine-scale mask generation in a second stage. Coarse detection provides a fast technique which can be employed real-time to measure roughly how much the upper and lower eyelids and eyelashes cover the iris in the captured eye image. In particular, coarse detection is able to provide fast and efficient results by only testing the areas in the image that are most susceptible to eyelid/eyelash occlusion. As such, the first stage may advantageously be employed at the time of image capture to reject quickly the images with low iris image quality resulting from heavy eyelid/eyelash occlusion (Tosa, Para. [0082]). Tosa further teaches the exemplary embodiment first computes a set of training histograms from an unwrapped image from regions empirically believed to be free of occlusion. Test histograms are then be computed from neighborhoods of all pixels and tested for dissimilarity in relation to the training set histograms. Thus, this embodiment employs iris intensity modeling from regions free of eyelid/eyelash occlusion as a basis for determining whether other regions of the eye image 1001 correspond to eyelid/eyelash occlusion (Tosa, Para. [0084]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Sha with the teachings of Tosa by including an eyelash segmentation that categorizes the images into an eyelash occlusion class (i.e., heavy eyelid/eyelash occlusion) and a non-eyelash occlusion class based on a threshold and training the model using images from the non-eyelash occlusion class. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for determining eyelash occlusion in real-time, as recognized by Tosa.
Although Sha in view of Tosa teaches obtaining a final image without eyelashes (Sha, Para. [0097]), they do not explicitly teach “inputting the set of occluded images and the set of segmentation masks into a dual super-resolution learning network (DSRLN) for training to obtain a DSRLN model capable of segmenting the one or more occluding eyelashes in the set of occluded images to generate a set of refined segmentation masks, each member of the set of refined segmentation masks corresponding to a member of the set of occluded images” and “generating a set of inflated segmentation masks by processing each member of the set of refined segmentation masks to produce a corresponding member of the set of inflated segmentation masks, wherein the inflated segmentation masks correspond to fundus images belonging to the eyelash occlusion class”. However, in an analogous field of endeavor, Wang teaches a dual super-resolution learning framework for semantic segmentation that is trained using images and person instances labeled with keypoints (i.e., set of occluded images and set of segmentation masks) (Wang, pg. 3780). The qualitative segmentation results (i.e., refined segmentation masks) in Figure 8 demonstrate the effectiveness for structured objects (Wang, pg. 3779; Fig. 8). Following the same design for semantic segmentation, we append an extra upsampling model at the end of the existing network to predict the keypoint (i.e., generating a set of inflated segmentation masks) (Wang, pg. 3780).
Therefore, it would have been obvious to one having ordinary skill to modify the method of Sha in view of Tosa with the teachings of Wang by including inputting the occluded images and segmentation masks of Sha into the dual super-resolution learning network of Wang to generate refined segmentation masks and performing extra upsampling to generate a set of inflated segmentation masks. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for performing semantic segmentation while keeping high-resolution representation, improving performance without increasing inference speed, as recognized by Wang.
Although Sha in view of Tosa further in view of Wang teaches obtaining a final image without eyelashes using eyelash mask (i.e., inflated segmentation masks) and original ultra-wide-field fundus images in the image restoration model (Sha, Para. [0097]), they do not explicitly teach “each member of the set of final generated images comprising a member of the set of original ultra-wide-field fundus images with the corresponding one or more occluding eyelashes virtually removed by inpainting in a region of the corresponding member of the set of inflated segmentation masks” and “displaying the set of final generated images to a user”. However, in an analogous field of endeavor, Bermano teaches an inpainting method can be used to remove the eyelashes from the input images before reconstruction. When the eyelashes are removed, areas such as “holes” appear in the location of the removed eyelashes. The holes, which appear after removing the eyelashes, are filled by performing inpainting. The holes are filled by propagating color values from neighboring pixels which have not been removed (Bermano, Para. [0082]). The input/output device includes a display unit for displaying graphical user interfaces (Bermano, Para. [0195]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Sha in view of Tosa further in view of Wang with the teachings of Bermano by including virtually removing the eyelashes from the images using inpainting and displaying the output on a display unit. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for capturing aspects of the eye region without occlusions caused by concavities and eyelashes, as recognized by Bermano. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 2, Sha in view of Wang further in view of Bermano teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 1, wherein the DSRL model uses a dual-stream framework comprising a semantic segmentation super-resolution module, a single-image super-resolution module and a feature attention module (Wang, pg. 3776; Figure 2, the overview of the proposed DSRL framework, which includes three parts: Semantic Segmentation Super-Resolution (SSSR) branch (i.e., a semantic segmentation super-resolution module), Single Image Super-Resolution (SISR) branch (i.e., a single-image super-resolution module), and Feature Affinity (FA) module (i.e., a feature attention module)),
wherein the single-image super-resolution module shares a feature extractor with the semantic segmentation super-resolution module (Wang, pg. 3775, the high-resolution features of the SSSR stream are further enhanced by the fine-grained structural representation from the SISR stream with Feature Affinity (FA) module. Moreover, these two streams share the same feature extractor).
The proposed combination as well as the motivation for combining the Sha, Tosa, Wang and Bermano references presented in the rejection of Claim 1, apply to Claim 2 and are incorporated herein by reference. Thus, the method recited in Claim 2 is met by Sha in view of Tosa further in view of Wang and Bermano.
Regarding claim 3, Sha in view of Tosa further in view of Wang and Bermano teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 2, wherein the semantic segmentation super-resolution module further comprises an additional up-sampling step to generate the members of the set of inflated segmentation masks (Wang, pg. 3780, following the same design for semantic segmentation, we append an extra upsampling model at the end of the existing network to predict the keypoint).
The proposed combination as well as the motivation for combining the Sha, Tosa, Wang and Bermano references presented in the rejection of Claim 1, apply to Claim 3 and are incorporated herein by reference. Thus, the method recited in Claim 3 is met by Sha in view of Tosa further in view of Wang and Bermano.
Regarding claim 13, Sha in view of Tosa further in view of Wang and Bermano teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 1, wherein a member of the set of original ultra-wide-field fundus images is not added to the set of occluded images unless an occluded area comprising all areas of the member of the set of original ultra-wide-field fundus images occluded by the one or more occluding eyelashes exceeds a fixed minimum area of the member of the set of original ultra-wide-field fundus images (Tosa, Para. [0081], detecting edge points representing eyelids and/or eyelashes. Para. [0089], Step 1231 determines the number of edge pixels in the test region (i.e., member of the set of original ultra-wide-format fundus images). If the number of edge pixels (i.e., occluding eyelashes) in the test region exceeds a threshold (i.e., fixed minimum area), e.g., 10%, of the total pixels in the test region, the test region is marked as an eyelid occlusion (i.e., member of the set of original ultra-wide-format fundus images is added to the set of occluded images).
The proposed combination as well as the motivation for combining the Sha, Tosa, Wang and Bermano references presented in the rejection of Claim 1, apply to Claim 13 and are incorporated herein by reference. Thus, the method recited in Claim 13 is met by Sha in view of Tosa further in view of Wang and Bermano.
Regarding claim 14, Sha in view of Wang further in view of Bermano teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 2, wherein a member of the set of original ultra-wide-field fundus images is not added to the set of occluded images unless an occluded area comprising all areas of the member of the set of original ultra-wide-field fundus images occluded by the one or more occluding eyelashes exceeds a fixed minimum area of the member of the set of original ultra-wide-field fundus images (Tosa, Para. [0081], detecting edge points representing eyelids and/or eyelashes. Para. [0089], Step 1231 determines the number of edge pixels in the test region (i.e., member of the set of original ultra-wide-format fundus images). If the number of edge pixels (i.e., occluding eyelashes) in the test region exceeds a threshold (i.e., fixed minimum area), e.g., 10%, of the total pixels in the test region, the test region is marked as an eyelid occlusion (i.e., member of the set of original ultra-wide-format fundus images is added to the set of occluded images).
The proposed combination as well as the motivation for combining the Sha, Tosa, Wang and Bermano references presented in the rejection of Claim 1, apply to Claim 14 and are incorporated herein by reference. Thus, the method recited in Claim 14 is met by Sha in view of Tosa further in view of Wang and Bermano.
Regarding claim 15, Sha in view of Wang further in view of Bermano teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 3, wherein a member of the set of original ultra-wide-field fundus images is not added to the set of occluded images unless an occluded area comprising all areas of the member of the set of original ultra-wide-field fundus images occluded by the one or more occluding eyebrows exceeds a fixed minimum area of the member of the set of original ultra-wide-field fundus images (Tosa, Para. [0081], detecting edge points representing eyelids and/or eyelashes. Para. [0089], Step 1231 determines the number of edge pixels in the test region (i.e., member of the set of original ultra-wide-format fundus images). If the number of edge pixels (i.e., occluding eyelashes) in the test region exceeds a threshold (i.e., fixed minimum area), e.g., 10%, of the total pixels in the test region, the test region is marked as an eyelid occlusion (i.e., member of the set of original ultra-wide-format fundus images is added to the set of occluded images).
The proposed combination as well as the motivation for combining the Sha, Tosa, Wang and Bermano references presented in the rejection of Claim 1, apply to Claim 15 and are incorporated herein by reference. Thus, the method recited in Claim 15 is met by Sha in view of Tosa further in view of Wang and Bermano.
Claim 18 recites a system with elements corresponding to the steps recited in Claim 1. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Sha, Tosa, Wang and Bermano references, presented in rejection of Claim 1, apply to this claim. Finally, the combination of the Sha, Tosa, Wang and Bermano references discloses a processor, a memory, a fixed storage system, and input system, and an output system (Bermano, Para. [0192], the system includes a processor, a memory, a storage device, and an input/output interface).
Claims 4-6, 16 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Sha et al. (CN 116934609 A, machine translation used herein for mapping purposes) in view of Tosa et al. (US 2008/0253622 A1) further in view of Wang et al. (“Dual Super-Resolution Learning for Semantic Segmentation”) and Bermano et al. (US 2017/0024907 A1), as applied to claims 1-3, 13-15 and 18 above, and further in view of Chi et al. (“Fast Fourier Convolution”).
Regarding claim 4, Sha in view of Tosa further in view of Wang and Bermano teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 1, as described above.
Although Sha in view of Tosa further in view of Wang and Bermano teaches removing eyelashes from images using inpainting (Bermano, Para. [0082]), they do not explicitly teach “wherein the image restoration model uses a Fast Fourier Convolution process, the Fast Fourier Convolution process comprising a Fast Fourier Transform process generating a global branch output and a traditional convolution process generating a local branch output, the global branch output and the local branch output being merged to create a final output of the Fast Fourier Convolution process”. However, in an analogous field of endeavor, Chi teaches FFC is comprised of two inter-connected paths (i.e., branches are merged): a spatial (or local) path that conducts ordinary convolutions on a part of input feature channels (i.e., traditional convolution process generating a local branch output), and a spectral (or global) path that operates in the spectral domain (i.e., FFT process generating a global branch output) (Chi, pg. 3). The Fourier Unit (FU) transforms original spatial features into some spectral domain, conducts efficient global update on spectral data, and converts data back to the spatial format (see FU pseudocode in Figure 2, which shows performing FFT on input x) (i.e., FFT process generating a global branch output) (Chi, pg. 4).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Sha in view of Tosa further in view of Wang and Bermano with the teachings of Chi by including a Fast Fourier Convolution process that merges a traditional convolution process generating a local branch output with a Fast Fourier Transform process generating a global branch output. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for large receptive field in convolution for context-sensitive tasks, as recognized by Chi. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 5, Sha in view of Tosa further in view of Wang and Bermano teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 2, as described above.
Although Sha in view of Tosa further in view of Wang and Bermano teaches removing eyelashes from images using inpainting (Bermano, Para. [0082]), they do not explicitly teach “wherein the image restoration model uses a Fast Fourier Convolution process, the Fast Fourier Convolution process comprising a Fast Fourier Transform process generating a global branch output and a traditional convolution process generating a local branch output, the global branch output and the local branch output being merged to create a final output of the Fast Fourier Convolution process”. However, in an analogous field of endeavor, Chi teaches FFC is comprised of two inter-connected paths (i.e., branches are merged): a spatial (or local) path that conducts ordinary convolutions on a part of input feature channels (i.e., traditional convolution process generating a local branch output), and a spectral (or global) path that operates in the spectral domain (i.e., FFT process generating a global branch output) (Chi, pg. 3). The Fourier Unit (FU) transforms original spatial features into some spectral domain, conducts efficient global update on spectral data, and converts data back to the spatial format (see FU pseudocode in Figure 2, which shows performing FFT on input x) (i.e., FFT process generating a global branch output) (Chi, pg. 4).
The proposed combination as well as the motivation for combining the Sha, Tosa, Wang, Bermano and Chi references presented in the rejection of Claim 4, apply to Claim 5 and are incorporated herein by reference. Thus, the method recited in Claim 5 is met by Sha in view of Tosa further in view of Wang, Bermano and Chi.
Regarding claim 6, Sha in view of Tosa further in view of Wang and Bermano teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 3, as described above.
Although Sha in view of Tosa further in view of Wang and Bermano teaches removing eyelashes from images using inpainting (Bermano, Para. [0082]), they do not explicitly teach “wherein the image restoration model uses a Fast Fourier Convolution process, the Fast Fourier Convolution process comprising a Fast Fourier Transform process generating a global branch output and a traditional convolution process generating a local branch output, the global branch output and the local branch output being merged to create a final output of the Fast Fourier Convolution process”. However, in an analogous field of endeavor, Chi teaches FFC is comprised of two inter-connected paths (i.e., branches are merged): a spatial (or local) path that conducts ordinary convolutions on a part of input feature channels (i.e., traditional convolution process generating a local branch output), and a spectral (or global) path that operates in the spectral domain (i.e., FFT process generating a global branch output) (Chi, pg. 3). The Fourier Unit (FU) transforms original spatial features into some spectral domain, conducts efficient global update on spectral data, and converts data back to the spatial format (see FU pseudocode in Figure 2, which shows performing FFT on input x) (i.e., FFT process generating a global branch output) (Chi, pg. 4).
The proposed combination as well as the motivation for combining the Sha, Tosa, Wang, Bermano and Chi references presented in the rejection of Claim 4, apply to Claim 6 and are incorporated herein by reference. Thus, the method recited in Claim 6 is met by Sha in view of Tosa further in view of Wang, Bermano and Chi.
Regarding claim 16, Sha in view of Tosa further in view of Wang, Bermano and Chi teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 4, wherein a member of the set of original ultra-wide-field fundus images is not added to the set of occluded images unless an occluded area comprising all areas of the member of the set of original ultra-wide-format field images occluded by the one or more occluding eyelashes exceeds a fixed minimum area of the member of the set of original ultra-wide-field fundus images (Tosa, Para. [0081], detecting edge points representing eyelids and/or eyelashes. Para. [0089], Step 1231 determines the number of edge pixels in the test region (i.e., member of the set of original ultra-wide-format fundus images). If the number of edge pixels (i.e., occluding eyelashes) in the test region exceeds a threshold (i.e., fixed minimum area), e.g., 10%, of the total pixels in the test region, the test region is marked as an eyelid occlusion (i.e., member of the set of original ultra-wide-format fundus images is added to the set of occluded images).
The proposed combination as well as the motivation for combining the Sha, Tosa, Wang, Bermano and Chi references presented in the rejection of Claim 4, apply to Claim 16 and are incorporated herein by reference. Thus, the method recited in Claim 16 is met by Sha in view of Tosa further in view of Wang, Bermano and Chi.
Claim 19 recites a system with elements corresponding to the steps recited in Claim 4. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Sha, Tosa, Wang, Bermano and Chi references, presented in rejection of Claim 4, apply to this claim. Finally, the combination of the Sha, Tosa, Wang, Bermano and Chi references discloses a processor, a memory, a fixed storage system, and input system, and an output system (Bermano, Para. [0192], the system includes a processor, a memory, a storage device, and an input/output interface).
Claims 7-9, 17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Sha et al. (CN 116934609 A, machine translation used herein for mapping purposes) in view of Tosa et al. (US 2008/0253622 A1) further in view of Wang et al. (“Dual Super-Resolution Learning for Semantic Segmentation”) and Bermano et al. (US 2017/0024907 A1), as applied to claims 1-3, 13-15 and 18 above, and further in view of Xu et al. (US 2025/0259264 A1, filed February 13, 2024).
Regarding claim 7, Sha in view of Wang further in view of Bermano teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 1,
wherein the set of inflated segmentation masks is generated for fundus images categorized into the eyelash occlusion class (Sha, Para. [0085], traverse all points on the center line of the eyelash to be generated, construct Gaussian heatmaps with these points as the center points in turn and superimpose them, and then perform Gaussian blur to obtain the eyelash mask image (i.e., segmentation mask for fundus image with eyelash occlusion). (Wang, pg. 3780, we append an extra upsampling model at the end of the existing network to predict the keypoint (i.e., generating a set of inflated segmentation masks)).
Although Sha in view of Tosa further in view of Wang and Bermano teaches performing extra upsampling to generate the inflated segmentation masks (Wang, pg. 3780), they do not explicitly teach “selecting a convolutional kernel size of appropriate size for the resolution of the members of the set of original ultra-wide-field fundus images” and “using the convolutional kernel size as an inflation size guide when generating the set of inflated segmentation masks”. However, in an analogous field of endeavor, Xu teaches the scaling of the input (i.e., segmentation masks) is further based on a convolutional kernel size selected based on a target resolution for the input (i.e., resolution of the members of the set of fundus images) (Xu, Para. [0054]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Sha in view of Tosa further in view of Wang and Bermano with the teachings of Xu by including selecting a convolutional kernel based on a target resolution and scaling the segmentation masks based on the convolutional kernel size. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for efficient scaling of inputs using machine learning models, as recognized by Xu. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 8, Sha in view of Tosa further in view of Wang and Bermano teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 2, as described above.
Although Sha in view of Tosa further in view of Wang and Bermano teaches performing extra upsampling to generate the inflated segmentation masks (Wang, pg. 3780), they do not explicitly teach “selecting a convolutional kernel size of appropriate size for the resolution of the members of the set of original ultra-wide-field fundus images” and “using the convolutional kernel size as an inflation size guide when generating the set of inflated segmentation masks”. However, in an analogous field of endeavor, Xu teaches the scaling of the input (i.e., segmentation masks) is further based on a convolutional kernel size selected based on a target resolution for the input (i.e., resolution of the members of the set of fundus images) (Xu, Para. [0054]).
The proposed combination as well as the motivation for combining the Sha, Tosa, Wang, Bermano and Xu references presented in the rejection of Claim 7, apply to Claim 8 and are incorporated herein by reference. Thus, the method recited in Claim 8 is met by Sha in view of Tosa further in view of Wang, Bermano and Xu.
Regarding claim 9, Sha in view of Tosa further in view of Wang and Bermano teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 3, as described above.
Although Sha in view of Tosa further in view of Wang and Bermano teaches performing extra upsampling to generate the inflated segmentation masks (Wang, pg. 3780), they do not explicitly teach “selecting a convolutional kernel size of appropriate size for the resolution of the members of the set of original ultra-wide-field fundus images” and “using the convolutional kernel size as an inflation size guide when generating the set of inflated segmentation masks”. However, in an analogous field of endeavor, Xu teaches the scaling of the input (i.e., segmentation masks) is further based on a convolutional kernel size selected based on a target resolution for the input (i.e., resolution of the members of the set of fundus images) (Xu, Para. [0054]).
The proposed combination as well as the motivation for combining the Sha, Tosa, Wang, Bermano and Xu references presented in the rejection of Claim 7, apply to Claim 9 and are incorporated herein by reference. Thus, the method recited in Claim 9 is met by Sha in view of Tosa further in view of Wang, Bermano and Xu.
Regarding claim 17, Sha in view of Tosa further in view of Wang, Bermano and Xu teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 8, wherein a member of the set of original ultra-wide-format fundus images is not added to the set of occluded images unless an occluded area comprising all areas of the member of the set of original ultra-wide-format fundus images occluded by the one or more occluding eyebrows exceeds a fixed minimum area of the member of the set of original ultra-wide-format fundus images (Tosa, Para. [0081], detecting edge points representing eyelids and/or eyelashes. Para. [0089], Step 1231 determines the number of edge pixels in the test region (i.e., member of the set of original ultra-wide-format fundus images). If the number of edge pixels (i.e., occluding eyelashes) in the test region exceeds a threshold (i.e., fixed minimum area), e.g., 10%, of the total pixels in the test region, the test region is marked as an eyelid occlusion (i.e., member of the set of original ultra-wide-format fundus images is added to the set of occluded images).
The proposed combination as well as the motivation for combining the Sha, Tosa, Wang, Bermano and Xu references presented in the rejection of Claim 7, apply to Claim 17 and are incorporated herein by reference. Thus, the method recited in Claim 17 is met by Sha in view of Tosa further in view of Wang, Bermano and Xu.
Claim 20 recites a system with elements corresponding to the steps recited in Claim 7. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Sha, Tosa, Wang, Bermano and Xu references, presented in rejection of Claim 4, apply to this claim. Finally, the combination of the Sha, Tosa, Wang, Bermano and Xu references discloses a processor, a memory, a fixed storage system, and input system, and an output system (Bermano, Para. [0192], the system includes a processor, a memory, a storage device, and an input/output interface).
Claims 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over Sha et al. (CN 116934609 A, machine translation used herein for mapping purposes) in view of Tosa et al. (US 2008/0253622 A1) further in view of Wang et al. (“Dual Super-Resolution Learning for Semantic Segmentation”), Bermano et al. (US 2017/0024907 A1) and Chi et al. (“Fast Fourier Convolution”), as applied to claims 4-6, 16 and 19 above, and further in view of Xu et al. (US 2025/0259264 A1, filed February 13, 2024).
Regarding claim 10, Sha in view of Tosa further in view of Wang, Bermano and Chi teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 4, as described above.
Although Sha in view of Tosa further in view of Wang, Bermano and Chi teaches performing extra upsampling to generate the inflated segmentation masks (Wang, pg. 3780), they do not explicitly teach “selecting a convolutional kernel size of appropriate size for the resolution of the members of the set of original ultra-wide-format fundus images” and “using the convolutional kernel size as an inflation size guide when generating the set of inflated segmentation masks”. However, in an analogous field of endeavor, Xu teaches the scaling of the input (i.e., segmentation masks) is further based on a convolutional kernel size selected based on a target resolution for the input (i.e., resolution of the members of the set of fundus images) (Xu, Para. [0054]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Sha in view of Tosa further in view of Wang, Bermano and Chi with the teachings of Xu by including selecting a convolutional kernel based on a target resolution and scaling the segmentation masks based on the convolutional kernel size. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for efficient scaling of inputs using machine learning models, as recognized by Xu. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 11, Sha in view of Tosa further in view of Wang, Bermano and Chi teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 5, as described above.
Although Sha in view of Tosa further in view of Wang, Bermano and Chi teaches performing extra upsampling to generate the inflated segmentation masks (Wang, pg. 3780), they do not explicitly teach “selecting a convolutional kernel size of appropriate size for the resolution of the members of the set of original ultra-wide-format fundus images” and “using the convolutional kernel size as an inflation size guide when generating the set of inflated segmentation masks”. However, in an analogous field of endeavor, Xu teaches the scaling of the input (i.e., segmentation masks) is further based on a convolutional kernel size selected based on a target resolution for the input (i.e., resolution of the members of the set of fundus images) (Xu, Para. [0054]).
The proposed combination as well as the motivation for combining the Sha, Tosa, Wang, Bermano, Chi and Xu references presented in the rejection of Claim 10, apply to Claim 11 and are incorporated herein by reference. Thus, the method recited in Claim 11 is met by Sha in view of Tosa further in view of Wang, Bermano, Chi and Xu.
Regarding claim 12, Sha in view of Tosa further in view of Wang, Bermano and Chi teaches the method of virtual eyelash removal for ultra-wide-field fundus images of claim 6, as described above.
Although Sha in view of Tosa further in view of Wang, Bermano and Chi teaches performing extra upsampling to generate the inflated segmentation masks (Wang, pg. 3780), they do not explicitly teach “selecting a convolutional kernel size of appropriate size for the resolution of the members of the set of original ultra-wide-format fundus images” and “using the convolutional kernel size as an inflation size guide when generating the set of inflated segmentation masks”. However, in an analogous field of endeavor, Xu teaches the scaling of the input (i.e., segmentation masks) is further based on a convolutional kernel size selected based on a target resolution for the input (i.e., resolution of the members of the set of fundus images) (Xu, Para. [0054]).
The proposed combination as well as the motivation for combining the Sha, Tosa, Wang, Bermano, Chi and Xu references presented in the rejection of Claim 10, apply to Claim 12 and are incorporated herein by reference. Thus, the method recited in Claim 12 is met by Sha in view of Tosa further in view of Wang, Bermano, Chi and Xu.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Emma Rose Goebel whose telephone number is (703)756-5582. The examiner can normally be reached Monday - Friday 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Emma Rose Goebel/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights