Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on March 4, 2024 and September 5, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant's arguments filed 3/27/2026 have been fully considered but they are not persuasive.
Applicant argues that the prior arts of record Tang et. al. and Shajkofci et. al. do not disclose the feature of "generating training data comprising pairs of images, each pair of images
comprising a clean source image and a degraded source image by, for each clean source image, generating a corresponding noisy image by adding spatially invariant noise to the clean source image, and blending the noisy image with the clean source image according to varying intensity levels defined by a spatially variant mask to obtain the degraded image". However, Examiner maintains that the prior art of record in combination successfully captures the essence and solution of this feature in producing noise needed to train the neural network. Under broadest reasonable interpretation, in Shajkofci et al. (which assumes two independent separate blur and noise steps) the spatially variant effect is result of point spread function (PSF), a feature of the optical system, which is deterministic and physics-driven blurring. Furthermore, Figures 2 and 4, and highlights, in the description of Figure 2 noise is added to the cleaned image "From a large library of sharp microscopy images, small patches are created, blurred with a PSF generated from random parameters, and degraded with a Poisson- Gaussian noise mixture". The spatially variant noise is created from the PSF, which when invariant Poisson-Gaussian noise mixture is added, becomes spatially invariant noise. Thus, the solution and result, despite the steps taken, is the same as the claimed invention. Therefore, the rejections by the prior arts of record is maintained.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 9-11, 18-19 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Tang et. al. (US Patent 2022/0148130 A1).
Regarding claim 9, Tang et. al. discloses (Fig. 2) a computer implemented super resolution imaging method (Fig. 23) for generating a higher resolution image with reduced noise (i.e., super-resolved image, 2330) from a degraded lower resolution image that includes noise (i.e., downsample image plus random noise, 2310), comprising the steps of:
PNG
media_image1.png
592
842
media_image1.png
Greyscale
PNG
media_image2.png
794
688
media_image2.png
Greyscale
training a first neural network and using the trained first neural network to obtain a pixel level degradation map from the degraded lower resolution image by generating training data comprising pairs of images, each pair of images comprising a clean source image and a degraded source image by, for each clean source image, generating a corresponding noisy image by adding spatially invariant noise to the clean source image, and blending the noisy image with the clean source image according to varying intensity levels defined by a spatially variant mask to obtain the degraded image; using the training data to train the neural network by inputting each degraded source image to the neural network and extracting a degradation map from the degraded source image such that when the degradation map is applied to its corresponding clean source image the loss between the degraded source image and its corresponding clean source image after the degradation map is applied is minimised, wherein each degraded source image is a lower resolution version of its corresponding clean source image (2310 and 2320 on figure 23, [0285]-[0293]); and obtaining a feature map of the degraded lower resolution image, inputting the feature map to a second trained neural network to perform a pixel-wise feature modulation based on the pixel level degradation map to generate the higher resolution image with reduced noise (2330 on figure 23, [0284]-[0293]).
The same argument holds for claim 18.
Regarding claim 10, Tang et. al. discloses the method according to claim 9, wherein the second trained neural network comprises a plurality of spatial feature transformation blocks configured to transform the feature map based on the degradation map (figures 15, 20-22).
Regarding claim 11, Tang et. al. discloses the method according to claim 10, wherein each spatial feature transformation block comprises convolutional layers having a 1X1 filter size (figures 15, 20-22).
Regarding claim 19, Tang et. al. discloses the apparatus according to claim 18, wherein the second trained neural network comprises a plurality of spatial feature transformation blocks comprising convolutional layers having a 1X1 filter size configured to transform the feature map based on the degradation map (figures 15, 20-22).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-8, 12-17 are rejected under 35 U.S.C. 103 as being unpatentable over Tang et. al. (US Patent 2022/0148130 A1) in view of Shajkofci et. al. (Shajkofci, A., & Liebling, M. (2020). Spatially-Variant CNN-Based Point Spread Function Estimation for Blind Deconvolution and Depth Estimation in Optical Microscopy. IEEE Transactions on Image Processing, 29, 5848-5861.)
Regarding claim 1, Tang et. al. discloses a method of training a neural network to extract a degradation map from a degraded image, comprising:
generating training data (2310 on figure 23, [0285]-[0291]) comprising pairs of images, each pair of images comprising a clean source image and a degraded source image by, for each clean source image, generating a corresponding noisy image by adding spatially invariant noise to the clean source image, and blending the noisy image with the clean source image according to varying intensity levels defined by a spatially variant mask to obtain the degraded image (2310 on figure 23, [0285]-[0291] together with [0252]-[0257]); using the training data to train the neural network by inputting each degraded source image to the neural network and extracting a degradation map (2320 and 2330 on figure 23, [0292], [0293], [0046] which discloses a step of generating a noise map and inputting this into a CNN (SR subnetwork). The two steps of a NE subnetwork and a SR subnetwork are also discussed in paragraphs [0254] and [0255]) from the degraded source image such that when the degradation map is applied to its corresponding clean source image the loss between the degraded source image and its corresponding clean source image after the degradation map is applied is minimised (2340, 2350 and 2360 on figure 23, [0294]-[0297]). However, Tang et. al. does not fully disclose the use of a spatially variant mask to generate the training data.
Shajkofci et. al. teaches the method of using a convolutional neural network (CNN) to estimate the parameters of a spatially-variant Point Spread Function (PSF) model from images obtained from optical microscopy. Using an adequate PSF in a deconvolution algorithm can restore detail in the image to reverse local degradation . Furthermore, Shajkofci et. al. demonstrates the spatially variant masking step and deconvolution steps in Figure 2 and 4.
PNG
media_image3.png
424
838
media_image3.png
Greyscale
The spatially variant masking step and creating a clean high-resolution image by adding spatially variant noise to the training data or using a spatially variant mask to obtain the degraded images is a key distinction for the claimed invention. Thus, it would have been obvious for a person having ordinary skill in the art (PHOSITA) prior to the effective filing date of the claimed invention to combine the teachings of Tang et. al. and the teachings of Shajkofci et. al. so that the training data is robust with the degraded images obtained from a spatially variant mask. The motivation behind this lies in improving the performance of the super-resolution network denoising of real-world low-resolution images simply acquired from the downsampler function.
The same argument holds for claims 12 (a non-transitory computer-readable medium storing a program comprising instructions which, when the program is executed by the computer, cause the computer to carry out the method of claim 1) and 13 (an apparatus for extracting a degradation map from a degraded image).
PNG
media_image4.png
864
422
media_image4.png
Greyscale
Regarding claim 2, Tang et. al. further teaches the method according to claim 1, wherein the degradation map is a pixel level degradation map (2320 and 2330 on figure 23, [0292], [0293]).
Regarding claim 3 and 4, Tang et. al. further teaches the method according to claim 1, wherein, for each clean source image, the spatially variant mask is generated to correspond to a brightness distribution in the clean source image and includes higher values in areas with lower brightness ([0255], only refers to a scaling factor, but the skilled person would likely create a mask indicating that the noise is higher in the low brightness areas, as in the claims 3 and 4).
Regarding claim 5, Tang et. al. further teaches the method according to claim 1, wherein the neural network comprises an extraction convolutional layer configured to extract features from each degraded source image, a plurality of degradation feature extraction blocks configured to process the features to extract spatially invariant degradation features, and a plurality of mapping convolutional layers configured to map the remaining degradation features to a plurality of channels to generate the degradation map (figures 15, 20-22).
Regarding claim 6, Tang et. al. further teaches wherein each degraded source image is a lower resolution version of its corresponding clean source image (Figure 23, [0285]).
PNG
media_image5.png
430
528
media_image5.png
Greyscale
Regarding claim 7, Tang et. al. further teaches the method according to claim 6, wherein, to generate the training data, each clean source image is downsampled to a lower resolution before the spatially invariant noise is added (Figure 23, [0285]).
Regarding claim 8, Tang et. al. further teaches the method according to claim 6, wherein the training of the neural network comprises downsampling each clean source image to the same resolution as its corresponding degraded source image, and applying the generated degradation map to the downsampled clean source image, wherein the neural network generates the degradation map to minimise the loss between the degraded source image and the downsampled clean source image after application of the generated degradation map (Figure 23, [0284]).
PNG
media_image6.png
390
512
media_image6.png
Greyscale
Regarding claim 14, Tang et. al. further teaches the apparatus according to claim 13, wherein, in the training data, each degraded source image is generated by adding spatially invariant noise to its corresponding clean source image to obtain a noisy image, and blending the noisy image with the clean source image according to varying intensity levels defined by a spatially variant mask (2310 on figure 23, [0285]-[0291] together with [0252]-[0257]).
Regarding claim 15, Tang et. al. further teaches the apparatus according to claim 13, wherein, in the training data, the spatially variant noise varies corresponding to a brightness distribution in the clean source image to have higher values in areas with lower brightness (2310 on figure 23, [0285]-[0291] together with [0252]-[0257]).
Regarding claim 16, Tang et. al. further teaches the apparatus according to claim 13, wherein the neural network comprises an extraction convolutional layer configured to extract features from each degraded source image, a plurality of degradation feature extraction blocks configured to process the features to extract spatially invariant degradation features, and a plurality of mapping convolutional layers configured to map the remaining degradation features to a plurality of channels to generate the degradation map (figures 15, 20-22).
Regarding claim 17, Tang et. al. further teaches the apparatus according to claim 13, wherein each degraded source image is a downsampled version of the clean source image (figure 23, [0284]).
Response to Amendment
Examiner acknowledges the amendments made to the specification, drawings, and claims 13 and 18 to overcome the 112(f) and 112(a) rejections previously noted in the non-final rejection.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JESSICA YIFANG LIN whose telephone number is (571)272-6435. The examiner can normally be reached M-F 7:00am-6:15pm, with optional day off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at 571-272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JESSICA YIFANG LIN/Examiner, Art Unit 2668 April 4, 2026
/VU LE/Supervisory Patent Examiner, Art Unit 2668