DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
A Preliminary Amendment was entered 03/11/2025 with pending claims 1-12 in which amendments were entered for claims 1-2, 4-5, 7-8, 10 and a minor amendment to the specification.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/13/2024, 04/23/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are considered by examiner.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-3, 7-9 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Yang et al (US 2023/0022753).
Regarding Claim 1, Yang et al teach a method of restoring a video (process of using device 101 to stabilize and deblur image data; Fig 1, 10 and ¶ [0033]-[0036], [0039]), the method comprising:
obtaining a plurality of blurred images due to object motion and camera motion of an object captured in the video (device 101 performs image data augmentation with obtaining video image data, with data containing motion blur; Fig 1, 2 and ¶ [0033]-[0036], [0039], [0100]-[0103]);
generating a first kernel (kernel data 1008; Fig 10 and ¶ [0102]) comprising degradation information on the plurality of blurred images, optical flow information on the plurality of blurred images, and first motion information on each of the plurality of blurred images comprised in the video (blur kernels 1008 are generated representing motion data based on object movement to create optical flow between frames, thereby including blur data generated from the object and camera movement based on using long frame capture data; ¶ [0051]-[[0053], [0101]-[0102]);
generating dynamic filtering information to filter the plurality of blurred images (refined (deconvolution) blur kernels 1012 are generated using a kernel refine network 1010, where the blur kernels are a spatially varying set of filter kernels to sharpen an image; ¶ [0102]-[0103]), based on the plurality of blurred images, the first motion information, the optical flow information, and the first kernel (the blur kernels 1008 are based on the motion identified in the frames representing the camera and object motion as well as the optical flow motion detected between the frames of the video; ¶ [0033]-[0036], [0102]-[0103]); and
restoring a target image to be restored among the plurality of blurred images, based on the dynamic filtering information (restoration performed using dynamic filtering including FRMA and occlusion mask data analysis, specification pg 15) and the plurality of blurred images (the kernels are refined 1012, with further refinement performed with occlusion analysis to recover a sharp image frame 1020; ¶ [0103]-[0106]).
Regarding Claim 2, Yang et al teach the method of claim 1 (as described above), wherein the generating the dynamic filtering information comprises:
generating second motion information to adjust the optical flow information, based on the plurality of blurred images and the first motion information (motion is determined between a second a third frame to determine a second motion point to generate the optical flow data; ¶ [0102]);
adjusting the optical flow information, based on the second motion information, the optical flow information, and the first kernel (the optical flow data can be interpolated between the optical flow data points of the blur kernel 1008; ¶ [0102]); and
generating distorted information on how distorted each of the remaining images excluding the target image is compared with respect to the target image, based on the second motion information, the optical flow information, and the first kernel (a blur kernel 1008 approximate the motion direction for the long frame 1002, which is based on the short frame 1003 and a second short frame 1004, understood as a second blur kernel generated between the long frame 1002 and second short frame 1004, which may then be compared to a ground truth sharp image to determine loss (distortion); ¶ [0102]-[0103]),
wherein the dynamic filtering information comprises adjusted optical flow information and the distorted information (the refined kernels 1012 are based on the blur kernels 1008 and the associated optical flow data at each pixel, which undergo deconvolution (filtering including an occlusion mask) to sharpen the pixel; ¶ [0102]-[0106]).
Regarding Claim 3, Yang et al teach the method of claim 2 (as described above), wherein the restoring the target image comprises restoring the target image by filtering the plurality of blurred images, based on the adjusted optical flow information and the distorted information (the sharp image 1020 is generated based on an occlusion-aware deconvolution operation 1016 by applying the refined kernels 1012 to the long frame 1002 and the use of the occlusion mask 1014 to short frames 1003, 1004, followed by a static background mask to the long frame to render the background 1018; ¶ [0104]-[0106]).
Regarding Claim 7, Yang et al teach an electronic device (electronic device101; Fig 1 and ¶ [0038]) comprising: a memory comprising instructions (memory 130 with commands and data for device 101; Fig 1 and ¶ [0040]); and a processor electrically connected to the memory and configured to execute the instructions (processor 120 executes commands stored on memory associated with program 140 for image data analysis; Fig 1 and ¶ [0039]-[0040]), wherein the instructions, when executed by the processor, cause the electronic device to: perform steps identical to claim 1 (as described above).
Regarding Claim 8, Yang et al teach the electronic device of claim 7 (as described above), with further steps identical to claim 2 (as described above).
Regarding Claim 9, Yang et al teach the electronic device of claim 8 (as described above), with further steps identical to claim 3 (as described above).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4-6, 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al (US 2023/0022753) in view of Yu et al (Split-Attention Multiframe Alignment Network for Image Restoration).
Regarding Claim 4, Yang et al teach the method of claim 3 (as described above), including restoring the target image (restoring the image to a sharp image frame; ¶ [0035], [0039], [0051]-[0052], [0100]) and warping the objects comprised in each of the plurality of blurred images, based on the adjusted optical flow information (optical flow data generated for the object motion identified between the short frames is used to generate feature maps, which are aligned to the long frame through warping operations 214-215; ¶ [0055]-[0057]).
Yang et al does not explicitly teach generating a second kernel to filter warped images obtained by warping the objects, based on the distorted information; and restoring the target image by filtering the warped images using the second kernel.
Yu et al is analogous art pertinent to the technological problem addressed in the current application and teaches generating a second kernel to filter warped images obtained by warping the objects, based on the distorted information (the target image is warped toward the reference image using a ground truth and a kernel dilation operation (with second kernel) is used to filter the original warping result, which also considers the optical flow; Fig 5, 7 and IV.A.1. Ghost in Occlusion Regions ¶ 2, IV.B. Warping Repetition Detection Module ¶ 2); and
restoring the target image by filtering the warped images using the second kernel (the final alignment result is generated by replacing the pixels under the ghost mask with the pixels in the reference image; Fig 7 and IV.B. Warping Repetition Detection Module ¶ 2).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Yang et al with Yu et al including generating a second kernel to filter warped images obtained by warping the objects, based on the distorted information; and restoring the target image by filtering the warped images using the second kernel. By using the multiple kernels, a ghosting effect is successfully eliminated thereby generating an improved alignment between the foreground and background and resulting in improved super-resolution for image restoration, as recognized by Yu et al (I. Introduction ¶ 9).
Regarding Claim 5, Yang et al in view of Yu et al teach the method of claim 4 (as described above), wherein the restoring the target image by filtering the warped images using the second kernel comprises restoring low-frequency components of the target image (Yu et al, the morphological erosion operation with the second kernel replaces the noise caused by the ghost mask; IV.B. Warping Repetition Detection Module ¶ 2).
Regarding Claim 6, Yang et al in view of Yu et al teach the method of claim 5 (as described above), further comprising restoring high-frequency components of the target image, based on the distorted information (Yu et al, the components remaining in the final alignment image are those representing a signal (high-frequency components) from the reference image and original warping result, thereby removing the ghost pixels (low-frequency, noise, components); IV.B. Warping Repetition Detection Module ¶ 2).
Regarding Claim 10, Yang et al teach the electronic device of claim 9 (as described above), with further steps identical to claim 4 (as described above).
Regarding Claim 11, Yang et al teach the electronic device of claim 10 (as described above), with further steps identical to claim 5 (as described above).
Regarding Claim 12, Yang et al teach the electronic device of claim 11 (as described above), with further steps identical to claim 6 (as described above).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Lian et al (An image deblurring method using improved U-Net model based on multilayer fusion and attention mechanism) teach a method and system for image deblurring based on a U-net model with dense multi-receptive field attention block and multilayer feature fusion to extract details within the image using a FFT and frequency reconstruction loss.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATHLEEN M BROUGHTON whose telephone number is (571)270-7380. The examiner can normally be reached Monday-Friday 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATHLEEN M BROUGHTON/Primary Examiner, Art Unit 2661
Prosecution Insights