DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-20 are pending.
Information Disclosure Statement
The Information Disclosure Statement filed on 02/05/2024 is in compliance with the provisions of 37 CFR 1.97 and have been considered. An initialed copy of the Form 1449 is enclosed herewith.
Specification
The title of the invention is not descriptive. Examiner suggests that title maybe changed to provide more description regarding the instant invention. Therefore, a new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 7-11, and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Cao et al., US 2023/0298135 in view of Kim et al., US 2024/0153045.
Regarding claim 1, Cao discloses a method for generating a super-resolution image model (super-resolution model refers to performing super-resolution processing on the first image through the non-blind super-resolution model to generate the super-resolved image of the first image, paragraph 126), comprising:
acquiring a first image with a first resolution and a second image with a second resolution, the first image corresponding to the second image (the size of the first image has first image resolution such as first size is 512 pix*512 pix then first image is acquired to perform image preprocessing on a first image to obtain/acquire a second image of size which has second resolution as cropping on the first image is performed to crop the first image into a plurality of second images of the first size, paragraphs 64, 68, 71-73);
generating a first super-resolution image with a first super resolution and a second super-resolution image with a second super resolution based on the first image according to an initial super-resolution image model (to generate a super-resolution result for the first image, it is necessary to obtain super-resolution results for different second images and then fuses said super-resolution images for different second images into the super-resolution result for the first image, wherein, the computer device performs super-resolution processing on the second image on the basis of the blur kernel prediction model to obtain super-resolved image, which is achieved by super-resolution model, which refers to performing super-resolution processing on the first image through the non-blind super-resolution model to generate the super-resolved image of the first image, paragraphs 100-102, 126-127);
transforming the first super-resolution image into a first frequency-domain representation (the first image is estimated on the basis of a frequency domain feature of the image in addition to being subjected to super-resolution processing. Since an image and a corresponding blur kernel have shape and structure correlations in respect of the frequency domain feature, compared with image spatial domain-based blur kernel estimation methods, the accuracy of blur kernel prediction can be improved and then the quality of a high-resolution image obtained after super-resolution processing is improved, paragraph 62);
transforming the second super-resolution image into a second frequency-domain representation (computer device performs frequency domain transformation on the second image obtained by cropping or expansion to obtain the spectrogram of the second image in addition to performing blur kernel prediction and super-resolution processing on basis of the second image. That is, the computer device performs blur kernel prediction on the basis of the spectrogram of the second image to obtain a blur kernel corresponding to the second image, performs super-resolution processing on the second image on the basis of the blur kernel to obtain a high-definition image corresponding to the second image, and then crops or stitches the obtained high-definition image to obtain a super-resolved image corresponding to the second image, paragraph 81);
and generating a trained super-resolution image model based on a loss (computer device calculates a model loss on the basis of a loss function. In one possible implementation, the loss function corresponding to the kernel prediction model or trained super-resolution model since kernel prediction model and the non-blind super-resolution model are combined to obtain a model for performing blind super-resolution processing on images, paragraphs 172, 125).
Cao fails to explicitly disclose generating an image model based on a loss between first frequency-domain representation and second frequency-domain representation and a reference frequency-domain representation of second image.
However, Kim teaches generating a trained image model based on a loss (training of a kernel estimation model using a loss of a training reference image, paragraph 60) between first frequency-domain representation and second frequency-domain representation (training sharp image 802 represents first frequency domain and second frequency domain since training sharp image 802 corresponds to a sharp version of the training input image 801 and is transformed into second frequency domain from first frequency domain of input image 801 which represents a first frequency image in a frequency domain, paragraph 60) and a reference frequency-domain representation of second image (third frequency image may be transformed into a training reference image in spatial frequency domain based on subtracting the blur kernel 811 from the frequency image (i.e. second image), paragraph 61) (and note that loss function 831 between the GT sharp image 830 and the training reference image 821 may be determined, wherein training sharp image 802 and a GT sharp image 830 may be the same, paragraphs 60-61).
Cao and Kim are combinable because they both are in the same field of endeavor dealing with imaging model and frequency domain representation of images.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Cao to incorporate the teachings of Kim for the benefit of efficiently performing kernel-based deblurring on the input image using the blur kernel to obtain a deconvolved image, and generating an output image by performing kernel-free deblurring based on the deconvolved image as taught by Kim at paragraph 4.
Regarding claim 2, Combination of Cao with Kim further teaches wherein generating the trained super-resolution image model comprises: determining a first frequency-domain difference between the first frequency-domain representation and the reference frequency-domain representation (Kim, training sharp image 802 represents first frequency domain and second frequency domain since training sharp image 802 corresponds to a sharp version of the training input image 801 and is transformed into second frequency domain from first frequency domain of input image 801 which represents a first frequency image in a frequency domain, paragraph 60);
determining a second frequency-domain difference between the second frequency-domain representation and the reference frequency-domain representation (Kim, third frequency image may be transformed into a training reference image in spatial frequency domain based on subtracting the blur kernel 811 from the frequency image (i.e. second image), paragraph 61);
determining a frequency-domain loss based on the first frequency-domain difference and the second frequency-domain difference (Kim, loss function 831 between the GT sharp image 830 and the training reference image 821 may be determined, wherein training sharp image 802 and a GT sharp image 830 may be the same, paragraphs 60-61);
and training the initial super-resolution image model (Cao, the loss function corresponding to the kernel prediction model or trained super-resolution model since kernel prediction model and the non-blind super-resolution model are combined to obtain a model for performing blind super-resolution processing on images, paragraphs 172, 125) based on the frequency-domain loss (Kim, the kernel estimation model 810 may be trained to reduce the loss 831, paragraph 61).
Cao and Kim are combinable because they both are in the same field of endeavor dealing with imaging model and frequency domain representation of images.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Cao to incorporate the teachings of Kim for the benefit of efficiently performing kernel-based deblurring on the input image using the blur kernel to obtain a deconvolved image, and generating an output image by performing kernel-free deblurring based on the deconvolved image as taught by Kim at paragraph 4.
Regarding claim 7, Combination of Cao with Kim further teaches wherein transforming the first super-resolution image into the first frequency-domain representation comprises: extracting a first feature map of the first super-resolution image (Cao, the spectral feature of the first image is extracted and then transposed convolution is performed to obtain the corresponding blur kernel, thereby achieving prediction of the blur kernel. The kernel prediction model and the non-blind super-resolution model are combined to achieve super-resolution processing on the first image, paragraph 128); determining a feature vector of each pixel in the first super-resolution image in the first feature map to obtain a set of first feature vectors (Cao, perform convolution processing on the spectrogram through the at least one convolution layer in the kernel prediction model to obtain a feature vector of the spectrogram and performing convolution processing on the spectrogram through the seven convolution modules 901 to obtain the feature vector corresponding to the spectrogram to output the blur kernel, paragraphs 120-123); and performing Fourier transform on the set of first feature vectors to obtain a set of sub-frequency-domain representations as the first frequency-domain representation (Cao, the spectral feature of the first image is obtained by performing Fourier transform on the image wherein, the spectral features obtained by fast Fourier transform are more correlated with the frequency domain features of the blur kernels and the computer device performs discrete fast Fourier transform on the second image obtain the spectrogram, paragraphs 38, 54, 82).
Cao and Kim are combinable because they both are in the same field of endeavor dealing with imaging model and frequency domain representation of images.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Cao to incorporate the teachings of Kim for the benefit of efficiently performing kernel-based deblurring on the input image using the blur kernel to obtain a deconvolved image, and generating an output image by performing kernel-free deblurring based on the deconvolved image as taught by Kim at paragraph 4.
Regarding claim 8, Combination of Cao with Kim further teaches acquiring the second image as a real value image (Kim, training sharp image 802 may be transformed into a second frequency image 804 in the frequency domain. The training sharp image 802 may be a sharp image corresponding to a sharp version of the training input image 801, or may be a temporary sharp version generated through a temporary deblurring task on the training input image 801, paragraph 61); and reducing the second resolution of the second image to obtain the first image (Cao, a second image of size which has second resolution as cropping on the first image is performed to crop the first image into a plurality of second images of the first size, paragraphs 71-73 and to generate a super-resolution result for the first image, it is necessary to obtain super-resolution results for different second images and then fuses said super-resolution images for different second images into the super-resolution result for the first image, paragraphs 100-102, 126-127).
Cao and Kim are combinable because they both are in the same field of endeavor dealing with imaging model and frequency domain representation of images.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Cao to incorporate the teachings of Kim for the benefit of efficiently performing kernel-based deblurring on the input image using the blur kernel to obtain a deconvolved image, and generating an output image by performing kernel-free deblurring based on the deconvolved image as taught by Kim at paragraph 4.
Regarding claim 9, Combination of Cao with Kim further teaches receiving an access request for a stored video with a third resolution from an electronic device (Cao, receiving request to have the image super-resolution method can also be applied to quality enhancement for video pictures with a low resolution (i.e., third resolution) from external electronic device of fig. 16 (computer device 1600 may be connected to a network 1612 through a network interface unit 1611 connected onto the system bus 1605, or may be connected to another type of network or a remote computer system (not shown) through a network interface unit 1611, paragraph 248), paragraph 43); determining a target resolution and a target frame rate corresponding to the access request and generating a target video with the target resolution and the target frame rate based on the video, the target resolution, and the target frame rate and according to the trained super-resolution image model (Cao, for a low-definition video, a terminal performs frequency domain transformation on the picture of each video frame (or extracts some video frames), performs prediction to obtain a corresponding blur kernel, and then performs super-resolution processing on each video frame, thereby improving the quality of the entire video to obtain a high-definition video. Or, in recent popular short video and live streaming applications, after a video is generated, a video sending terminal may compress a video file, making the image quality lowered, and then after receiving a video code stream, a receiving terminal can use the image super-resolution method to restore the resolution of a video picture, thereby reducing the transmission bandwidth and the storage pressure, and simultaneously ensuring the high resolution of the video picture when the video is played, paragraph 43).
Cao and Kim are combinable because they both are in the same field of endeavor dealing with imaging model and frequency domain representation of images.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Cao to incorporate the teachings of Kim for the benefit of efficiently performing kernel-based deblurring on the input image using the blur kernel to obtain a deconvolved image, and generating an output image by performing kernel-free deblurring based on the deconvolved image as taught by Kim at paragraph 4.
Regarding claim 10, Cao further discloses an electronic device, comprising: at least one processor; and a memory coupled to the at least one processor, the memory having instructions stored therein that, when executed by the at least one processor, cause the electronic device to perform actions (computer device 1600 includes a central processing unit (CPU) 1601, a system memory 1604 including a random access memory (RAM) 1602 and a read only memory (ROM) 1603, wherein, memory (non-transitory computer-readable storage medium) further includes at least one program, and the at least one program is stored in the memory and configured to be executed by one or more processors (processing circuitry), to implement the image super-resolution method, paragraphs 245, 249, 253) comprising: Rest of the claim recites similar features as claim 1 and thus is rejected on the same rationale.
Regarding claim 11, is an apparatus version of claim 2 reciting similar features and thus is rejected on the same rationale.
Regarding claim 16, is an apparatus version of claim 7 reciting similar features and thus is rejected on the same rationale.
Regarding claim 17, is an apparatus version of claim 8 reciting similar features and thus is rejected on the same rationale.
Regarding claim 18, is an apparatus version of claim 9 reciting similar features and thus is rejected on the same rationale.
Regarding claim 19, which recites a computer program product tangibly stored on a non-transitory computer-readable medium version of claim 1, see rationale as applied above. Note that computer program product tangibly stored on a non-transitory computer-readable medium is taught by Cao in paragraphs 249-253.
Regarding claim 20, which recites a computer program product tangibly stored on a non-transitory computer-readable medium version of claim 2, see rationale as applied above. Note that computer program product tangibly stored on a non-transitory computer-readable medium is taught by Cao in paragraphs 249-253.
Allowable Subject Matter
Claims 3-6 and 12-15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: closest cited prior arts fails to teach all the limitations of claim 3 such as “the method according to claim 2, wherein determining the frequency-domain loss comprises: determining a frequency-domain error based on the square of the first frequency-domain difference and the square of the second frequency-domain difference; determining an error weight based on the first frequency-domain difference and the second frequency-domain difference; and determining the frequency-domain loss based on the frequency-domain error and the error weight”;
and claim 4 such as “wherein generating the first super-resolution image and the second super-resolution image comprises: determining a first scaling factor based on the first resolution and the first super resolution; determining a second scaling factor based on the first resolution and the first super resolution; and determining a first pixel value and a second pixel value of each pixel in the first image at the first super resolution and the second super resolution respectively based on the first image, the first scaling factor, and the second scaling factor and according to the initial super-resolution image model, to obtain the first super-resolution image and the second super-resolution image”.
Claims 5-6 are further dependent upon claim 4.
Claim 12 is an apparatus version of claim 3 reciting similar features.
Claim 13 is an apparatus version of claim 4 reciting similar features.
Claims 14-15 are further dependent upon claim 13.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Li et al., US 2026/0024162
Lee et al., US 2024/0196160
Nagumo, US 2008/0175519
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAWANDEEP DHINGRA whose telephone number is (571) 270-1231. The examiner can normally be reached 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abderrahim Merouan can be reached at (571) 270-5254. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAWAN DHINGRA/Examiner, Art Unit 2683
/ABDERRAHIM MEROUAN/Supervisory Patent Examiner, Art Unit 2683
Prosecution Insights