DETAILED ACTION
This Office action is in response to the Application filed on December 10, 2024, which claims priority to Korean Patent Application No. 10-2023-0182852, filed on December 15, 2023. An action on the merits follows. Claims 1-20 are pending on the application.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Priority
Acknowledgment is made of applicant's claim for foreign priority based on an application filed in Korea on December 15, 2023. It is noted, however, that applicant has not filed a certified copy of the Korean Patent Application No. 10-2023-0182852 application as required by 37 CFR 1.55.
Claim Objections
Claims 1 and 17 are objected to because of the following informalities:
Claim 1 recites the limitation “an HR feature in which a certain frequency component corresponding to the feature map is restored” in lines 4-5 of the claim. However, the claimed “certain frequency component” term recited in line 4 of the claim is not defined by the claims.
Par. [0044-56] of specification of instant application indicate “restore a high-resolution (HR) image at a target magnification by restoring a high-frequency component… an HR feature in which a certain frequency component corresponding to the feature map mapped in operation 120 is restored… The ‘certain frequency component’ may, for example, correspond to a high-frequency component that is higher than a predetermined reference but is not necessarily limited thereto… the certain frequency component (e.g., the high-frequency component) corresponding to the feature map is restored… scaling factor and/or the bias factor may affect factors required for the high-frequency component to be restored”.
Therefore, based on above, for examination purposes the claimed “an HR feature in which a certain frequency component corresponding to the feature map is restored” in lines 4-5 of the claim will be interpreted as an HR feature in which a certain frequency component corresponding to the feature map is restored, wherein the certain frequency component corresponds to a high-frequency component”.
Claim 17 recites the limitation “an HR feature in which a certain frequency component corresponding to the feature map is restored” in lines 6-7 of the claim. However, the claimed “certain frequency component” term recited in lines 6-7 of the claim is not defined by the claims.
Par. [0044-56] of specification of instant application indicate “restore a high-resolution (HR) image at a target magnification by restoring a high-frequency component… an HR feature in which a certain frequency component corresponding to the feature map mapped in operation 120 is restored… The ‘certain frequency component’ may, for example, correspond to a high-frequency component that is higher than a predetermined reference but is not necessarily limited thereto… the certain frequency component (e.g., the high-frequency component) corresponding to the feature map is restored… scaling factor and/or the bias factor may affect factors required for the high-frequency component to be restored”.
Therefore, based on above, for examination purposes the claimed “an HR feature in which a certain frequency component corresponding to the feature map is restored” in lines 6-7 of the claim will be interpreted as an HR feature in which a certain frequency component corresponding to the feature map is restored, wherein the certain frequency component corresponds to a high-frequency component”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3-4, 6-7, and 11-15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 3 recites the limitation “a learned normal distribution from pixel values of the feature map” in lines 2-3 of the claim. However, it is not clear if the claimed “pixel values” recited in line 3 of claim 3 encompass embodiments corresponding to the claimed “pixel information” recited in line 6 of claim 1, or if the claimed “pixel values” recited in line 3 of claim 3 encompass embodiments corresponding to “pixel values” different from the claimed “pixel information” recited in line 6 of claim 1, for example, because the claimed “pixel information” recited in line 6 of claim 1 is not clearly defined by the claims. Therefore, the metes and bounds of the claim are not clearly set forth and the examiner cannot clearly determine which elements are encompassed by the claim language, which renders the claim indefinite.
Claim 4 is rejected by virtue of being dependent upon rejected claim 3.
Claim 6 recites the limitation “extracting a scaling factor and a bias factor from the feature map” in lines 2-3 of the claim. However, the claimed “scaling factor” and “ bias factor” terms are not defines buy the claims. Additionally, it is not clear if the claimed “scaling factor” is related to the claimed “target magnification” corresponding to the feature map” previously recited in claim 1, or if the claimed “scaling factor” is not related to the claimed “target magnification” corresponding to the feature map” previously recited in claim 1, for example, because the claimed “scaling factor” is not defined by the claims it cannot be clearly determined if the claimed “scaling factor” is related to the claimed “target magnification” corresponding to the feature map” previously recited in claim 1, or not. Therefore, the metes and bounds of the claim are not clearly set forth and the examiner cannot clearly determine which elements are encompassed by the claim language, which renders the claim indefinite.
Claim 7 is rejected by virtue of being dependent upon rejected claim 6.
Claim 11 recites the limitation “estimating frequency information from the feature map” in line 3 of the claim. However, it is not clear if the claimed “frequency information” recited in line 3 of claim 11 encompass embodiments corresponding to the claimed “certain frequency component” corresponding to the feature map previously recited in line 6 of claim 1, or if the claimed “frequency information” recited in line 3 of claim 11 encompass embodiments that do not correspond to the claimed “certain frequency component” corresponding to the feature map previously recited in line 6 of claim 1, for example, because the claimed “frequency component” and “frequency information” are not clearly defined by the claims. Therefore, the metes and bounds of the claim are not clearly set forth and the examiner cannot clearly determine which elements are encompassed by the claim language, which renders the claim indefinite.
Claim 12 is rejected by virtue of being dependent upon rejected claim 11.
Claim 13 recites the limitation “estimating pixel values of the HR image … and the pixel information of the HR image” in lines 3-5 of the claim. However, it is not clear if the claimed “pixel values” recited in line 2 of claim 13 encompass embodiments corresponding to the claimed “pixel information” recited in line 6 of claim 1, or if the claimed “pixel values” recited in line 2 of claim 13 encompass embodiments corresponding to “pixel values” different from the claimed “pixel information” recited in line 6 of claim 1, for example, because the claimed “pixel information” recited in line 6 of claim 1 is not clearly defined by the claims. Therefore, the metes and bounds of the claim are not clearly set forth and the examiner cannot clearly determine which elements are encompassed by the claim language, which renders the claim indefinite.
Claim 14 is rejected by virtue of being dependent upon rejected claim 13.
Claim 14 recites the limitation “estimating pixel values of the HR image… the pixel information of the HR image” in lines 2-4 of the claim. However, it is not clear if the claimed “pixel values” recited in line 2 of claim 14 encompass embodiments corresponding to the claimed “pixel information” recited in line 6 of claim 1, or if the claimed “pixel values” recited in line 2 of claim 14 encompass embodiments corresponding to “pixel values” different from the claimed “pixel information” recited in line 6 of claim 1, for example, because the claimed “pixel information” recited in line 6 of claim 1 is not clearly defined by the claims. Therefore, the metes and bounds of the claim are not clearly set forth and the examiner cannot clearly determine which elements are encompassed by the claim language, which renders the claim indefinite.
Claim 15 is rejected by virtue of being dependent upon rejected claim 14.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 5, 16-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over JIN et al. (KN Patent Application Publication No. 20230111885 A), hereafter referred to as JIN, in view of QIN et al. (CN Patent Application Publication No. 113920013 A), hereafter referred to as QIN.
Regarding claim 1, JIN discloses a processor-implemented (Pg. 2: methods and other systems for implementing the present disclosure, and a computer-readable recording medium in which a computer program for executing the method is stored; Pg. 3: image restoration system 1 May include an image restoration apparatus 100, a user terminal 200, a server 300; Pg. 4: image restoration apparatus… include… a processor… memory 130 May be connected to one or more processors 140… and may store codes which, when executed by the processor 140, cause the processor 140 to control the image restoration apparatus 100… processor 140 May be connected to the configuration of the image restoration apparatus 100 including the memory 130, and may execute at least one command stored in the memory 130 to overall control the operation of the image restoration apparatus 100 ; Pg. 13: present disclosure described above may be implemented in the form of a computer program that can be executed through various components on a computer) method (Pg. 1: image restoration method and apparatus; Pg. 2: image restoration method) with high-resolution (HR) image restoration (Pg. 1: an image restoration method and apparatus… restoring an image… reconstruct a high-resolution image; Pg. 2: image restoration method… include… generating a high-resolution image… reconstructing the high-resolution image; Pg. 3: restore (reconstruct) the high-resolution image; Pg. 4: server… controls an operation of the image restoration device 100 for a whole process of reconstructing the low-resolution image into a high-resolution image;), the method comprising:
mapping an input image with low resolution (LR) to a feature map of a latent domain (Pg. 1: Single image super-resolution (SISR)… SISR (hereinafter, referred to as SR) aims to reconstruct a high-resolution image in a degraded low-resolution image; Pg. 2: restoring an image to an arbitrary resolution in a continuous manner… image restoration method… include: obtaining a low-resolution image… obtaining a low-resolution image; estimating Fourier information of the low-resolution image based on a pre-trained image restoration algorithm, and deriving a color value corresponding to each coordinate of the low-resolution image at an arbitrary resolution; Pg. 4: server that acquires a low-resolution image… and controls an operation of the image restoration device 100 for a whole process of reconstructing the low-resolution image into a high-resolution image in a continuous manner; Pg. 5: processor… obtain a low-resolution image… the processor 140 May encode the low-resolution image to extract a latent feature vector, and may estimate a dominant frequency and a Fourier coefficient corresponding to each coordinate of the low-resolution image based on the latent feature vector… image restoration algorithm may be a learning model trained to output a color value mapped to each two-dimensional coordinate by querying, as an input, a dominant frequency and a Fourier coefficient derived from a local latent feature vector corresponding to each two-dimensional coordinate, based on each two-dimensional coordinate of the low-resolution image when the low-resolution image is input… in order to restore a low-resolution image to an arbitrary high-resolution image without expressing an image at a fixed resolution, a continuous representation of the image should be learned. Accordingly, the processor 140 May reconstruct an image at an arbitrary resolution by modeling the image as a function defined in a continuous area; Pg. 6: SR encoder may extract a feature map having the same height and width as the low-resolution image… the processor 140 May train the encoder with the local implicit neural network function representation through the super-resolution self-supervised operation to generate a continuous representation of the image. The local implicit neural network function representation is to represent an image as a latent code set dispersed in a spatial dimension… When a coordinate is given, the decoder may take coordinate information and query a local latent code around the coordinate as an input, and then predict an RGB value in a given coordinate as an output… since the coordinates are continuous, the local implicit neural network function may represent the low-resolution image as an arbitrary high resolution image… decoder may map the latent tensor and the local coordinates to a color value (RGB value); mapping an input image with low resolution (LR) to a feature map of a latent domain (e.g. image restoration method and apparatus include a super resolution (SR) encoder used to extract a feature map (i.e. mapping) having a same height and width as a low-resolution (LR) image when input, for example, including a processor programmed to encode the low-resolution image to extract a latent feature vector (i.e. mapping an input image with low resolution (LR) to a feature map of a latent domain), as indicated above), for example);
generating, using a model, an HR feature in which a certain frequency component corresponding to the feature map is restored[, wherein the certain frequency component corresponds to a high-frequency component] (Pg. 2: improve the accuracy of image restoration by estimating a dominant frequency for a natural image by being configured as a single network, and learning fine details while restoring an image to an arbitrary resolution in a continuous manner… estimate a dominant frequency and essential Fourier information for a natural image, so that an implicit neural network function for any resolution prioritizes high-frequency detail learning… image restoration method according to an embodiment of the disclosure may include: obtaining a low-resolution image; estimating Fourier information of the low-resolution image based on a pre-trained image restoration algorithm, and deriving a color value corresponding to each coordinate of the low-resolution image at an arbitrary resolution; and generating a high-resolution image of an arbitrary resolution based on a color value corresponding to each coordinate of the low-resolution image… by estimating the dominant frequency and essential Fourier information for the natural image, the implicit neural network function may prioritize the high-frequency detail learning, thereby reconstructing the high-resolution image at any resolution even in a significant scale factor; Pg. 3: FIG. 2 is a diagram illustrating an overview of a local texture estimator (LTE)… local texture estimator may estimate a dominant frequency and a corresponding Fourier coefficient for the natural image. The multilayer perceptron (MLP) may then restore (reconstruct) the high-resolution image to any resolution using the estimated essential Fourier information; Pg. 4: image restoration apparatus 100 May be implemented in the server 300… server 300 May be a server for operating the image restoration system 1 including the image restoration apparatus 100 or a server implementing a portion or all of the image restoration apparatus 100… a server that acquires a low-resolution image, learns a high-frequency component of the low-resolution image through a local texture estimator-based any-scale SR network, and controls an operation of the image restoration device 100 for a whole process of reconstructing the low-resolution image into a high-resolution image in a continuous manner; Pg. 5: processor 140 May obtain a low-resolution image, estimate Fourier information of the low-resolution image based on a pre-trained image reconstruction algorithm, and derive a color value corresponding to each coordinate of the low-resolution image at an arbitrary resolution (high resolution). Here, the Fourier information of the image may include at least the dominant frequency and amplitude of the image. In addition, in any resolution, each coordinate of the low-resolution image may refer to a coordinate representing the position of each pixel when it is assumed that the low-resolution image is represented as high-resolution… the processor 140 May encode the low-resolution image to extract a latent feature vector, and may estimate a dominant frequency and a Fourier coefficient corresponding to each coordinate of the low-resolution image based on the latent feature vector… derive a color value corresponding to each coordinate of the low-resolution image based on the dominant frequency and the Fourier coefficient… the pre-trained image restoration algorithm may be a learning model trained to output a color value mapped to each two-dimensional coordinate by querying, as an input, a dominant frequency and a Fourier coefficient derived from a local latent feature vector corresponding to each two-dimensional coordinate, based on each two-dimensional coordinate of the low-resolution image when the low-resolution image is input… generate a high-resolution image of an arbitrary resolution based on a color value corresponding to each coordinate of the low-resolution image; Pg. 6: SR encoder may extract a feature map having the same height and width as the low-resolution image. The local texture estimator can then retrieve the feature map at the encoder and estimate the main frequency and the corresponding Fourier coefficient for the natural image… neural network function may represent the low-resolution image as an arbitrary high resolution image; generating, using a model, an HR feature in which a certain frequency component corresponding to the feature map is restored, wherein the certain frequency component corresponds to a high-frequency component (e.g. image restoration method and apparatus include a super resolution (SR) encoder used to extract a feature map (i.e. the feature map) having a same height and width as a low-resolution (LR) image when input, for example, including a processor programmed to encode the low-resolution image to extract a latent feature vector (i.e. mapping an input image with low resolution (LR) to a feature map of a latent domain), by estimating a dominant frequency (i.e. a certain frequency component) and essential Fourier information for a natural image, such as an input image, for example, in which an implicit neural network function prioritizes (i.e. using a model, algorithm, etc.) high-frequency detail (i.e. HR feature) learning (i.e. generating, using a model, an HR feature in which a certain frequency component corresponding to the feature map is restored, wherein the certain frequency component corresponds to a high-frequency component), thereby reconstructing a high-resolution image at any resolution, as indicated above), for example); and
based on the HR feature and coordinate information and pixel information of an HR image at a target magnification, restoring the HR image at the target magnification corresponding to the feature map (Pg. 2: improve the accuracy of image restoration by estimating a dominant frequency for a natural image by being configured as a single network, and learning fine details while restoring an image to an arbitrary resolution in a continuous manner… estimate a dominant frequency and essential Fourier information for a natural image, so that an implicit neural network function for any resolution prioritizes high-frequency detail learning… image restoration method according to an embodiment of the disclosure may include: obtaining a low-resolution image; estimating Fourier information of the low-resolution image based on a pre-trained image restoration algorithm, and deriving a color value corresponding to each coordinate of the low-resolution image at an arbitrary resolution; and generating a high-resolution image of an arbitrary resolution based on a color value corresponding to each coordinate of the low-resolution image… by estimating the dominant frequency and essential Fourier information for the natural image, the implicit neural network function may prioritize the high-frequency detail learning, thereby reconstructing the high-resolution image at any resolution even in a significant scale factor; Pg. 5: processor 140 May obtain a low-resolution image, estimate Fourier information of the low-resolution image based on a pre-trained image reconstruction algorithm, and derive a color value corresponding to each coordinate of the low-resolution image at an arbitrary resolution (high resolution). Here, the Fourier information of the image may include at least the dominant frequency and amplitude of the image. In addition, in any resolution, each coordinate of the low-resolution image may refer to a coordinate representing the position of each pixel when it is assumed that the low-resolution image is represented as high-resolution… the processor 140 May encode the low-resolution image to extract a latent feature vector, and may estimate a dominant frequency and a Fourier coefficient corresponding to each coordinate of the low-resolution image based on the latent feature vector… derive a color value corresponding to each coordinate of the low-resolution image based on the dominant frequency and the Fourier coefficient… the pre-trained image restoration algorithm may be a learning model trained to output a color value mapped to each two-dimensional coordinate by querying, as an input, a dominant frequency and a Fourier coefficient derived from a local latent feature vector corresponding to each two-dimensional coordinate, based on each two-dimensional coordinate of the low-resolution image when the low-resolution image is input… the processor 140 according to an embodiment may perform the scale-dependent phase encoding and the low-resolution image skip connection to learn the high-frequency texture, thereby showing excellent performance in representing the continuous domain signal… a network structure of an image restoration algorithm of the image restoration apparatus 100. That is, FIG. 4 is an embodiment of a network structure that enables any scale SR using a local texture estimator; Pg. 6: SR encoder may extract a feature map having the same height and width as the low-resolution image. The local texture estimator can then retrieve the feature map at the encoder and estimate the main frequency and the corresponding Fourier coefficient for the natural image… neural network function may represent the low-resolution image as an arbitrary high resolution image; Pg. 9: in the case of the SR operation, when the scale factor is changed, the position of the edge may be changed within the small neighborhood of the high-resolution (HR) domain… the image restoration algorithm according to an embodiment may be trained with a training image generated by setting an arbitrary scale factor, selecting a patch having a size reflecting a scale factor from the high-resolution image, and then downsampling the selected patch by a scale factor… in the image restoration algorithm, the color value may be predicted based on the center coordinates of the pixels in the image domain; and based on the HR feature and coordinate information and pixel information of an HR image at a target magnification, restoring the HR image at the target magnification corresponding to the feature map (e.g. image restoration method and apparatus include a super resolution (SR) encoder used to extract a feature map having a same height and width as a low-resolution (LR) image when input, for example, including a processor programmed to encode the low-resolution image to extract a latent feature vector (i.e. mapping an input image with low resolution (LR) to a feature map of a latent domain), by estimating a dominant frequency (i.e. a certain frequency component) and essential Fourier information for a natural image, such as an input image, for example, in which an implicit neural network function prioritizes (i.e. using a model) high-frequency detail (i.e. the HR feature) learning, thereby reconstructing a high-resolution image at any resolution (i.e. an HR image) even in a significant scale factor (i.e. a target magnification), for example, and generate the high-resolution image of an arbitrary resolution based on a color value corresponding to each coordinate of the low-resolution image, in which the color value corresponding to each coordinate of the low-resolution image is derived based on the dominant frequency and the Fourier coefficient, as indicated above, including a color predicted based on center coordinates of the pixels in the image domain (i.e. and based on the HR feature and coordinate information and pixel information of an HR image at a target magnification), for example, by using a pre-trained image restoration algorithm including a learning model trained to output a color value mapped to each two-dimensional coordinate by querying, as an input, a dominant frequency and a Fourier coefficient derived from a local latent feature vector corresponding to each two-dimensional coordinate, based on each two-dimensional coordinate of the low-resolution image when the low-resolution image is input, as indicated above), for example), but fails to teach that the neural network function (i.e. image restoration algorithm, learning model, etc.) is a diffusion model.
However, QIN teaches a diffusion model (Pg. 2: method includes: obtaining a first resolution image of an original scene; converting the first resolution image into a second resolution image by using a reversible neural network model, and then transmitting the first resolution image to a first resolution image, wherein a resolution of the second resolution image is lower than a first resolution image; inputting the restored first resolution image into a trained super-resolution diffusion model, and performing super-resolution reconstruction through a random iterative denoising process to output an ultra-high-resolution image; Pg. 3: performing super-resolution reconstruction on the scaled image to obtain the ultra-high-resolution image. For example, the output recovered image is super-resolved to a high-resolution size by using a super-resolution diffusion model).
JIN and QIN are considered to be analogous art because they pertain to image processing applications. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the image restoration method and apparatus in which an implicit neural network (i.e. model, algorithm, etc.) function prioritizes high-frequency detail learning (as disclosed by JIN) with a diffusion model (as taught by QIN, Abstract, Pg. 2-3) to perform super-resolution reconstruction (QIN, Abstract, Pg. 2-3).
Regarding claim 2, claim 1 is incorporated and the combination of JIN and QIN, as a whole, teaches the method (JIN, Pg. 1), wherein the mapping of the input image to the feature map of the latent domain comprises mapping the input image to the feature map of the latent domain using an encoder network (JIN, Pg. 1: Single image super-resolution (SISR)… SISR (hereinafter, referred to as SR) aims to reconstruct a high-resolution image in a degraded low-resolution image; Pg. 2: restoring an image to an arbitrary resolution in a continuous manner… image restoration method… include: obtaining a low-resolution image… obtaining a low-resolution image; estimating Fourier information of the low-resolution image based on a pre-trained image restoration algorithm, and deriving a color value corresponding to each coordinate of the low-resolution image at an arbitrary resolution; Pg. 4: server that acquires a low-resolution image… and controls an operation of the image restoration device 100 for a whole process of reconstructing the low-resolution image into a high-resolution image in a continuous manner; Pg. 5: processor… obtain a low-resolution image… the processor 140 May encode the low-resolution image to extract a latent feature vector, and may estimate a dominant frequency and a Fourier coefficient corresponding to each coordinate of the low-resolution image based on the latent feature vector… image restoration algorithm may be a learning model trained to output a color value mapped to each two-dimensional coordinate by querying, as an input, a dominant frequency and a Fourier coefficient derived from a local latent feature vector corresponding to each two-dimensional coordinate, based on each two-dimensional coordinate of the low-resolution image when the low-resolution image is input… in order to restore a low-resolution image to an arbitrary high-resolution image without expressing an image at a fixed resolution, a continuous representation of the image should be learned. Accordingly, the processor 140 May reconstruct an image at an arbitrary resolution by modeling the image as a function defined in a continuous area; Pg. 6: SR encoder may extract a feature map having the same height and width as the low-resolution image… the processor 140 May train the encoder with the local implicit neural network function representation through the super-resolution self-supervised operation to generate a continuous representation of the image. The local implicit neural network function representation is to represent an image as a latent code set dispersed in a spatial dimension… When a coordinate is given, the decoder may take coordinate information and query a local latent code around the coordinate as an input, and then predict an RGB value in a given coordinate as an output… since the coordinates are continuous, the local implicit neural network function may represent the low-resolution image as an arbitrary high resolution image… decoder may map the latent tensor and the local coordinates to a color value (RGB value); mapping an input image with low resolution (LR) to a feature map of a latent domain (e.g. image restoration method and apparatus include a super resolution (SR) encoder used (i.e. an encoder network) to extract a feature map (i.e. mapping) having a same height and width as a low-resolution (LR) image when input, for example, including a processor programmed to encode the low-resolution image to extract a latent feature vector (i.e. wherein the mapping of the input image to the feature map of the latent domain comprises mapping the input image to the feature map of the latent domain using an encoder network), as indicated above), for example).
Regarding claim 5, claim 1 is incorporated and the combination of JIN and QIN, as a whole, teaches the method (JIN, Pg. 1), wherein the generating of the HR feature comprises generating, using the diffusion model (QIN, Pg. 2: method includes: obtaining a first resolution image of an original scene; converting the first resolution image into a second resolution image by using a reversible neural network model, and then transmitting the first resolution image to a first resolution image, wherein a resolution of the second resolution image is lower than a first resolution image; inputting the restored first resolution image into a trained super-resolution diffusion model, and performing super-resolution reconstruction through a random iterative denoising process to output an ultra-high-resolution image; Pg. 3: performing super-resolution reconstruction on the scaled image to obtain the ultra-high-resolution image. For example, the output recovered image is super-resolved to a high-resolution size by using a super-resolution diffusion model), the HR feature in which the certain frequency component corresponding to the feature map is restored (JIN, Pg. 2: improve the accuracy of image restoration by estimating a dominant frequency for a natural image by being configured as a single network, and learning fine details while restoring an image to an arbitrary resolution in a continuous manner… estimate a dominant frequency and essential Fourier information for a natural image, so that an implicit neural network function for any resolution prioritizes high-frequency detail learning… by estimating the dominant frequency and essential Fourier information for the natural image, the implicit neural network function may prioritize the high-frequency detail learning, thereby reconstructing the high-resolution image; Pg. 6: SR encoder may extract a feature map having the same height and width as the low-resolution image. The local texture estimator can then retrieve the feature map at the encoder and estimate the main frequency and the corresponding Fourier coefficient for the natural image… neural network function may represent the low-resolution image as an arbitrary high resolution image), by concatenating or adding gradually changing input noise with or to the feature map (JIN, Pg. 13: image restoration apparatus 100 May concatenate the 3 × 3 adjacent latent feature vectors to generate an unfolded latent feature vector; Pg. 6: SR encoder may extract a feature map having the same height and width as the low-resolution image. The local texture estimator can then retrieve the feature map at the encoder and estimate the main frequency and the corresponding Fourier coefficient for the natural image… neural network function may represent the low-resolution image as an arbitrary high resolution image).
The same motivation to combine above-mentioned teachings applies, as previously indicated in claim 1.
Regarding claim 16, claim 1 is incorporated and the combination of JIN and QIN, as a whole, teaches the method (JIN, Pg. 1), including a non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 1 (QIN, Pg. 4: computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device).
The same motivation to combine above-mentioned teachings applies, as previously indicated in claim 1.
Regarding claim 17, is a corresponding apparatus claim rejected as applied to method claim 1 above.
Regarding claim 18, claim 17 and the combination of JIN and QIN, as a whole, teaches the apparatus (JIN, Pg. 1), wherein, for the generating of the HR feature, the one or more processors are configured to generate, using the diffusion model (QIN, Pg. 2: method includes: obtaining a first resolution image of an original scene; converting the first resolution image into a second resolution image by using a reversible neural network model, and then transmitting the first resolution image to a first resolution image, wherein a resolution of the second resolution image is lower than a first resolution image; inputting the restored first resolution image into a trained super-resolution diffusion model, and performing super-resolution reconstruction through a random iterative denoising process to output an ultra-high-resolution image; Pg. 3: performing super-resolution reconstruction on the scaled image to obtain the ultra-high-resolution image. For example, the output recovered image is super-resolved to a high-resolution size by using a super-resolution diffusion model), the HR feature in which the certain frequency component corresponding to the feature map is restored (JIN, Pg. 2: improve the accuracy of image restoration by estimating a dominant frequency for a natural image by being configured as a single network, and learning fine details while restoring an image to an arbitrary resolution in a continuous manner… estimate a dominant frequency and essential Fourier information for a natural image, so that an implicit neural network function for any resolution prioritizes high-frequency detail learning… by estimating the dominant frequency and essential Fourier information for the natural image, the implicit neural network function may prioritize the high-frequency detail learning, thereby reconstructing the high-resolution image; Pg. 6: SR encoder may extract a feature map having the same height and width as the low-resolution image. The local texture estimator can then retrieve the feature map at the encoder and estimate the main frequency and the corresponding Fourier coefficient for the natural image… neural network function may represent the low-resolution image as an arbitrary high resolution image), by concatenating gradually changing input noise with the feature map (JIN, Pg. 13: image restoration apparatus 100 May concatenate the 3 × 3 adjacent latent feature vectors to generate an unfolded latent feature vector; Pg. 6: SR encoder may extract a feature map having the same height and width as the low-resolution image. The local texture estimator can then retrieve the feature map at the encoder and estimate the main frequency and the corresponding Fourier coefficient for the natural image… neural network function may represent the low-resolution image as an arbitrary high resolution image).
The same motivation to combine above-mentioned teachings applies, as previously indicated in claim 1.
Regarding claim 20, claim 17 and the combination of JIN and QIN, as a whole, teaches the apparatus (JIN, Pg. 1), wherein the apparatus is any one or any combination of any two or more of a smartphone, a camera, a closed-circuit television (CCTV), medical image equipment, semiconductor measurement equipment, an autonomous vehicle camera, a mixed reality (MR) device, and an augmented reality (AR) device (JIN, Pg. 3: user terminal 200 May be a desktop computer, a smartphone, a notebook, a tablet PC, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop, a media player, a micro server, a global positioning system (GPS) device, an electronic book terminal, a digital broadcasting terminal, a navigation device, a kiosk, an MP3 player, a digital camera, a home appliance, and other mobile or non-mobile computing devices operated by a user).
Allowable Subject Matter
Claims 8-10 and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: The prior art of record fails to anticipate or render obvious the following limitations as claimed:
In view of claim 1 in its entirety, the further limitations of “… wherein the generating of the HR feature comprises generating the HR feature in which the certain frequency component corresponding to the feature map is restored, by applying the feature map to a cross-attention technique” as recited in claim 8.
In view of claim 1 in its entirety, the further limitations of “… receiving image quality-related keywords corresponding to the input image;
extracting a text feature corresponding to the image quality-related keywords; and generating the HR feature in which the certain frequency component corresponding to the feature map is restored, by mapping the text feature to the feature map” as recited in claim 9.
Claim 10 is dependent upon claim 9 above.
In view of claim 17 in its entirety, the further limitations of “… extract, using a text encoder based on a pre-trained vision language model (VLM), extract a text feature corresponding to image quality-related keywords corresponding to the input image; and
for the generating of the HR feature, generate, using the diffusion model, the HR feature in which the certain frequency component corresponding to the feature map is restored, by mapping the text feature to the feature map” as recited in claim 19.
Examiner was not able to find art similar to aforementioned claimed subject matter above.
Conclusion
Due to the inability to determine a reasonable interpretation of claims 3-4, 6-7, and 11-15, based on indefiniteness rejections above, no prior art rejection or determination of allowability over the prior art was possible during examination of instant application.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GUILLERMO M RIVERA-MARTINEZ whose telephone number is (571) 272-4979. The examiner can normally be reached on 9 am to 5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached on 571-270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GUILLERMO M RIVERA-MARTINEZ/ Primary Examiner, Art Unit 2677
Prosecution Insights