DETAILED ACTION
This action is in response to the application filed on March 28, 2024. Claims 1-20 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on May 09, 2024 and October 02, 2025 are being considered by the examiner.
Claim Objections
Claim(s) 3 and 7 are objected to because of the following informalities: A colon “ : ” is missing after “comprises” and “further comprising” respectively. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Yuan et al, US 20090324079 in view of Chen et al, US 20200334819 in view of Batra et al, US 20190355154.
Regarding claim 1, Salvador teaches a computer-implemented method comprising:
selecting, by at least one processor, a set of image patches corresponding to high-frequency portions of a digital image at a first resolution (see Salvador, Paragraph [0023], “for each of a plurality of patches of the low-frequency portion L.sub.1 of the upscaled high resolution data structure, a best matching block in the low-frequency portion L.sub.0 of the low resolution digital input data structure is searched, and its corresponding block in the high-frequency portion H.sub.0 of the low resolution digital input data structure is determined”);
generating, by the at least one processor (see Salvador, Paragraph [0022], “The invention is interesting for interactive applications, offering low computational load and parallelizable design that allows e.g. straight-forward GPU implementations”) utilizing an image super-resolution model (see Salvador, Paragraph [0001], “an apparatus for performing single-image super-resolution”), upscaled image patches for the set of image patches corresponding to the high-frequency portions to a second resolution higher than the first resolution according to an upscaling factor of at least two (see Salvador, Paragraph [0023], “steps of upscaling and low-pass filtering the single low resolution digital input data structure to obtain a low-frequency portion L.sub.1 of an upscaled high resolution data structure,” and Paragraph [0034], “an up-scaling with s=2 is implemented as an initial up-scaling”);
Salvador does not expressively teach
generating, by the at least one processor, a segmentation map for the digital image based on the upscaled image patches
However, Yuan in a similar invention in the same field of endeavor teaches
generating, by the at least one processor, a segmentation map for the digital image based on the upscaled image patches (see Yuan, Paragraph [0042], “.alpha..sup.h 50 is the up-scaled version of the low-resolution segmentation map a 44”)
The combination of Salvador and Yuan are analogous art because they are both in the same field of endeavor of image upscaling. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to generate an up-scaled version of the low-resolution map as taught in the method Yuan in the method of Salvador as region-based up-scaling processes may be desirable to avoid introducing artifacts in a graphics region which reduces the perceived visual quality of the up-scaled image when using conventional image up-scaling methods (Yuan, Paragraph [0034]).
Salvador in view of Yuan does not expressively teach
and an upscaled segmentation corresponding to low-frequency portions of the digital image;
However, Chen in a similar invention in the same field of endeavor teaches
and an upscaled segmentation corresponding to low-frequency portions of the digital image (see Paragraph [0013], Chen, “The image segmentation method further comprises: obtaining a binarized mask image of the input image based on a low-frequency semantic features generated by a define-refine network”);
The combination of Salvador, Yuan, and Chen are analogous art because they are all in the same field of endeavor of image upscaling. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, for the image segmentation method to further comprise obtaining a binarized mask based on a low-frequency semantic features and the low frequency semantic feature may be upscaled as taught in the system of Chen in the system of Salvador in view of Yuan to achieve an ideal segmentation effect (Chen, Paragraph [0004]).
Salvador in view of Yuan in view of Chen does not expressively teach
and generating, by the at least one processor, a vectorized digital image for the digital image according to the segmentation map.
However, Batra in a similar invention in the same field of endeavor teaches
and generating, by the at least one processor, a vectorized digital image for the digital image according to the segmentation map (see Batra, Paragraph [0090], “a raster image may provide an original input, and may then undergo vectorization to obtain a vector image for further operations. In this regard, it will be appreciated that various techniques for vectorization generally include segmenting the input image using edge detection, and then computing colors for each of the contours in the segmented image”).
The combination of Salvador, Yuan, Chen, and Batra are analogous art because they are all in the same field of endeavor of image processing to improve image quality. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to vectorize a raster image by segmenting the input image using edge detection as taught in the technique of Batra in the method of Salvador in view of Yuan in view of Chen to utilize an efficient, fast, accurate, complete, and widely-applicable algorithm(s) to colorize vector images (Batra, Paragraph [0021]).
Regarding claim 7, Salvador in view of Yuan in view of Chen in view of Batra further teaches the computer-implemented method of claim 1, further comprising
generating the upscaled segmentation corresponding to the low-frequency portions of the digital image by generating a segmentation for the low-frequency portions and upscaling the segmentation of the low-frequency portions to the second resolution (see Chen, Paragraph [0056], “When the scale of the low-frequency semantic feature output from the last dense-refine network in the cascade is smaller than a scale of a desired binarized mask image, the low-frequency semantic feature may be upscaled by the upscaling network 230. As shown in FIG. 2, the low-frequency semantic feature having the ½ scale and generated by the third dense-refine network 223 may be for example 2 upscaled, to obtain a binarized mask image having a 1 scale. In some embodiments, the binarized mask image may also be filtered to filter out the interference caused by a glitch region in the input image, to obtain a more accurate image segmentation result”).
The rationale of claim 1 has been applied herein.
Claim(s) 2 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Yuan et al, US 20090324079 in view of Chen et al, US 20200334819 in view of Batra et al, US 20190355154 in view of Park et al, US 20190355125 in view of Huang et al, US 20210065413.
Regarding claim 2, Salvador in view of Yuan in view of Chen in view of Batra further teaches the computer-implemented method of claim 1,
wherein selecting the set of image patches further comprises: generating, utilizing an edge detection model (see Batra, Paragraph [0090], “segmenting the input image using edge detection”),
Salvador in view of Yuan in view of Chen in view of Batra does not expressively teach
generating, utilizing an edge detection model, an edge map that indicates the high-frequency portions and the low-frequency portions of the digital image based on detected edges in the edge map;
However, Park in a similar invention in the same field of endeavor teaches
utilizing an edge detection model (see Park, Paragraph [0084], “the common edge map generation unit 504 may generate the common edge map by way of spectral decomposition of the image”), an edge map that indicates the high-frequency portions and the low-frequency portions of the digital image based on detected edges in the edge map (see Park, Paragraph [0084], “When the low spatial frequencies form k-space are suppressed, little contrast appears in the image, yet edge definition remains. It is exploiting the fact that the fine details of the image such as edges are contained in the high spatial frequency portion that are in the peripheries of k-space,” high frequency portions are located in the peripheries with edge definition and low frequency portions are in the center of the image appearing with little contrast);
The combination of Salvador, Yuan, Chen, Batra, and Park are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, for the edge map generation unit to generate a common edge map in which high frequency portions are located in the peripheries with edge definition and low frequencies are in the center as taught in the method of Park in the method of Salvador in view of Yuan in view of Chen in view of Batra to use less data to speed up the scanning process (Park, Paragraph [0003]).
Salvador in view of Yuan in view of Chen in view of Batra in view of Park does not expressively teach
and selecting the set of image patches from the digital image according to the high-frequency portions indicated by the edge map.
However, Huang in a similar invention in the same field of endeavor teaches
and selecting the set of image patches from the digital image according to the high-frequency portions indicated by the edge map (see Huang, Paragraph [0039], “a two-pass, high frequency region detection process is performed on the input representation received from color test module 13 based on the edge-map representation obtained from edge detector 12,” detecting the high frequency region on the input representation based on the edge-map representation is considered to be selecting patches from the digital image according to high-frequency portions indicated by the edge map).
The combination of Salvador, Yuan, Chen, Batra, Park, and Huang are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to use a high frequency region detection process based on the edge-map representation as taught in the method Huang in the method Salvador in view of Yuan in view of Chen in view of Batra in view of Park to improve reliability and the accuracy of the subsampling model (Huang, Paragraph [0053]).
Claim(s) 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Yuan et al, US 20090324079 in view of Chen et al, US 20200334819 in view of Batra et al, US 20190355154 in view of Munkberg et al, US 20200126191.
Regarding claim 4, Salvador in view of Yuan in view of Chen in view of Batra does not expressively the computer-implemented method of claim 1, further comprising:
generating a training dataset by determining, for a vector image, an image pair comprising a first rasterized image with aliasing and a second rasterized image with anti-aliasing;
and adjusting parameters of the image super-resolution model based on the first rasterized image and the second rasterized image.
However, Munkberg in a similar invention in the same field of endeavor teaches
generating a training dataset by determining, for a vector image, an image pair comprising a first rasterized image with aliasing and a second rasterized image with anti-aliasing (see Munkberg, Paragraph [0023], “The warped external recurrent neural network is trained end-to-end to minimize the errors, between pairs of aliased and antialiased images”);
and adjusting parameters of the image super-resolution model based on the first rasterized image and the second rasterized image (see Munkberg, Paragraph [0030], “a reconstructed image that is antialiased,” and Paragraph [0086], “The parameter adjustment unit 245 receives the reconstructed image frames and target image frames included in the training dataset and adjusts parameters of the temporal adaptive sampling and denoising system 200 based on errors between the reconstructed data and the target data frames,” the reconstructed image frames is antialiased and the target image frames contains aliasing).
The combination of Salvador, Yuan, Chen, Batra, and Munkberg are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, for a parameter adjustment unit to adjust parameters of the antialiased reconstructed frames and the target image frames which contain aliasing and compute and minimize a loss function as taught in the method Munkberg in the method Salvador in view of Yuan in view of Chen in view of Batra to achieve significantly improved image quality and temporal stability compared with conventional adaptive sampling techniques (Munkberg, Abstract).
Regarding claim 5, Salvador in view of Yuan in view of Chen in view of Batra in view of Munkberg further teaches the computer-implemented method of claim 4,
wherein adjusting the parameters of the image super-resolution model comprises adjusting the parameters of the image super-resolution model to reduce an output of a loss function determined by comparing the first rasterized image with aliasing to the second rasterized image with anti-aliasing (Munkberg, Paragraph [0087], “denoising system 200 that minimizes the error or loss function on the training data without overfitting,” and Paragraph [0093], “At step 265, the parameter adjustment unit 245 determines if the training is complete. A loss function may be computed by the parameter adjustment unit 245 to measure distances (i.e., differences or gradients) between the target output data and the reconstructed data. The temporal adaptive sampling and denoising system 200 is deemed to be sufficiently trained when the reconstructed data generated for the input data from the training dataset match the target output data or a threshold accuracy is achieved for the training dataset”).
The rationale of claim 4 has been applied herein.
Claim(s) 8 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Yuan et al, US 20090324079 in view of Chen et al, US 20200334819 in view of Batra et al, US 20190355154 in view of Tong et al, US 20220207658.
Regarding claim 8, Salvador in view of Yuan in view of Chen in view of Batra does not expressively the computer-implemented method of claim 1, further comprising:
wherein selecting the set of image patches corresponding to the high-frequency portions comprises selecting the set of image patches based on the high-frequency portions satisfying a density threshold.
However, Tong in a similar invention in the same field of endeavor teaches
wherein selecting the set of image patches corresponding to the high-frequency portions comprises selecting the set of image patches based on the high-frequency portions satisfying a density threshold (see Tong, Paragraph [0097], “A portion of an image with an edge density that is larger than the threshold can indicate that the portion of the image includes fine details and thin structures. Based on comparing the edge density of a portion of the image to a threshold, the electronic device 101 can identify areas of an image with fine details, thin structures, and areas within text itself,” portions with edge density larger than the threshold is considered to be high-frequency portions).
The combination of Salvador, Yuan, Chen, Batra, and Tong are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, for a portion of an image with an edge density greater than the threshold to identify areas with fine details and thin structures is considered to be areas with high-frequency portions as taught in the method Tong in the method of Salvador in view of Yuan in view of Chen in view of Batra to generate a sharpened image of the scene (Tong, Paragraph [0005]).
Claim(s) 9 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Yuan et al, US 20090324079 in view of Chen et al, US 20200334819 in view of Batra et al, US 20190355154 in view of Ho et al, US 20210319536.
Regarding claim 9, Salvador in view of Yuan in view of Chen in view of Batra does not expressively the computer-implemented method of claim 1, further comprising:
wherein generating the segmentation map comprises: generating, utilizing a segmentation model, a segmentation of the high-frequency portions from the upscaled image patches;
and generating the segmentation map by combining the segmentation of the high-frequency portions with the upscaled segmentation corresponding to the low-frequency portions.
However, Ho in a similar invention in the same field of endeavor teaches
wherein generating the segmentation map comprises: generating, utilizing a segmentation model, a segmentation of the high-frequency portions from the upscaled image patches (see Ho, Paragraph [0101], “an upscaling of the content map may be performed,” and Paragraph [0076], “The sub-band splitter 430 splits the noise reduced version 422 of the original input image 402 into the high frequency component image data HF(0) and the low frequency component image data LF(0),” the image is split into high frequency component data and low frequency component data, a content map which is considered to be a segmentation map see Paragraph [0084] may be upscaled if the content map has a lower resolution than the input image);
and generating the segmentation map by combining the segmentation of the high-frequency portions with the upscaled segmentation corresponding to the low-frequency portions (see Ho, Paragraph [0079], “The sub-band merger 352 merges processed high frequency component image data HF(N)′ and processed low frequency component image data LF(N)′ to generate a processed LF(N-1),” and Paragraph [0081], “FIG. 5 is a block diagram illustrating providing of a content map 504 (also referred to as a “segmentation map” herein) by neural processor circuit 218 to image signal processor 206”).
The combination of Salvador, Yuan, Chen, Batra, and Ho are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to merge high frequency component image data and processed low frequency component image data providing a content map as taught in the method Ho in the method of Salvador in view of Yuan in view of Chen in view of Batra to sharpen segments of an image based on content within the segments as indicated by a content map (Ho, Paragraph [0029]).
Claim(s) 10 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Chen et al, US 20200334819 in view of Yuan et al, US 20090324079 in view of Batra et al, US 20190355154.
Regarding claim 10, Salvador teaches a system comprising:
one or more memory devices (see Salvador, Paragraph [0063], “Various memories MemL.sub.0, MemL.sub.1, MemH.sub.0, MemH.sub.1 with appropriate sizes can be used for intermediate storage, which may however be implemented as one single or more physical memories”) comprising a digital image at first resolution and an image super-resolution model (see Salvador, Paragraph [0001], “an apparatus for performing single-image super-resolution”);
and one or more processors (see Salvador, Paragraph [0022], “The invention is interesting for interactive applications, offering low computational load and parallelizable design that allows e.g. straight-forward GPU implementations”) configured to cause the system to: select a first set of image patches corresponding to high-frequency portions of the digital image (see Salvador, Paragraph [0023], “for each of a plurality of patches of the low-frequency portion L.sub.1 of the upscaled high resolution data structure, a best matching block in the low-frequency portion L.sub.0 of the low resolution digital input data structure is searched, and its corresponding block in the high-frequency portion H.sub.0 of the low resolution digital input data structure is determined”);
generate, utilizing the image super-resolution model, upscaled image patches for the first set of image patches corresponding to the high-frequency portions to a second resolution higher than the first resolution according to an upscaling factor of at least two (see Salvador, Paragraph [0023], “steps of upscaling and low-pass filtering the single low resolution digital input data structure to obtain a low-frequency portion L.sub.1 of an upscaled high resolution data structure,” and Paragraph [0034], “an up-scaling with s=2 is implemented as an initial up-scaling”);
Salvador does not expressively teach
generate an upscaled segmentation for a second set of image patches corresponding to low-frequency portions of the digital image by upscaling a segmentation of the second set of image patches according to the upscaling factor;
However, Chen in a similar invention in the same field of endeavor teaches
generate an upscaled segmentation for a second set of image patches corresponding to low-frequency portions of the digital image by upscaling a segmentation of the second set of image patches according to the upscaling factor (see Chen, Paragraph [0013], “The image segmentation method further comprises: obtaining a binarized mask image of the input image based on a low-frequency semantic features generated by a define-refine network”);
The combination of Salvador and Chen are analogous art because they are both in the same field of endeavor of image upscaling. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, for the image segmentation method to further comprise obtaining a binarized mask based on a low-frequency semantic features and the low frequency semantic feature may be upscaled as taught in the apparatus of Chen in the apparatus of Salvador to achieve an ideal segmentation effect (Chen, Paragraph [0004]).
Salvador in view of Chen does not expressively teach
determine a segmentation map for the digital image based on the upscaled image patches and the upscaled segmentation;
However, Yuan in a similar invention in the same field of endeavor teaches
determine a segmentation map for the digital image based on the upscaled image patches and the upscaled segmentation (see Yuan, Paragraph [0039], “.alpha..sup.h 50 is the up-scaled version of the low-resolution segmentation map a 44”);
The combination of Salvador, Chen, and Yuan are analogous art because they are all in the same field of endeavor of image upscaling. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to generate an up-scaled version of the low-resolution map as taught in the system of Yuan in the apparatus of Salvador in view of Chen as region-based up-scaling processes may be desirable to avoid introducing artifacts in a graphics region which reduces the perceived visual quality of the up-scaled image when using conventional image up-scaling methods (Yuan, Paragraph [0034]).
Salvador in view of Chen in view of Yuan does not expressively teach
and generate a vectorized digital image for the digital image according to the segmentation map.
However, Batra in a similar invention in the same field of endeavor teaches
and generate a vectorized digital image for the digital image according to the segmentation map (see Batra, Paragraph [0090], “a raster image may provide an original input, and may then undergo vectorization to obtain a vector image for further operations. In this regard, it will be appreciated that various techniques for vectorization generally include segmenting the input image using edge detection, and then computing colors for each of the contours in the segmented image”).
The combination of Salvador, Chen, Yuan, and Batra are analogous art because they are all in the same field of endeavor of image processing to improve image quality. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to vectorize a raster image by segmenting the input image using edge detection as taught in the system of Batra in the apparatus of Salvador in view of Chen in view of Yuan to utilize an efficient, fast, accurate, complete, and widely-applicable algorithm(s) to colorize vector images (Batra, Paragraph [0021]).
Claim(s) 11 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Chen et al, US 20200334819 in view of Yuan et al, US 20090324079 in view of Batra et al, US 20190355154 in view of Park et al, US 20190355125 in view of Tong et al, US 20220207658.
Regarding claim 11, Salvador in view of Chen in view of Yuan in view of Batra further teaches the system of claim 10,
wherein the one or more processors are configured to cause the system to: generate, utilizing(see Batra, Paragraph [0090], “segmenting the input image using edge detection”),
Salvador in view of Chen in view of Yuan in view of Batra does not expressively teach
utilizing an edge detection model, an edge map that indicates the high-frequency portions and the low-frequency portions of the digital image based on detected edges in the edge map;
However, Park in a similar invention in the same field of endeavor teaches
utilizing an edge detection model (see Park, Paragraph [0084], “the common edge map generation unit 504 may generate the common edge map by way of spectral decomposition of the image”), an edge map that indicates the high-frequency portions and the low-frequency portions of the digital image based on detected edges in the edge map (see Park, Paragraph [0084], “When the low spatial frequencies form k-space are suppressed, little contrast appears in the image, yet edge definition remains. It is exploiting the fact that the fine details of the image such as edges are contained in the high spatial frequency portion that are in the peripheries of k-space,” high frequency portions are located in the peripheries with edge definition and low frequency portions are in the center of the image appearing with little contrast);
The combination of Salvador, Chen, Yuan, Batra, and Park are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, for the edge map generation unit to generate a common edge map in which high frequency portions are located in the peripheries with edge definition and low frequencies are in the center as taught in the system of Park in the apparatus of Salvador in view of Chen in view of Yuan in view of Batra to use less data to speed up the scanning process (Park, Paragraph [0003]).
Salvador in view of Chen in view of Yuan in view of Batra in view of Park does not expressively teach
and select the first set of image patches corresponding the high-frequency portions based on the detected edges in the edge map satisfying a density threshold.
However, Tong in a similar invention in the same field of endeavor teaches
and select the first set of image patches corresponding the high-frequency portions based on the detected edges in the edge map satisfying a density threshold (see Tong, Paragraph [0097], “A portion of an image with an edge density that is larger than the threshold can indicate that the portion of the image includes fine details and thin structures. Based on comparing the edge density of a portion of the image to a threshold, the electronic device 101 can identify areas of an image with fine details, thin structures, and areas within text itself”).
The combination of Salvador, Chen, Yuan, Batra, Park, and Tong are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, for a portion of an image with an edge density greater than the threshold to identify areas with fine details and thin structures is considered to be areas with high-frequency portions as taught in the method of Tong in the apparatus of Salvador in view of Chen in view of Yuan in view of Batra in view of Park to generate a sharpened image of the scene (Tong, Paragraph [0005]).
Claim(s) 12 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Chen et al, US 20200334819 in view of Yuan et al, US 20090324079 in view of Batra et al, US 20190355154 in view of Kim et al, US 20190130543.
Regarding claim 12, Salvador in view of Chen in view of Yuan in view of Batra does not expressively teach the system of claim 10,
wherein the one or more processors are configured to cause the system to select the first set of image patches by utilizing a patch selection model that minimizes, for a predetermined image patch size, a number of image patches corresponding to the high-frequency portions of the digital image.
However, Kim in a similar invention in the same field of endeavor teaches
wherein the one or more processors are configured to cause the system to select the first set of image patches by utilizing a patch selection model that minimizes, for a predetermined image patch size, a number of image patches corresponding to the high-frequency portions of the digital image (see Kim, Paragraph [0056], “The processor 120 may enable fine correction by making the size of the pixel group small by having a small number of pixels in the pixel group in the case of a high-frequency area having a large number of edges in the input image”).
The combination of Salvador, Chen, Yuan, Batra, and Kim are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to enable fine correction by making the size of the pixel group small by having a small number of pixels in the group in the case the high-frequency area having a large number of edges as taught in the apparatus of Kim in the apparatus of Salvador in view of Chen in view of Yuan in view of Batra to reduce the amount of calculation (Kim, Paragraph [0056]).
Claim(s) 13 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Chen et al, US 20200334819 in view of Yuan et al, US 20090324079 in view of Batra et al, US 20190355154 in view of Munkberg et al, US 20200126191.
Regarding claim 13, Salvador in view of Chen in view of Yuan in view of Batra does not expressively teach the system of claim 10,
wherein the one or more processors are configured to cause the system to: generate a training dataset by determining, for a vector image, an image pair comprising a first rasterized image with aliasing and a second rasterized image with anti-aliasing;
and adjust parameters of the image super-resolution model to reduce an output of a loss function determined by comparing the first rasterized image with aliasing to the second rasterized image with anti-aliasing to determine a loss.
However, Munkberg in a similar invention in the same field of endeavor teaches
wherein the one or more processors are configured to cause the system to: generate a training dataset by determining, for a vector image, an image pair comprising a first rasterized image with aliasing and a second rasterized image with anti-aliasing (see Munkberg, Paragraph [0030], “a reconstructed image that is antialiased,” and Paragraph [0086], “The parameter adjustment unit 245 receives the reconstructed image frames and target image frames included in the training dataset and adjusts parameters of the temporal adaptive sampling and denoising system 200 based on errors between the reconstructed data and the target data frames,” the reconstructed image frames is antialiased and the target image frames contains aliasing);
and adjust parameters of the image super-resolution model to reduce an output of a loss function determined by comparing the first rasterized image with aliasing to the second rasterized image with anti-aliasing to determine a loss (Munkberg, Paragraph [0093], “At step 265, the parameter adjustment unit 245 determines if the training is complete. A loss function may be computed by the parameter adjustment unit 245 to measure distances (i.e., differences or gradients) between the target output data and the reconstructed data. The temporal adaptive sampling and denoising system 200 is deemed to be sufficiently trained when the reconstructed data generated for the input data from the training dataset match the target output data or a threshold accuracy is achieved for the training dataset”).
The combination of Salvador, Chen, Yuan, Batra, and Munkberg are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, for a parameter adjustment unit to adjust parameters of the antialiased reconstructed frames and the target image frames which contain aliasing and compute and minimize a loss function as taught in the system of Munkberg in the apparatus of Salvador in view of Chen in view of Yuan in view of Batra to achieve significantly improved image quality and temporal stability compared with conventional adaptive sampling techniques (Munkberg, Abstract).
Claim(s) 15 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Chen et al, US 20200334819 in view of Yuan et al, US 20090324079 in view of Batra et al, US 20190355154 in view of Ho et al, US 20210319536.
Regarding claim 15, Salvador in view of Chen in view of Yuan in view of Batra does not expressively teach the system of claim 10,
wherein the one or more processors are configured to cause the system to determine the segmentation map for the digital image by: generate, utilizing a segmentation model, a segmentation of the high-frequency portions from the upscaled image patches;
and generate the segmentation map by combining the segmentation of the high-frequency portions with the upscaled segmentation for the second set of image patches.
However, Ho in a similar invention in the same field of endeavor teaches
wherein the one or more processors are configured to cause the system to determine the segmentation map for the digital image by: generate, utilizing a segmentation model, a segmentation of the high-frequency portions from the upscaled image patches (see Ho, Paragraph [0101], “an upscaling of the content map may be performed,” and Paragraph [0076], “The sub-band splitter 430 splits the noise reduced version 422 of the original input image 402 into the high frequency component image data HF(0) and the low frequency component image data LF(0),” the image is split into high frequency component data and low frequency component data, a content map which is considered to be a segmentation map see Paragraph [0084] may be upscaled if the content map has a lower resolution than the input image);
and generate the segmentation map by combining the segmentation of the high-frequency portions with the upscaled segmentation for the second set of image patches (see Ho, Paragraph [0079], “The sub-band merger 352 merges processed high frequency component image data HF(N)′ and processed low frequency component image data LF(N)′ to generate a processed LF(N-1),” and Paragraph [0081], “FIG. 5 is a block diagram illustrating providing of a content map 504 (also referred to as a “segmentation map” herein) by neural processor circuit 218 to image signal processor 206”).
The combination of Salvador, Chen, Yuan, Batra, and Ho are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to merge high frequency component image data and processed low frequency component image data providing a content map as taught in the method of Ho in the apparatus of Salvador in view of Chen in view of Yuan in view of Batra to sharpen segments of an image based on content within the segments as indicated by a content map (Ho, Paragraph [0029]).
Claim(s) 16 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Park et al, US 20190355125 in view of Chen et al, US 20200334819 in view of Yuan et al, US 20090324079 in view of Batra et al, US 20190355154.
Regarding claim 16, Salvador teaches a non-transitory computer readable medium storing executable instructions which, when executed by at least one processing device, cause the at least one processing device to perform operations comprising (see Salvador, Paragraph [0063], “Various memories MemL.sub.0, MemL.sub.1, MemH.sub.0, MemH.sub.1 with appropriate sizes can be used for intermediate storage, which may however be implemented as one single or more physical memories”):
generating, utilizing an image super-resolution model, upscaled image patches for a first set of image patches corresponding to the high-frequency portions of the digital image to a second resolution higher than the first resolution according to an upscaling factor of at least two (see Salvador, Paragraph [0023], “steps of upscaling and low-pass filtering the single low resolution digital input data structure to obtain a low-frequency portion L.sub.1 of an upscaled high resolution data structure,” and Paragraph [0034], “an up-scaling with s=2 is implemented as an initial up-scaling”);
Salvador does not expressively teach
generating, utilizing an edge detection model on a digital image at a first resolution, an edge map comprising indications of high-frequency portions and low-frequency portions of the digital image;
However, Park in a similar invention in the same field of endeavor teaches
generating, utilizing an edge detection model on a digital image at a first resolution (see Park, Paragraph [0084], “the common edge map generation unit 504 may generate the common edge map by way of spectral decomposition of the image”), an edge map comprising indications of high-frequency portions and low-frequency portions of the digital image (see Park, Paragraph [0090], “When the low spatial frequencies form k-space are suppressed, little contrast appears in the image, yet edge definition remains. It is exploiting the fact that the fine details of the image such as edges are contained in the high spatial frequency portion that are in the peripheries of k-space”);
The combination of Salvador and Park are analogous art because they are both in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, for the edge map generation unit to generate a common edge map in which high frequency portions are located in the peripheries with edge definition and low frequencies are in the center as taught in the method of Park in the method of Salvador to use less data to speed up the scanning process (Park, Paragraph [0003]).
Salvador in view of Park does not expressively teach
generating an upscaled segmentation for a second set of image patches corresponding to the low-frequency portions of the digital image by upscaling a segmentation of the second set of image patches according to the upscaling factor;
However, Chen in a similar invention in the same field of endeavor teaches
generating an upscaled segmentation for a second set of image patches corresponding to the low-frequency portions of the digital image by upscaling a segmentation of the second set of image patches according to the upscaling factor (see Chen, Paragraph [0013], “The image segmentation method further comprises: obtaining a binarized mask image of the input image based on a low-frequency semantic features generated by a define-refine network”);
The combination of Salvador, Park, and Chen are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, for the image segmentation method to further comprise obtaining a binarized mask based on a low-frequency semantic features and the low frequency semantic feature may be upscaled as taught in the method of Chen in the method of Salvador in view of Park to achieve an ideal segmentation effect (Chen, Paragraph [0004]).
Salvador in view of Park in view of Chen does not expressively teach
determining a segmentation map for the digital image based on a combination of the upscaled image patches and the upscaled segmentation;
However, Yuan in a similar invention in the same field of endeavor teaches
determining a segmentation map for the digital image based on a combination of the upscaled image patches and the upscaled segmentation (see Yuan, Paragraph [0039], “.alpha..sup.h 50 is the up-scaled version of the low-resolution segmentation map a 44,” and Paragraph [0080], “The up-scaled substantially uniform regions 199 may be combined 202 with the up-scaled edge areas 201 to form the up-scaled graphics region 203”);
The combination of Salvador, Park, Chen, and Yuan are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to generate an up-scaled version of the low-resolution map as taught in the method of Yuan in the method of Salvador in view of Park in view of Chen as region-based up-scaling processes may be desirable to avoid introducing artifacts in a graphics region which reduces the perceived visual quality of the up-scaled image when using conventional image up-scaling methods (Yuan, Paragraph [0034]).
Salvador in view of Park in view of Chen in view of Yuan does not expressively teach
and generating a vectorized digital image for the digital image according to the segmentation map.
However, Batra in a similar invention in the same field of endeavor teaches
and generating a vectorized digital image for the digital image according to the segmentation map (see Batra, Paragraph [0090], “a raster image may provide an original input, and may then undergo vectorization to obtain a vector image for further operations. In this regard, it will be appreciated that various techniques for vectorization generally include segmenting the input image using edge detection, and then computing colors for each of the contours in the segmented image”).
The combination of Salvador, Park, Chen, Yuan, and Batra are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to vectorize a raster image by segmenting the input image using edge detection as taught in the technique of Batra in the method of Salvador in view of Park in view of Chen in view of Yuan to utilize an efficient, fast, accurate, complete, and widely-applicable algorithm(s) to colorize vector images (Batra, Paragraph [0021]).
Claim(s) 17 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Park et al, US 20190355125 in view of Chen et al, US 20200334819 in view of Yuan et al, US 20090324079 in view of Batra et al, US 20190355154 in view of Kim et al, US 20190130543.
Regarding claim 17, Salvador in view of Park in view of Chen in view of Yuan in view of Batra does not expressively teach the non-transitory computer-readable medium of claim 16,
wherein selecting the first set of image patches comprises minimizing a number of image patches including high-frequency data corresponding to the high-frequency portions and the low-frequency portions of the digital image based on detected edges in the edge map.
However, Kim in a similar invention in the same field of endeavor teaches
wherein selecting the first set of image patches comprises minimizing a number of image patches including high-frequency data corresponding to the high-frequency portions and the low-frequency portions of the digital image based on detected edges in the edge map (see Kim, Paragraph [0056], “The processor 120 may enable fine correction by making the size of the pixel group small by having a small number of pixels in the pixel group in the case of a high-frequency area having a large number of edges in the input image”).
The combination of Salvador, Park, Chen, Yuan, Batra, and Kim are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to enable fine correction by making the size of the pixel group small by having a small number of pixels in the group in the case the high-frequency area having a large number of edges as taught in the method of Kim in the method of Salvador in view of Park in view of Chen in view of Yuan in view of Batra to reduce the amount of calculation (Kim, Paragraph [0056]).
Claim(s) 18 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Park et al, US 20190355125 in view of Chen et al, US 20200334819 in view of Yuan et al, US 20090324079 in view of Batra et al, US 20190355154 in view of Tong et al, US 20220207658 in view of Ho et al, US 20210319536.
Regarding claim 18, Salvador in view of Park in view of Chen in view of Yuan in view of Batra does not expressively teach the non-transitory computer-readable medium of claim 16, wherein generating the segmentation map comprises:
identifying the second set of image patches based on the second set of image patches failing to satisfy a density threshold indicated by the edge map to further generate the upscaled segmentation for the second set of image patches;
However, Tong in a similar invention in the same field of endeavor teaches
identifying the second set of image patches based on the second set of image patches failing to satisfy a density threshold indicated by the edge map to further generate the upscaled segmentation for the second set of image patches (see Tong, Paragraph [0097], “An edge density that is lower than the threshold indicates a relatively low number of edges within the region,”);
The combination of Salvador, Park, Chen, Yuan, Batra, and Tong are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, for an edge density that is lower than the threshold to indicate relatively low number of edges within the region which is considered to be failing to satisfy a density threshold as taught in the method Tong in the method of Salvador in view of Park in view of Chen in view of Yuan in view of Batra to generate a sharpened image of the scene (Tong, Paragraph [0005]).
Salvador in view of Park in view of Chen in view of Yuan in view of Batra in view of Tong does not expressively teach
generating, utilizing a segmentation model, a segmentation of the high-frequency portions from the upscaled image patches;
and combining the segmentation of the high-frequency portions with the upscaled segmentation for the second set of image patches.
However, Ho in a similar invention in the same field of endeavor teaches
generating, utilizing a segmentation model, a segmentation of the high-frequency portions from the upscaled image patches (see Ho, Paragraph [0101], “an upscaling of the content map may be performed,” and Paragraph [0076], “The sub-band splitter 430 splits the noise reduced version 422 of the original input image 402 into the high frequency component image data HF(0) and the low frequency component image data LF(0),” the image is split into high frequency component data and low frequency component data, a content map which is considered to be a segmentation map see Paragraph [0084] may be upscaled if the content map has a lower resolution than the input image);
and combining the segmentation of the high-frequency portions with the upscaled segmentation for the second set of image patches (see Ho, Paragraph [0079], “The sub-band merger 352 merges processed high frequency component image data HF(N)′ and processed low frequency component image data LF(N)′ to generate a processed LF(N-1),” and Paragraph [0081], “FIG. 5 is a block diagram illustrating providing of a content map 504 (also referred to as a “segmentation map” herein) by neural processor circuit 218 to image signal processor 206”).
The combination of Salvador, Park, Chen, Yuan, Batra, Tong, and Ho are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to merge high frequency component image data and processed low frequency component image data providing a content map as taught in the method of Ho in the method of Salvador in view of Park in view of Chen in view of Yuan in view of Batra in view of Tong to sharpen segments of an image based on content within the segments as indicated by a content map (Ho, Paragraph [0029]).
Claim(s) 19 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Park et al, US 20190355125 in view of Chen et al, US 20200334819 in view of Yuan et al, US 20090324079 in view of Batra et al, US 20190355154 in view of Munkberg et al, US 20200126191.
Regarding claim 19, Salvador in view of Park in view of Chen in view of Yuan in view of Batra does not expressively teach the non-transitory computer-readable medium of claim 16,
wherein the operations further comprise adjusting parameters of the image super-resolution model utilizing a training dataset by: determining, for a vector image, an image pair comprising a first rasterized image with aliasing and a second rasterized image with anti-aliasing;
and adjusting the parameters of the image super-resolution model to reduce an output of a loss function determined by comparing the first rasterized image with aliasing with the second rasterized image with anti-aliasing to determine a loss.
However, Munkberg in a similar invention in the same field of endeavor teaches
wherein the operations further comprise adjusting parameters of the image super-resolution model utilizing a training dataset by: determining, for a vector image, an image pair comprising a first rasterized image with aliasing and a second rasterized image with anti-aliasing (see Munkberg, Paragraph [0030], “a reconstructed image that is antialiased,” and Paragraph [0086], “The parameter adjustment unit 245 receives the reconstructed image frames and target image frames included in the training dataset and adjusts parameters of the temporal adaptive sampling and denoising system 200 based on errors between the reconstructed data and the target data frames,” the reconstructed image frames is antialiased and the target image frames contains aliasing);
and adjusting the parameters of the image super-resolution model to reduce an output of a loss function determined by comparing the first rasterized image with aliasing with the second rasterized image with anti-aliasing to determine a loss (Munkberg, Paragraph [0093], “At step 265, the parameter adjustment unit 245 determines if the training is complete. A loss function may be computed by the parameter adjustment unit 245 to measure distances (i.e., differences or gradients) between the target output data and the reconstructed data. The temporal adaptive sampling and denoising system 200 is deemed to be sufficiently trained when the reconstructed data generated for the input data from the training dataset match the target output data or a threshold accuracy is achieved for the training dataset”).
The combination of Salvador, Park, Chen, Yuan, Batra, and Munkberg are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, for a parameter adjustment unit to adjust parameters of the antialiased reconstructed frames and the target image frames which contain aliasing and compute and minimize a loss function as taught in the method of Munkberg in the method of Salvador in view of Park in view of Chen in view of Yuan in view of Batra to achieve significantly improved image quality and temporal stability compared with conventional adaptive sampling techniques (Munkberg, Abstract).
Claim(s) 20 are rejected under 35 U.S.C. 103 as being unpatentable over Salvador et al, US 20150324953 in view of Park et al, US 20190355125 in view of Chen et al, US 20200334819 in view of Yuan et al, US 20090324079 in view of Batra et al, US 20190355154 in view of Liu et al, US 20210073944 in view of Munkberg et al, US 20200126191.
Regarding claim 20, Salvador in view of Park in view of Chen in view of Yuan in view of Batra does not expressively teach the non-transitory computer-readable medium of claim 16, wherein the operations further comprise:
generating, for a vector image, an image pair comprising a first rasterized image with aliasing and a modified version of the first rasterized image with aliasing or a second rasterized image with anti-aliasing by: downsampling the first rasterized image or the second rasterized image from the first resolution of the digital image to a third resolution lower than the first resolution and upsampling the first rasterized image or the second rasterized image from the third resolution to the first resolution;
However, Liu in a similar invention in the same field of endeavor teaches
generating, for a vector image, an image pair comprising a first rasterized image with aliasing and a modified version of the first rasterized image with aliasing or a second rasterized image with anti-aliasing by: downsampling the first rasterized image or the second rasterized image from the first resolution of the digital image to a third resolution lower than the first resolution and upsampling the first rasterized image or the second rasterized image from the third resolution to the first resolution (see Liu, Paragraph [0050], “pairs of images are used for training, including an image to be upsampled and a corresponding anti-aliased, upsampled, higher resolution image, an image upsampled is considered to be the first rasterized image and anti-aliased upsampled image is considered to be upsampling the second rasterized image”);
The combination of Salvador, Park, Chen, Yuan, Batra, and Liu are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, , to use pairs of images including an image to be upsampled and an anti-aliased upsampled higher resolution image as taught in the method of Liu in the method of Salvador in view of Park in view of Chen in view of Yuan in view of Batra for improved temporal smoothing (Liu, Paragraph [0050]).
Salvador in view of Park in view of Chen in view of Yuan in view of Batra in view of Liu does not expressively teach
applying one or more blur filters to the first rasterized image or the second rasterized image; modifying one or more color properties of the first rasterized image or the second rasterized image; or applying one or more compression models to the first rasterized image or the second rasterized image;
and adjusting parameters of the image super-resolution model to reduce an output of a loss function determined by comparing the first rasterized image with the modified version of the first rasterized image or the second rasterized image.
However, Munkberg in a similar invention in the same field of endeavor teaches
applying one or more blur filters to the first rasterized image or the second rasterized image; modifying one or more color properties of the first rasterized image or the second rasterized image; or applying one or more compression models to the first rasterized image or the second rasterized image (see Munkberg, Paragraph [0125], “The ROP unit 450 performs graphics raster operations related to pixel color, such as color compression, pixel blending, and the like”);
and adjusting parameters of the image super-resolution model to reduce an output of a loss function determined by comparing the first rasterized image with the modified version of the first rasterized image or the second rasterized image (see Munkberg, Paragraph [0030], “a reconstructed image that is antialiased,” and Paragraph [0086], “The parameter adjustment unit 245 receives the reconstructed image frames and target image frames included in the training dataset and adjusts parameters of the temporal adaptive sampling and denoising system 200 based on errors between the reconstructed data and the target data frames,” the process of adjusting parameters based on comparing two images does not change therefore the same method is applied here to the modified images).
The combination of Salvador, Park, Chen, Yuan, Batra, Liu, and Munkberg are analogous art because they are all in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, for a parameter adjustment unit to adjust parameters of the antialiased reconstructed frames and the target image frames which contain aliasing and compute and minimize a loss function as taught in the method of Munkberg in the method of Salvador in view of Park in view of Chen in view of Yuan in view of Batra in view of Liu to achieve significantly improved image quality and temporal stability compared with conventional adaptive sampling techniques (Munkberg, Abstract).
Allowable Subject Matter
Claim(s) 3, 6, and 14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DOMINIQUE JAMES whose telephone number is (703)756-1655. The examiner can normally be reached 9:00 am - 6:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571)270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DOMINIQUE JAMES/Examiner, Art Unit 2666
/MING Y HON/Primary Examiner, Art Unit 2666
Prosecution Insights