DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments (see remarks), filed 04/08/2026, with respect to claims 1-21, have been fully considered but are respectfully unpersuasive.
On page 9-10, the Applicant argues “Applicant respectfully submits that the cited portions of WANG do not teach a band size map that "specifies, for each of a plurality of pixels of the input image, a predicted size of a corresponding banding artifact that is at least partially depicted by the pixel," as now recited…Applicant respectfully submits that the cited XU disclosure likewise does not teach a band size map that "specifies, for each of a plurality of pixels of the input image, a predicted size of a corresponding banding artifact.".
In response, the Office respectfully finds this argument unpersuasive. Based on the breadth of the claim language, the prior art by WANG et al. (US 20210321020 A1) explicitly teaches processing the input image to generate a band size map that identifies at least one banding artifact in the input image (Fig. 6. Paragraph [0042]-WANG discloses [0042] FIG. 6 illustrates an example method for generating banding maps. The banding map may be created for an image or video frame from the detected banding pixels. Specifically, starting from a blank map of the same size of an image or video frame, the positions of the detected banding pixels 600 in the image or video frame are marked on the blank map at operation 602. This process results in a binary map of the same size as the image or video frame being assessed, where the positions of detected banding pixels are marked as a first value (e.g., 1, true, etc.), and the rest of the map is marked as a second value (e.g., 0, false, etc). The signal activity is then normalized based on the significance threshold at operation 606, resulting in a normalized signal activity. The intensity value at the banding pixels in the map (marked as the first value) may then be replaced by the normalized signal activity measure at the pixels at operation 608. This leads to the banding map 610, where the region of the image with no banding detected remains at the second value, but at the banding pixels the intensity values are equal to the local normalized signal activity measurement), wherein the band size map specifies, for each of a plurality of pixels of the input image, a predicted size of a corresponding banding artifact that is at least partially depicted by the pixel (Fig. 6. Paragraph [0026]-WANG discloses contour tracing methods applied after the banding pixels are detected may be used to refine the connections and shapes of banding contours for better appearance, and may be an optional further processing for the purposes of banding detection and banding level assessment. The collection of detected banding pixels 208 may subsequently be processed by further operations to generate various measures (such as per-color channel or per-frame banding spread, banding strength, banding map, and areas/regions of banding) about the banding effect, and may also be aggregated to banding effect measurements at each spatial location (in an image or video frame), each time instance (frame) in a video, and at larger time scales (per-scene, per-second, per-minute, per-hour, and so on).) In paragraph [0042]-WANG discloses this banding map reveals how much the banding effect is visible at each spatial location in an image or video frame (wherein the banding maps are computed for all color channels at operation 704, a per-channel banding spread is computed by computing the percentage of areas in the image or video frame that contain banding pixels, a per-channel banding spread 710 and per-channel banding strength 712, are then combined to an overall banding level measure of the channel at operation 714 and the per-channel banding level measures are combined into a single per-frame banding level measure at operation 71, which is a scalar banding assessment). Please also read paragraph [0019, 0023-0030, 0040-0041 and 0043-0046]);
Although WANG explicitly teaches processing the input image to generate a band size map that identifies at least one banding artifact in the input image, wherein the band size map specifies, for each of a plurality of pixels of the input image, a predicted size of a corresponding banding artifact that is at least partially depicted by the pixel.
WANG fails to explicitly teach processing, using a neural network, the input image to generate a band size map that identifies at least one banding artifact in the input image.
However, XU et al. (US 20160360202 A1) explicitly teaches processing, using a neural network (Fig. 2, #34 called a feed forward network. Paragraph [0038]. Further in paragraph [0033]-XU discloses the encoder 16 implements a compression algorithm using encoder parameters. The artifact predictor 14 is adapted to generate an artifact prediction map, indicating the likelihood of whether the artifact will occur given use of a particular encoder parameter or range of encoder parameters. In paragraph [0035]-Xu discloses FIG. 2 is a block diagram illustrating a second example system 30 that is adapted to predict banding artifacts in reconstructed video. In paragraph [0038]-XU discloses the feed-forward neural network 34 is adapted to output banding metadata as a map (e.g., table or grid) predicting or indicating whether banding is predicted to occur as a function of QPs for each image block of each frame of video data. In paragraph [0062]-XU discloses regarding the architecture of the feed-forward neural network 34, given the features extracted from each image block, a the feed-forward neural network 34 is trained, e.g., via the training module 40, to learn a classifier that maps a feature vector to a binary value, e.g., 0 or 1, where 0 represents “no banding” and 1 represents “banding.” (wherein stochastic training and optimization and multiple filters, including a smoothing filter may be used)), the input image to generate a band size map that identifies at least one banding artifact in the input image (Fig. 1. Paragraph [0036]-XU discloses the second example system 30 includes a feature extractor module 32 for extracting feature data from input video data. A feed-forward neural network 34 is adapted to process the feature data to classify each video block (also called video patch, image block, or video frame patch) as “no banding” or “banding” for one or more QPs that may be used by the video encoder. In paragraph [0037]-XU discloses the example feature extractor module 32 is adapted to capture color and texture information for each block (e.g., a block representing 8×8 pixels of a video frame). In paragraph [0038]-XU discloses the feed-forward neural network 34 is adapted to output banding metadata as a map (e.g., table or grid) predicting or indicating whether banding is predicted to occur as a function of QPs for each image block of each frame of video data).
On page 10, the Applicant argues “Applicant respectfully submits that the cited portions of SU do not teach that "the blur filter removes at least a portion of the at least one banding artifact from the input image based on the predicted size specified in the band size map," as now recited.”.
In response, the Office respectfully finds this argument unpersuasive. Based on the breadth of the claim language, the prior art by SU et al. (US 20160021391 A1) explicitly teaches generating, using a blur filter (Fig. 1. Paragraph [0067]-SU discloses a blur filter can be used at least as a part of the smoothen operator to remove banding artifacts in a high bit depth image. A convolution of a 2-D (separable or non-separable) low pass kernel, etc., may be used to generate smooth area over flat areas or stair wise areas), a de-banded image based on the band size map and the input image (Fig. 1. Paragraph [0031]-SU discloses techniques may be used to enhance prediction of wide or high dynamic range image data based on relatively low bit depth (e.g., 8 bits, etc.) image data in a base layer and reduce false contouring artifacts in reconstructed wide or high dynamic range (e.g., 12+ bits VDR, etc.) images. In paragraph [0032]-SU discloses decontouring techniques include smoothen operations performed by a smoothen operator, selection/masking operations between smoothened pixel values and unsmoothened pixel values, etc. False contouring artifacts can be effectively removed by performing decontouring filtering in the predicted image data using the smoothen operations, the selection/masking operations, etc. In paragraph [0055]-SU discloses an input image 101 (e.g., a VDR image, a predicted high bit depth image predicted from a low bit depth image, a decoded image that comprises banding artifacts, etc.) to which the decontouring operations are to be performed is denoted as v), wherein the blur filter removes at least a portion of the at least one banding artifact from the input image based on the predicted size specified in the band size map (Fig. 3. Paragraph [0084]-SU discloses a smoothness threshold Δ can be used as a constraint in selecting operational parameters for the smoothen operator. A residual mask can be specifically set to limit maximal distortions introduced by smoothen operations in any given image area of an image (e.g., a predicted high bit depth image, etc.) to no greater than the smoothness threshold Δ, which may be set as one of a maximum stair step size as in expression (9), an average stair step size as in expression (10), or an adaptive maximal stair step size as in expression (11), etc.). In paragraph [0089]-SU discloses in block 302, image areas with a high confidence level to have false contouring are determined in an image. Please also read paragraph [0056-0066]).
On page 10, the Applicant argues “Accordingly, the cited combination of WANG, XU, and SU does not teach amended independent claim 1. Amended independent claims 19 and 20 are patentable for at least the same reasons.”.
In response, the Office respectfully finds this argument unpersuasive for the reasons stated above and below.
On page 11, the Applicant argues “Applicant respectfully submits that WANG, XU, and SU do not teach the amended limitations of independent claims 1, 19, and 20, and the rejection of claims 2-3, 7-10, 15, 18, and 21 should therefore also be withdrawn.”.
In response, the Office respectfully finds this argument unpersuasive for the reasons stated above and below.
On page 11, the Applicant argues “Accordingly, claims 4-5 and 13-14 are patentable for at least the same reasons as amended independent claim 1, and withdrawal of the rejection is respectfully requested.”.
In response, the Office respectfully finds this argument unpersuasive for the reasons stated above and below.
On page 12, the Applicant argues “Accordingly, claim 6 is patentable for at least the same reasons as amended independent claim 1, and withdrawal of the rejection is respectfully requested.”.
In response, the Office respectfully finds this argument unpersuasive for the reasons stated above and below.
On page 12, the Applicant argues “Accordingly, claims 11-12 are patentable for at least the same reasons as amended independent claim 1, and withdrawal of the rejection is respectfully requested.”.
In response, the Office respectfully finds this argument unpersuasive for the reasons stated above and below.
On page 13, the Applicant argues “Accordingly, claims 16-17 are patentable for at least the same reasons as amended independent claim 1, and withdrawal of the rejection is respectfully requested.”.
In response, the Office respectfully finds this argument unpersuasive for the reasons stated above and below.
On page 13, the Applicant argues “Accordingly, Applicant respectfully submits that the cited references do not teach the pending claims, either alone or in the combinations set forth in the Office Action, and withdrawal of the rejections under 35 U.S.C. § 103 is respectfully requested.”.
In response, the Office respectfully finds this argument unpersuasive for the reasons stated above and below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 7-10, 15 and 18-21 are rejected under 35 U.S.C. 103 as being unpatentable over WANG et al. (US 20210321020 A1), hereinafter referenced as WANG in view of XU et al. (US 20160360202 A1), hereinafter referenced as XU and in further view of SU et al. (US 20160021391 A1), hereinafter referenced as SU.
Regarding claim 1, WANG explicitly teaches a method (Fig. 5. Paragraph [0016]-WANG discloses aspects of the disclosure generally relate to automated detection and assessment of visual banding effect in digital images and video signals. Further in paragraph [0017]-WANG discloses another aspect of the present disclosure relates to creating a banding map. Yet another aspect of the present disclosure relates to creating a banding spread measure and a banding strength measure. Yet another aspect of the present disclosure relates to banding measurement of a video, where the per-frame banding spread, per-frame banding strength and per-frame banding level (or banding presence) measures are aggregated over a time scale. Please also see Fig. 2-4 and 6-7), comprising:
receiving an input image having a first dynamic range (Fig. 5. Paragraph [0018]-WANG discloses the present disclosure applies to both standard dynamic range (SDR) and high dynamic range (HDR) images and videos. In paragraph [0022]-WANG discloses the banding effect may be detected and assessed in HDR images/videos (wherein High dynamic range images/videos have a first dynamic range));
Although WANG explicitly teaches processing the input image to generate a band size map that identifies at least one banding artifact in the input image (Fig. 6. Paragraph [0042]-WANG discloses [0042] FIG. 6 illustrates an example method for generating banding maps. The banding map may be created for an image or video frame from the detected banding pixels. Specifically, starting from a blank map of the same size of an image or video frame, the positions of the detected banding pixels 600 in the image or video frame are marked on the blank map at operation 602. This process results in a binary map of the same size as the image or video frame being assessed, where the positions of detected banding pixels are marked as a first value (e.g., 1, true, etc.), and the rest of the map is marked as a second value (e.g., 0, false, etc). The signal activity is then normalized based on the significance threshold at operation 606, resulting in a normalized signal activity. The intensity value at the banding pixels in the map (marked as the first value) may then be replaced by the normalized signal activity measure at the pixels at operation 608. This leads to the banding map 610, where the region of the image with no banding detected remains at the second value, but at the banding pixels the intensity values are equal to the local normalized signal activity measurement), wherein the band size map specifies, for each of a plurality of pixels of the input image, a predicted size of a corresponding banding artifact that is at least partially depicted by the pixel (Fig. 6. Paragraph [0026]-WANG discloses contour tracing methods applied after the banding pixels are detected may be used to refine the connections and shapes of banding contours for better appearance, and may be an optional further processing for the purposes of banding detection and banding level assessment. The collection of detected banding pixels 208 may subsequently be processed by further operations to generate various measures (such as per-color channel or per-frame banding spread, banding strength, banding map, and areas/regions of banding) about the banding effect, and may also be aggregated to banding effect measurements at each spatial location (in an image or video frame), each time instance (frame) in a video, and at larger time scales (per-scene, per-second, per-minute, per-hour, and so on).) In paragraph [0042]-WANG discloses this banding map reveals how much the banding effect is visible at each spatial location in an image or video frame (wherein the banding maps are computed for all color channels at operation 704, a per-channel banding spread is computed by computing the percentage of areas in the image or video frame that contain banding pixels, a per-channel banding spread 710 and per-channel banding strength 712, are then combined to an overall banding level measure of the channel at operation 714 and the per-channel banding level measures are combined into a single per-frame banding level measure at operation 71, which is a scalar banding assessment). Please also read paragraph [0019, 0023-0030, 0040-0041 and 0043-0046]);
WANG fails to explicitly teach processing, using a neural network, the input image to generate a band size map that identifies at least one banding artifact in the input image.
However, XU explicitly teaches processing, using a neural network (Fig. 2, #34 called a feed forward network. Paragraph [0038]. Further in paragraph [0033]-XU discloses the encoder 16 implements a compression algorithm using encoder parameters. The artifact predictor 14 is adapted to generate an artifact prediction map, indicating the likelihood of whether the artifact will occur given use of a particular encoder parameter or range of encoder parameters. In paragraph [0035]-Xu discloses FIG. 2 is a block diagram illustrating a second example system 30 that is adapted to predict banding artifacts in reconstructed video. In paragraph [0038]-XU discloses the feed-forward neural network 34 is adapted to output banding metadata as a map (e.g., table or grid) predicting or indicating whether banding is predicted to occur as a function of QPs for each image block of each frame of video data. In paragraph [0062]-XU discloses regarding the architecture of the feed-forward neural network 34, given the features extracted from each image block, a the feed-forward neural network 34 is trained, e.g., via the training module 40, to learn a classifier that maps a feature vector to a binary value, e.g., 0 or 1, where 0 represents “no banding” and 1 represents “banding.” (wherein stochastic training and optimization and multiple filters, including a smoothing filter may be used)), the input image to generate a band size map that identifies at least one banding artifact in the input image (Fig. 1. Paragraph [0036]-XU discloses the second example system 30 includes a feature extractor module 32 for extracting feature data from input video data. A feed-forward neural network 34 is adapted to process the feature data to classify each video block (also called video patch, image block, or video frame patch) as “no banding” or “banding” for one or more QPs that may be used by the video encoder. In paragraph [0037]-XU discloses the example feature extractor module 32 is adapted to capture color and texture information for each block (e.g., a block representing 8×8 pixels of a video frame). In paragraph [0038]-XU discloses the feed-forward neural network 34 is adapted to output banding metadata as a map (e.g., table or grid) predicting or indicating whether banding is predicted to occur as a function of QPs for each image block of each frame of video data).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG of having a method, comprising: receiving an input image having a first dynamic range; processing the input image to generate a band size map to generate a band size map that identifies at least one banding artifact in the input image, wherein the band size map specifies, for each of a plurality of pixels of the input image, a predicted size of a corresponding banding artifact that is at least partially depicted by the pixel, with the teachings of XU of having processing, using a banding detector neural network, the input image to generate a band size map that identifies at least one banding artifact in the input image.
Wherein having WANG’s method having processing, using a neural network, the input image to generate a band size map that identifies at least one banding artifact in the input image, wherein the band size map specifies, for each of a plurality of pixels of the input image, a predicted size of a corresponding banding artifact that is at least partially depicted by the pixel.
The motivation behind the modification would have been to obtain a method that improves the accuracy and efficiency of banding artifact detection and removal, since both WANG and XU systems and methods for image and banding artifact analysis. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while XU provides systems and methods that improve the accuracy of banding prediction based on color and gradient features in YCbCr color spaces and use of a feed-forward neural network classifier 34. Please see WANG et al. (US 20210321020 A1), Abstract and Paragraph [0010] and XU et al. (US 20160360202 A1), Abstract and Paragraph [0069].
WANG in view of XU fails to explicitly teach generating, using a blur filter, a de-banded image based on the band size map and the input image, wherein the blur filter removes at least a portion of the at least one banding artifact from the input image based on the predicted size specified in the band size map; and generating, based on the de-banded image, an output image having a second dynamic range that is greater than the first dynamic range.
However, SU explicitly teaches generating, using a blur filter (Fig. 1. Paragraph [0067]-SU discloses a blur filter can be used at least as a part of the smoothen operator to remove banding artifacts in a high bit depth image. A convolution of a 2-D (separable or non-separable) low pass kernel, etc., may be used to generate smooth area over flat areas or stair wise areas), a de-banded image based on the band size map and the input image (Fig. 1. Paragraph [0031]-SU discloses techniques may be used to enhance prediction of wide or high dynamic range image data based on relatively low bit depth (e.g., 8 bits, etc.) image data in a base layer and reduce false contouring artifacts in reconstructed wide or high dynamic range (e.g., 12+ bits VDR, etc.) images. In paragraph [0032]-SU discloses decontouring techniques include smoothen operations performed by a smoothen operator, selection/masking operations between smoothened pixel values and unsmoothened pixel values, etc. False contouring artifacts can be effectively removed by performing decontouring filtering in the predicted image data using the smoothen operations, the selection/masking operations, etc. In paragraph [0055]-SU discloses an input image 101 (e.g., a VDR image, a predicted high bit depth image predicted from a low bit depth image, a decoded image that comprises banding artifacts, etc.) to which the decontouring operations are to be performed is denoted as v), wherein the blur filter removes at least a portion of the at least one banding artifact from the input image based on the predicted size specified in the band size map (Fig. 3. Paragraph [0084]-SU discloses a smoothness threshold Δ can be used as a constraint in selecting operational parameters for the smoothen operator. A residual mask can be specifically set to limit maximal distortions introduced by smoothen operations in any given image area of an image (e.g., a predicted high bit depth image, etc.) to no greater than the smoothness threshold Δ, which may be set as one of a maximum stair step size as in expression (9), an average stair step size as in expression (10), or an adaptive maximal stair step size as in expression (11), etc.). In paragraph [0089]-SU discloses in block 302, image areas with a high confidence level to have false contouring are determined in an image. Please also read paragraph [0056-0066]); and
generating, based on the de-banded image, an output image having a second dynamic range that is greater than the first dynamic range (Fig. 1. Paragraph [0119]-SU discloses the smoothen operator (410) is applied to the prediction image data (e.g., intermediate high dynamic range image, etc.) to generate smoothened prediction image data (e.g., a smoothened high bit depth image, etc.). Further in paragraph [0148]-SU discloses in block 502, a multi-layer video encoder (e.g., 402 of FIG. 4A) determines flat image areas in a predicted image, the predicted image being predicted from a first image mapped from a second image that has a higher dynamic range than the first image. Please also read paragraph [0122-0123, 0129, 0133 and 0139-0140]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU of having a method, comprising: receiving an input image having a first dynamic range with the teachings of SU of having generating, using a blur filter, a de-banded image based on the band size map and the input image, wherein the blur filter removes at least a portion of the at least one banding artifact from the input image based on the predicted size specified in the band size map; and generating, based on the de-banded image, an output image having a second dynamic range that is greater than the first dynamic range.
Wherein having WANG’s method having generating, using a blur filter, a de-banded image based on the band size map and the input image, wherein the blur filter removes at least a portion of the at least one banding artifact from the input image based on the predicted size specified in the band size map; and generating, based on the de-banded image, an output image having a second dynamic range that is greater than the first dynamic range.
The motivation behind the modification would have been to obtain a method that improves the accuracy and efficiency of banding artifact detection and removal, since both WANG and SU systems and methods for image analysis in the context of banding artifacts. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while SU provides systems and methods that provide optimal operational parameters for removing or reducing banding artifacts with relatively small distortion and improve bit depth expansion in the false contouring areas (or stair wise areas) significantly both perceptually and objectively. Please see WANG (US 20210321020 A1), Abstract and Paragraph [0010] and SU (US 20160021391 A1), Abstract and Paragraph [0031, 0084 and 0087].
Regarding claim 2, WANG in view of XU and in further view of SU explicitly teach the method of claim 1, WANG further teaches wherein the band size map identifies at least one of a predicted band size or a predicted pixel location of the at least one banding artifact in the image (Fig. 6. Paragraph [0022]-WANG discloses the detection and assessment of the banding effect may be determined from an image or video clip at the pixel level. Further in paragraph [0042]-WANG discloses FIG. 6 illustrates an example method for generating banding maps. The banding map may be created for an image or video frame from the detected banding pixels. This process results in a binary map of the same size as the image or video frame being assessed, where the positions of detected banding pixels are marked as a first value (e.g., 1, true, etc.), and the rest of the map is marked as a second value (e.g., 0, false, etc.). This banding map reveals how much the banding effect is visible at each spatial location in an image or video frame. Further in paragraph [0043]-WANG discloses the detected banding pixels at each image or video frame may be aggregated to several banding assessment measurement, including per-channel and per-frame banding spread, banding strength, and banding level measures. Please also read paragraph [0019 and 0040-0041]).
Regarding claim 3, WANG in view of XU and in further view of SU explicitly teach the method of claim 2, WANG further teaches wherein the band size map includes a plurality of channels (Fig. 6. Paragraph [0024]-WANG discloses at operation 202, the image or each video frame are decomposed into luminance or color channels. In paragraph [0025]-WANG discloses at operation 204, the signal activity is measured at each spatial location. In paragraph [0026]-WANG discloses at operation 206, banding pixels 208 are determined. Additionally in paragraph [0042]-WANG discloses the banding map may be created for an image or video frame from the detected banding pixels. This process results in a binary map of the same size as the image or video frame being assessed, where the positions of detected banding pixels are marked as a first value (e.g., 1, true, etc.), and the rest of the map is marked as a second value (e.g., 0, false, etc.) (wherein the signal activity of banding maps is normalized based on the significance threshold at operation 606, and both the collection of detected banding pixels and banding map may be aggregated with other measures such as per color/per frame banding). Please also see paragraph [0044-0046]), wherein at least one channel of the plurality of channels corresponds to a respective threshold band size in a plurality of threshold band sizes (Fig. 7. Paragraph [0045]-WANG discloses based on the banding map of a color channel, a per-channel banding spread is computed at operation 706 by computing the percentage of areas in the image or video frame that contain banding pixels. Based on the banding map, a per-channel banding strength is computed 708 by averaging the signal activities at all banding pixels. Further in paragraph [0046]-WANG discloses the resulting per-channel banding spread 710 and per-channel banding strength 712, are then combined to an overall banding level measure of the channel at operation 714. The combination may be performed by computing their sum, average, product, weighted-average, median, maximum, minimum, or other methods. Moreover, in paragraph [0047]-WANG discloses the per-channel banding level measures are combined into a single per-frame banding level measure at operation 718. The combination may be conducted by computing the sum, average, product, weighted-average, median, maximum, or minimum of per-channel banding level measures).
Regarding claim 7, WANG in view of XU and in further view of SU explicitly teach the method of claim 1, WANG in view of XU fail to explicitly teach wherein the blur filter removes the at least a portion of the at least one banding artifact from an area of the input image, wherein the area of the input image is identified based at least in part on at least one of the predicted pixel location or a particular threshold band size that corresponds to a channel of the band size map.
However, SU explicitly teaches wherein the blur filter removes the at least a portion of the at least one banding artifact from an area of the input image, wherein the area of the input image is identified based at least in part on at least one of the predicted pixel location or a particular threshold band size that corresponds to a channel of the band size map (Fig. 1. Paragraph [0067]-SU discloses a blur filter can be used at least as a part of the smoothen operator to remove banding artifacts in a high bit depth image (e.g., that is predicted from a low bit depth image that lost original real values in quantization, etc.). A convolution of a 2-D (separable or non-separable) low pass kernel, etc., may be used to generate smooth area over flat areas or stair wise areas).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU and in further view of SU of having a method, comprising: receiving an input image having a first dynamic range, with the teachings of SU of having wherein the blur filter removes the at least a portion of the at least one banding artifact from an area of the input image, wherein the area of the input image is identified based at least in part on at least one of the predicted pixel location or a particular threshold band size that corresponds to a channel of the band size map.
Wherein having WANG’s method having wherein the blur filter removes the at least a portion of the at least one banding artifact from an area of the input image, wherein the area of the input image is identified based at least in part on at least one of the predicted pixel location or a particular threshold band size that corresponds to a channel of the band size map.
The motivation behind the modification would have been to obtain a method that improves the accuracy and efficiency of banding artifact detection and removal, since both WANG and SU systems and methods for image analysis in the context of banding artifacts. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while SU provides systems and methods that provide optimal operational parameters for removing or reducing banding artifacts with relatively small distortion and improve bit depth expansion in the false contouring areas (or stair wise areas) significantly both perceptually and objectively. Please see WANG (US 20210321020 A1), Abstract and Paragraph [0010] and SU (US 20160021391 A1), Abstract and Paragraph [0031, 0084 and 0087].
Regarding claim 8, WANG in view of XU and in further view of SU explicitly teach the method of claim 7, WANG in view of XU fail to explicitly teach wherein the blur filter uses at least one of the predicted pixel location or the particular threshold band size to remove the respective band.
However, SU explicitly teaches wherein the blur filter uses at least one of the predicted pixel location or the particular threshold band size to remove the respective band (Fig. 1. Paragraph [0067]-SU discloses a blur filter can be used at least as a part of the smoothen operator to remove banding artifacts in a high bit depth image (e.g., that is predicted from a low bit depth image that lost original real values in quantization, etc.). A convolution of a 2-D (separable or non-separable) low pass kernel, etc., may be used to generate smooth area over flat areas or stair wise areas).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU and in further view of SU of having a method, comprising: receiving an input image having a first dynamic range, with the teachings of SU of having wherein the blur filter uses at least one of the predicted pixel location or the particular threshold band size to remove the respective band.
Wherein having WANG’s method having wherein the blur filter uses at least one of the predicted pixel location or the particular threshold band size to remove the respective band.
The motivation behind the modification would have been to obtain a method that improves the accuracy and efficiency of banding artifact detection and removal, since both WANG and SU systems and methods for image analysis in the context of banding artifacts. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while SU provides systems and methods that provide optimal operational parameters for removing or reducing banding artifacts with relatively small distortion and improve bit depth expansion in the false contouring areas (or stair wise areas) significantly both perceptually and objectively. Please see WANG (US 20210321020 A1), Abstract and Paragraph [0010] and SU (US 20160021391 A1), Abstract and Paragraph [0031, 0084 and 0087].
Regarding claim 9, WANG in view of XU and in further view of SU explicitly teach the method of claim 1, WANG further teaches wherein the band size map (Fig. 6. Paragraph [0042]-WANG discloses the banding map may be created for an image or video frame from the detected banding pixels. This process results in a binary map of the same size as the image or video frame being assessed, where the positions of detected banding pixels are marked as a first value (e.g., 1, true, etc.), and the rest of the map is marked as a second value (e.g., 0, false, etc.) (wherein the banding map may be used and/or aggregated with other measures such as the collection of detected banding pixel operations, per color/per frame banding). Please also see paragraph [0044-0046]) includes a plurality of scalar band size values, wherein each scalar band size value in the plurality of scalar band size values specifies a respective predicted size of a respective corresponding band (Fig. 7. Paragraph [0045]-WANG discloses based on the banding map of a color channel, a per-channel banding spread is computed at operation 706. Based on the banding map, a per-channel banding strength is computed 708. Further in paragraph [0046]-WANG discloses the resulting per-channel banding spread 710 and per-channel banding strength 712, are then combined to an overall banding level measure of the channel at operation 714. Moreover, in paragraph [0047]-WANG discloses the per-channel banding level measures are combined into a single per-frame banding level measure at operation 718. The combination may be conducted by computing the sum, average, product, weighted-average, median, maximum, or minimum of per-channel banding level measures. The resulting banding level measure 720 is a scalar banding assessment result for the image or video frame).
Regarding claim 10, WANG in view of XU and in further view of SU explicitly teach the method of claim 9, WANG further teaches wherein each respective corresponding band includes a pixel to which the scalar band size value corresponds in the band size map (Fig. 6. Paragraph [0024]-WANG discloses at operation 202, the image or each video frame are decomposed into luminance or color channels. In paragraph [0026]-WANG discloses at operation 206, banding pixels 208 are determined. In paragraph [0042]-WANG discloses the banding map may be created for an image or video frame from the detected banding pixels. This process results in a binary map of the same size as the image or video frame being assessed, where the positions of detected banding pixels are marked as a first value (e.g., 1, true, etc.), and the rest of the map is marked as a second value (e.g., 0, false, etc.). In paragraph [0045]-WANG discloses based on the banding map of a color channel, a per-channel banding spread is computed at operation 706. Based on the banding map, a per-channel banding strength is computed 708. Further in paragraph [0046]-WANG discloses the resulting per-channel banding spread 710 and per-channel banding strength 712, are then combined to an overall banding level measure of the channel at operation 714. Moreover, in paragraph [0047]-WANG discloses the per-channel banding level measures are combined into a single per-frame banding level measure at operation 718. The resulting banding level measure 720 is a scalar banding assessment result for the image or video frame).
Regarding claim 15, WANG in view of XU and in further view of SU explicitly teach the method of claim 1, WANG in view of XU fail to explicitly teach wherein the first dynamic range comprises Standard Dynamic Range (SDR) and the second dynamic range comprises High Dynamic Range (HDR).
However, SU explicitly teaches wherein the first dynamic range comprises Standard Dynamic Range (SDR) and the second dynamic range comprises High Dynamic Range (HDR) (Fig. 1. Paragraph [0033]-SU discloses a residual mask can be generated based on differences between a predicted image predicted from a lower dynamic range image (e.g., an SDR image, an intermediate dynamic range image, etc.) that is mapped from an original higher dynamic range (e.g., VDR, etc.). Further in paragraph [0119]-SU discloses the smoothen operator (410) is applied to the prediction image data (e.g., a predicted high bit depth image predicted from a SDR image, an intermediate high dynamic range image, etc.). Further in paragraph [0148]-SU discloses in block 502, a multi-layer video encoder (e.g., 402 of FIG. 4A) determines flat image areas in a predicted image, the predicted image being predicted from a first image mapped from a second image that has a higher dynamic range than the first image).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU and in further view of SU of having a method, comprising: receiving an input image having a first dynamic range, with the teachings of SU of having wherein the first dynamic range comprises Standard Dynamic Range (SDR) and the second dynamic range comprises High Dynamic Range (HDR).
Wherein having WANG’s method having wherein the first dynamic range comprises Standard Dynamic Range (SDR) and the second dynamic range comprises High Dynamic Range (HDR).
The motivation behind the modification would have been to obtain a method that improves the accuracy and efficiency of banding artifact detection and removal, since both WANG and SU systems and methods for image analysis in the context of banding artifacts. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while SU provides systems and methods that provide optimal operational parameters for removing or reducing banding artifacts with relatively small distortion and improve bit depth expansion in the false contouring areas (or stair wise areas) significantly both perceptually and objectively. Please see WANG (US 20210321020 A1), Abstract and Paragraph [0010] and SU (US 20160021391 A1), Abstract and Paragraph [0031, 0084 and 0087].
Regarding claim 18, WANG in view of XU and in further view of SU explicitly teach the method of claim 1, WANG further teaches wherein the input image is included in a frame of content being streamed or displayed (Fig. 1. Paragraph [0019]-WANG discloses a video stream is a sequence of multiple such images. The images of a video stream are often referred to as frames. The techniques for computing banding pixels and banding maps described herein operate on images, either as standalone images or as frames of video. Further in paragraph [0033]-WANG discloses the maximum and minimum luminance levels 406 of the display device may be used to determine the luminance range of the display device and subsequently adjust the significance threshold at operation 408. This information may be received from the display device in an example, or may be retrieved from storage including information regarding the capabilities of various display devices).
Regarding claim 19, WANG explicitly teaches a processor (Fig. 1. Paragraph [0052]-WANG discloses FIG. 9 illustrates an example computing device 900 for performing automated detection and assessment of visual banding effect in digital images and video signals. The computing device 900 may include memory 902, processor 904, and non-volatile storage 906) comprising:
one or more processing units to perform operations (Fig. 1. Paragraph [0052]-WANG discloses the processor 904 may include one or more devices selected from high-performance computing (HPC) systems including high-performance cores, microprocessors, micro-controllers, digital signal processors, microcomputers, central processing units, field programmable gate arrays, programmable logic devices, state machines, logic circuits, analog circuits, digital circuits, or any other devices that manipulate signals (analog or digital) based on computer-executable instructions residing in memory 902. Further in paragraph [0054]-WANG discloses upon execution by the processor 904, the computer-executable instructions of the program instructions 908 may cause the computing device 900 to implement one or more of the algorithms and/or methodologies disclosed herein) comprising:
receiving an input image having a first dynamic range (Fig. 5. Paragraph [0018]-WANG discloses the present disclosure applies to both standard dynamic range (SDR) and high dynamic range (HDR) images and videos. Further in paragraph [0022]-WANG discloses the banding effect may be detected and assessed in HDR images/videos (wherein a High dynamic image/video has a first dynamic range));
Although WANG explicitly teaches processing the input image to generate a band size map that identifies at least one banding artifact in the input image (Fig. 6. Paragraph [0042]-WANG discloses [0042] FIG. 6 illustrates an example method for generating banding maps. The banding map may be created for an image or video frame from the detected banding pixels. Specifically, starting from a blank map of the same size of an image or video frame, the positions of the detected banding pixels 600 in the image or video frame are marked on the blank map at operation 602. This process results in a binary map of the same size as the image or video frame being assessed, where the positions of detected banding pixels are marked as a first value (e.g., 1, true, etc.), and the rest of the map is marked as a second value (e.g., 0, false, etc). The signal activity is then normalized based on the significance threshold at operation 606, resulting in a normalized signal activity. The intensity value at the banding pixels in the map (marked as the first value) may then be replaced by the normalized signal activity measure at the pixels at operation 608. This leads to the banding map 610, where the region of the image with no banding detected remains at the second value, but at the banding pixels the intensity values are equal to the local normalized signal activity measurement), wherein the band size map specifies, for each of a plurality of pixels of the input image, a predicted size of a corresponding banding artifact that is at least partially depicted by the pixel (Fig. 6. Paragraph [0026]-WANG discloses contour tracing methods applied after the banding pixels are detected may be used to refine the connections and shapes of banding contours for better appearance, and may be an optional further processing for the purposes of banding detection and banding level assessment. The collection of detected banding pixels 208 may subsequently be processed by further operations to generate various measures (such as per-color channel or per-frame banding spread, banding strength, banding map, and areas/regions of banding) about the banding effect, and may also be aggregated to banding effect measurements at each spatial location (in an image or video frame), each time instance (frame) in a video, and at larger time scales (per-scene, per-second, per-minute, per-hour, and so on).) In paragraph [0042]-WANG discloses this banding map reveals how much the banding effect is visible at each spatial location in an image or video frame (wherein the banding maps are computed for all color channels at operation 704, a per-channel banding spread is computed by computing the percentage of areas in the image or video frame that contain banding pixels, a per-channel banding spread 710 and per-channel banding strength 712, are then combined to an overall banding level measure of the channel at operation 714 and the per-channel banding level measures are combined into a single per-frame banding level measure at operation 71, which is a scalar banding assessment). Please also read paragraph [0019, 0023-0030, 0040-0041 and 0043-0046]);
WANG fails to explicitly teach processing, using a neural network, the input image to generate a band size map that identifies at least one banding artifact in the input image.
However, XU explicitly teaches processing, using a neural network (Fig. 2, #34 called a feed forward network. Paragraph [0038]. Further in paragraph [0033]-XU discloses the encoder 16 implements a compression algorithm using encoder parameters. The artifact predictor 14 is adapted to generate an artifact prediction map, indicating the likelihood of whether the artifact will occur given use of a particular encoder parameter or range of encoder parameters. In paragraph [0035]-Xu discloses FIG. 2 is a block diagram illustrating a second example system 30 that is adapted to predict banding artifacts in reconstructed video. In paragraph [0038]-XU discloses the feed-forward neural network 34 is adapted to output banding metadata as a map (e.g., table or grid) predicting or indicating whether banding is predicted to occur as a function of QPs for each image block of each frame of video data. In paragraph [0062]-XU discloses regarding the architecture of the feed-forward neural network 34, given the features extracted from each image block, a the feed-forward neural network 34 is trained, e.g., via the training module 40, to learn a classifier that maps a feature vector to a binary value, e.g., 0 or 1, where 0 represents “no banding” and 1 represents “banding.” (wherein stochastic training and optimization and multiple filters, including a smoothing filter may be used)), the input image to generate a band size map that identifies at least one banding artifact in the input image (Fig. 1. Paragraph [0036]-XU discloses the second example system 30 includes a feature extractor module 32 for extracting feature data from input video data. A feed-forward neural network 34 is adapted to process the feature data to classify each video block (also called video patch, image block, or video frame patch) as “no banding” or “banding” for one or more QPs that may be used by the video encoder. In paragraph [0037]-XU discloses the example feature extractor module 32 is adapted to capture color and texture information for each block (e.g., a block representing 8×8 pixels of a video frame). In paragraph [0038]-XU discloses the feed-forward neural network 34 is adapted to output banding metadata as a map (e.g., table or grid) predicting or indicating whether banding is predicted to occur as a function of QPs for each image block of each frame of video data).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG of having a processor comprising: one or more processing units to perform operations comprising: receiving an input image having a first dynamic range; processing the input image to generate a band size map to generate a band size map that identifies at least one banding artifact in the input image, wherein the band size map specifies, for each of a plurality of pixels of the input image, a predicted size of a corresponding banding artifact that is at least partially depicted by the pixel, with the teachings of XU of having processing, using a banding detector neural network, the input image to generate a band size map that identifies at least one banding artifact in the input image.
Wherein having WANG’s processor having processing, using a banding detector neural network, the input image to generate a band size map to generate a band size map that identifies at least one banding artifact in the input image, wherein the band size map specifies, for each of a plurality of pixels of the input image, a predicted size of a corresponding banding artifact that is at least partially depicted by the pixel.
The motivation behind the modification would have been to obtain a processor that improves the accuracy and efficiency of banding artifact detection and removal, since both WANG and XU systems and methods for image and banding artifact analysis. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while XU provides systems and methods that improve the accuracy of banding prediction based on color and gradient features in YCbCr color spaces and use of a feed-forward neural network classifier 34. Please see WANG et al. (US 20210321020 A1), Abstract and Paragraph [0010] and XU et al. (US 20160360202 A1), Abstract and Paragraph [0069].
WANG in view of XU fails to explicitly teach generating, using a blur filter, a de-banded image based on the band size map and the input image, wherein the blur filter removes at least a portion of the at least one banding artifact from the input image based on the predicted size specified in the band size map; and generating, based on the de-banded image, an output image having a second dynamic range that is greater than the first dynamic range.
However, SU explicitly teaches generating, using a blur filter (Fig. 1. Paragraph [0067]-SU discloses a blur filter can be used at least as a part of the smoothen operator to remove banding artifacts in a high bit depth image. A convolution of a 2-D (separable or non-separable) low pass kernel, etc., may be used to generate smooth area over flat areas or stair wise areas), a de-banded image based on the band size map and the input image (Fig. 1. Paragraph [0031]-SU discloses techniques may be used to enhance prediction of wide or high dynamic range image data based on relatively low bit depth (e.g., 8 bits, etc.) image data in a base layer and reduce false contouring artifacts in reconstructed wide or high dynamic range (e.g., 12+ bits VDR, etc.) images. In paragraph [0032]-SU discloses decontouring techniques include smoothen operations performed by a smoothen operator, selection/masking operations between smoothened pixel values and unsmoothened pixel values, etc. False contouring artifacts can be effectively removed by performing decontouring filtering in the predicted image data using the smoothen operations, the selection/masking operations, etc. In paragraph [0055]-SU discloses an input image 101 (e.g., a VDR image, a predicted high bit depth image predicted from a low bit depth image, a decoded image that comprises banding artifacts, etc.) to which the decontouring operations are to be performed is denoted as v), wherein the blur filter removes at least a portion of the at least one banding artifact from the input image based on the predicted size specified in the band size map (Fig. 3. Paragraph [0084]-SU discloses a smoothness threshold Δ can be used as a constraint in selecting operational parameters for the smoothen operator. A residual mask can be specifically set to limit maximal distortions introduced by smoothen operations in any given image area of an image (e.g., a predicted high bit depth image, etc.) to no greater than the smoothness threshold Δ, which may be set as one of a maximum stair step size as in expression (9), an average stair step size as in expression (10), or an adaptive maximal stair step size as in expression (11), etc.). In paragraph [0089]-SU discloses in block 302, image areas with a high confidence level to have false contouring are determined in an image. Please also read paragraph [0056-0066]); and
generating, based on the de-banded image, an output image having a second dynamic range greater than the first dynamic range (Fig. 1. Paragraph [0119]-SU discloses the smoothen operator (410) is applied to the prediction image data (e.g., intermediate high dynamic range image, etc.) to generate smoothened prediction image data (e.g., a smoothened high bit depth image, etc.). Further in paragraph [0148]-SU discloses in block 502, a multi-layer video encoder (e.g., 402 of FIG. 4A) determines flat image areas in a predicted image, the predicted image being predicted from a first image mapped from a second image that has a higher dynamic range than the first image. Please also read paragraph [0122-0123, 0129, 0133 and 0139-0140]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU of having a processor comprising: one or more processing units to perform operations comprising: receiving an input image having a first dynamic range, with the teachings of SU of having generating, using a blur filter, a de-banded image based on the band size map and the input image, wherein the blur filter removes at least a portion of the at least one banding artifact from the input image; and generating, based on the de-banded image, an output image having a second dynamic range greater than the first dynamic range.
Wherein having WANG’s processor having generating, using a blur filter, a de-banded image based on the band size map and the input image, wherein the blur filter removes at least a portion of the at least one banding artifact from the input image; and generating, based on the de-banded image, an output image having a second dynamic range greater than the first dynamic range.
The motivation behind the modification would have been to obtain a processor that improves the accuracy and efficiency of banding artifact detection and removal, since both WANG and SU systems and methods for image analysis in the context of banding artifacts. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while SU provides systems and methods that provide optimal operational parameters for removing or reducing banding artifacts with relatively small distortion and improve bit depth expansion in the false contouring areas (or stair wise areas) significantly both perceptually and objectively. Please see WANG (US 20210321020 A1), Abstract and Paragraph [0010] and SU (US 20160021391 A1), Abstract and Paragraph [0031, 0084 and 0087].
Regarding claim 20, WANG explicitly teaches a system (Fig. 1. Paragraph [0052]-WANG discloses FIG. 9 illustrates an example computing device 900 for performing automated detection and assessment of visual banding effect in digital images and video signals. The computing device 900 may include memory 902, processor 904, and non-volatile storage 906) comprising:
one or more processors to perform operations (Fig. 1. Paragraph [0052]-WANG discloses the processor 904 may include one or more devices selected from high-performance computing (HPC) systems including high-performance cores, microprocessors, micro-controllers, digital signal processors, microcomputers, central processing units, field programmable gate arrays, programmable logic devices, state machines, logic circuits, analog circuits, digital circuits, or any other devices that manipulate signals (analog or digital) based on computer-executable instructions residing in memory 902. Further in paragraph [0054]-WANG discloses upon execution by the processor 904, the computer-executable instructions of the program instructions 908 may cause the computing device 900 to implement one or more of the algorithms and/or methodologies disclosed herein) comprising:
receiving an input image having a first dynamic range (Fig. 5. Paragraph [0018]-WANG discloses the present disclosure applies to both standard dynamic range (SDR) and high dynamic range (HDR) images and videos. In paragraph [0022]-WANG discloses the banding effect may be detected and assessed in HDR images/videos (wherein High dynamic range images/videos have a first dynamic range));
Although WANG explicitly teaches processing the input image to generate a band size map that identifies at least one banding artifact in the input image (Fig. 6. Paragraph [0042]-WANG discloses [0042] FIG. 6 illustrates an example method for generating banding maps. The banding map may be created for an image or video frame from the detected banding pixels. Specifically, starting from a blank map of the same size of an image or video frame, the positions of the detected banding pixels 600 in the image or video frame are marked on the blank map at operation 602. This process results in a binary map of the same size as the image or video frame being assessed, where the positions of detected banding pixels are marked as a first value (e.g., 1, true, etc.), and the rest of the map is marked as a second value (e.g., 0, false, etc). The signal activity is then normalized based on the significance threshold at operation 606, resulting in a normalized signal activity. The intensity value at the banding pixels in the map (marked as the first value) may then be replaced by the normalized signal activity measure at the pixels at operation 608. This leads to the banding map 610, where the region of the image with no banding detected remains at the second value, but at the banding pixels the intensity values are equal to the local normalized signal activity measurement), wherein the band size map specifies, for each of a plurality of pixels of the input image, a predicted size of a corresponding banding artifact that is at least partially depicted by the pixel (Fig. 6. Paragraph [0026]-WANG discloses contour tracing methods applied after the banding pixels are detected may be used to refine the connections and shapes of banding contours for better appearance, and may be an optional further processing for the purposes of banding detection and banding level assessment. The collection of detected banding pixels 208 may subsequently be processed by further operations to generate various measures (such as per-color channel or per-frame banding spread, banding strength, banding map, and areas/regions of banding) about the banding effect, and may also be aggregated to banding effect measurements at each spatial location (in an image or video frame), each time instance (frame) in a video, and at larger time scales (per-scene, per-second, per-minute, per-hour, and so on).) In paragraph [0042]-WANG discloses this banding map reveals how much the banding effect is visible at each spatial location in an image or video frame (wherein the banding maps are computed for all color channels at operation 704, a per-channel banding spread is computed by computing the percentage of areas in the image or video frame that contain banding pixels, a per-channel banding spread 710 and per-channel banding strength 712, are then combined to an overall banding level measure of the channel at operation 714 and the per-channel banding level measures are combined into a single per-frame banding level measure at operation 71, which is a scalar banding assessment). Please also read paragraph [0019, 0023-0030, 0040-0041 and 0043-0046]);
WANG fails to explicitly teach processing, using a neural network, the input image to generate a band size map that identifies at least one banding artifact in the input image.
However, XU explicitly teaches processing, using a neural network (Fig. 2, #34 called a feed forward network. Paragraph [0038]. Further in paragraph [0033]-XU discloses the encoder 16 implements a compression algorithm using encoder parameters. The artifact predictor 14 is adapted to generate an artifact prediction map, indicating the likelihood of whether the artifact will occur given use of a particular encoder parameter or range of encoder parameters. In paragraph [0035]-Xu discloses FIG. 2 is a block diagram illustrating a second example system 30 that is adapted to predict banding artifacts in reconstructed video. In paragraph [0038]-XU discloses the feed-forward neural network 34 is adapted to output banding metadata as a map (e.g., table or grid) predicting or indicating whether banding is predicted to occur as a function of QPs for each image block of each frame of video data. In paragraph [0062]-XU discloses regarding the architecture of the feed-forward neural network 34, given the features extracted from each image block, a the feed-forward neural network 34 is trained, e.g., via the training module 40, to learn a classifier that maps a feature vector to a binary value, e.g., 0 or 1, where 0 represents “no banding” and 1 represents “banding.” (wherein stochastic training and optimization and multiple filters, including a smoothing filter may be used)), the input image to generate a band size map that identifies at least one banding artifact in the input image (Fig. 1. Paragraph [0036]-XU discloses the second example system 30 includes a feature extractor module 32 for extracting feature data from input video data. A feed-forward neural network 34 is adapted to process the feature data to classify each video block (also called video patch, image block, or video frame patch) as “no banding” or “banding” for one or more QPs that may be used by the video encoder. In paragraph [0037]-XU discloses the example feature extractor module 32 is adapted to capture color and texture information for each block (e.g., a block representing 8×8 pixels of a video frame). In paragraph [0038]-XU discloses the feed-forward neural network 34 is adapted to output banding metadata as a map (e.g., table or grid) predicting or indicating whether banding is predicted to occur as a function of QPs for each image block of each frame of video data).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG of having a system comprising: one or more processors to perform operations comprising: receiving an input image having a first dynamic range; processing the input image to generate a band size map to generate a band size map that identifies at least one banding artifact in the input image, wherein the band size map specifies, for each of a plurality of pixels of the input image, a predicted size of a corresponding banding artifact that is at least partially depicted by the pixel, with the teachings of XU of having processing, using a banding detector neural network, the input image to generate a band size map that identifies at least one banding artifact in the input image.
Wherein having WANG’s system having processing, using a banding detector neural network, the input image to generate a band size map to generate a band size map that identifies at least one banding artifact in the input image, wherein the band size map specifies, for each of a plurality of pixels of the input image, a predicted size of a corresponding banding artifact that is at least partially depicted by the pixel.
The motivation behind the modification would have been to obtain a system that improves the accuracy and efficiency of banding artifact detection and removal, since both WANG and XU systems and methods for image and banding artifact analysis. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while XU provides systems and methods that improve the accuracy of banding prediction based on color and gradient features in YCbCr color spaces and use of a feed-forward neural network classifier 34. Please see WANG et al. (US 20210321020 A1), Abstract and Paragraph [0010] and XU et al. (US 20160360202 A1), Abstract and Paragraph [0069].
WANG in view of XU fails to explicitly teach generating, using a blur filter, a de-banded image based on the band size map and the input image, wherein the blur filter removes at least a portion of the at least one banding artifact from the input image based on the predicted size specified in the band size map; and generating, based on the de-banded image, an output image having a second dynamic range that is greater than the first dynamic range.
However, SU explicitly teaches generating, using a blur filter (Fig. 1. Paragraph [0067]-SU discloses a blur filter can be used at least as a part of the smoothen operator to remove banding artifacts in a high bit depth image. A convolution of a 2-D (separable or non-separable) low pass kernel, etc., may be used to generate smooth area over flat areas or stair wise areas), a de-banded image based on the band size map and the input image (Fig. 1. Paragraph [0031]-SU discloses techniques may be used to enhance prediction of wide or high dynamic range image data based on relatively low bit depth (e.g., 8 bits, etc.) image data in a base layer and reduce false contouring artifacts in reconstructed wide or high dynamic range (e.g., 12+ bits VDR, etc.) images. In paragraph [0032]-SU discloses decontouring techniques include smoothen operations performed by a smoothen operator, selection/masking operations between smoothened pixel values and unsmoothened pixel values, etc. False contouring artifacts can be effectively removed by performing decontouring filtering in the predicted image data using the smoothen operations, the selection/masking operations, etc. In paragraph [0055]-SU discloses an input image 101 (e.g., a VDR image, a predicted high bit depth image predicted from a low bit depth image, a decoded image that comprises banding artifacts, etc.) to which the decontouring operations are to be performed is denoted as v), wherein the blur filter removes at least a portion of the at least one banding artifact from the input image based on the predicted size specified in the band size map (Fig. 3. Paragraph [0084]-SU discloses a smoothness threshold Δ can be used as a constraint in selecting operational parameters for the smoothen operator. A residual mask can be specifically set to limit maximal distortions introduced by smoothen operations in any given image area of an image (e.g., a predicted high bit depth image, etc.) to no greater than the smoothness threshold Δ, which may be set as one of a maximum stair step size as in expression (9), an average stair step size as in expression (10), or an adaptive maximal stair step size as in expression (11), etc.). In paragraph [0089]-SU discloses in block 302, image areas with a high confidence level to have false contouring are determined in an image. Please also read paragraph [0056-0066]); and
generating, based on the de-banded image, an output image having a second dynamic range greater than the first dynamic range (Fig. 1. Paragraph [0119]-SU discloses the smoothen operator (410) is applied to the prediction image data (e.g., intermediate high dynamic range image, etc.) to generate smoothened prediction image data (e.g., a smoothened high bit depth image, etc.). Further in paragraph [0148]-SU discloses in block 502, a multi-layer video encoder (e.g., 402 of FIG. 4A) determines flat image areas in a predicted image, the predicted image being predicted from a first image mapped from a second image that has a higher dynamic range than the first image. Please also read paragraph [0122-0123, 0129, 0133 and 0139-0140]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU of having a system comprising: one or more processors to perform operations comprising: receiving an input image having a first dynamic range, with the teachings of SU of having generating, using a blur filter, a de-banded image based on the band size map and the input image, wherein the blur filter removes at least a portion of the at least one banding artifact from the input image based on the predicted size specified in the band size map; and generating, based on the de-banded image, an output image having a second dynamic range that is greater than the first dynamic range.
Wherein having WANG’s system having generating, using a blur filter, a de-banded image based on the band size map and the input image, wherein the blur filter removes at least a portion of the at least one banding artifact from the input image based on the predicted size specified in the band size map; and generating, based on the de-banded image, an output image having a second dynamic range that is greater than the first dynamic range.
The motivation behind the modification would have been to obtain a system that improves the accuracy and efficiency of banding artifact detection and removal, since both WANG and SU systems and methods for image analysis in the context of banding artifacts. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while SU provides systems and methods that provide optimal operational parameters for removing or reducing banding artifacts with relatively small distortion and improve bit depth expansion in the false contouring areas (or stair wise areas) significantly both perceptually and objectively. Please see WANG (US 20210321020 A1), Abstract and Paragraph [0010] and SU (US 20160021391 A1), Abstract and Paragraph [0031, 0084 and 0087].
Regarding claim 21, WANG in view of XU and in further view of SU explicitly teaches the system of claim 20, WANG further teaches wherein the system comprises at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more large language models (LLMs); a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources (Fig. 9. Paragraph [0052]-WANG discloses FIG. 9 illustrates an example computing device 900 for performing automated detection and assessment of visual banding effect in digital images and video signals. The non-volatile storage 906 may include one or more persistent data storage devices such as cloud storage. Please also read paragraph [0042]);
Claims 4-5 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over WANG et al. (US 20210321020 A1), hereinafter referenced as WANG in view of XU et al. (US 20160360202 A1), hereinafter referenced as XU and in further view of SU et al. (US 20160021391 A1), hereinafter referenced as SU and in further view of WANG et. al. (US 20230131228 A1), hereinafter referenced as WANG (2023).
Regarding claim 4, WANG in view of XU and in further view of SU explicitly teach the method of claim 3, although WANG further teaches wherein the at least one channel in the plurality of channels is associated with a plurality of binary values and at least one binary value in the plurality of binary values corresponds to a respective pixel of the input image (Fig. 6. Paragraph [0024]-WANG discloses at operation 202, the image or each video frame are decomposed into luminance or color channels. In paragraph [0026]-WANG discloses at operation 206, banding pixels 208 are determined. Banding pixels 208 may be determined as being any pixels in the image or video frame that 1) have significant signal activity at their locations as defined herein; and 2) where at least a minimum threshold percentage of their neighboring pixels have non-significant signal activity. In paragraph [0041]-WANG discloses the detected banding pixels 500 are measured for their signal activity at operation 502 (wherein if the signal activity is higher, than the pixel location is removed from the collection of detected banding pixels 508). In paragraph [0042]-WANG discloses this process results in a binary map of the same size as the image or video frame being assessed, where the positions of detected banding pixels are marked as a first value (e.g., 1, true, etc.), and the rest of the map is marked as a second value (e.g., 0, false, etc.) (wherein signal activity is normalized based on the significance threshold at operation 606). Please also read paragraph [0026, 0040 and 0043-0046]).
WANG in view of XU and in further view of SU fail to explicitly teach and at least one binary value in the plurality of binary values corresponds to a respective pixel of the input image and indicates whether a size of a banding artifact that includes the respective pixel is greater than a given threshold band size to which the given channel corresponds.
However, WANG (2023) explicitly teaches and at least one value in the plurality of values corresponds to a respective pixel of the input image and indicates whether a size of a banding artifact that includes the respective pixel is greater than a given threshold band size to which the given channel corresponds (Fig. 7. Paragraph [0108]-WANG discloses the luminance channel of the image and a banding edge map calculated according to Algorithm I are input to the BandingNet. An input image is larger than N×N (e.g., 1024×1024), the image can be split into N×N patches (e.g., 16 patches), and a respective banding score can be output for each patch. The respective banding scores of the patches can be combined to obtain a banding score for the image. Further in paragraph [0077]-WANG discloses at a test 705, if the banding score is greater than a threshold banding score, then UGC 702 can be input to a de_banding_net 706 (i.e., a trained DeBandingNet) to deband the image. The threshold banding score can be empirically selected such that below which banding artefacts in the image are not perceived by a human viewer. Please also read paragraph [0090-0095]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU and in further view of SU of having a method, comprising: receiving an input image having a first dynamic range, with the teachings of WANG (2023) of having and at least one value in the plurality of values corresponds to a respective pixel of the input image and indicates whether a size of a banding artifact that includes the respective pixel is greater than a given threshold band size to which the given channel corresponds.
Wherein having WANG’s method having wherein the at least one channel in the plurality of channels is associated with a plurality of binary values, and at least one binary value in the plurality of binary values corresponds to a respective pixel of the input image and indicates whether a size of a banding artifact that includes the respective pixel is greater than a given threshold band size to which the given channel corresponds.
The motivation behind the modification would have been to obtain a method that improves the accuracy and efficiency of banding artifact detection and removal, since both WANG and WANG (2023) systems and methods for image analysis in the context of banding artifacts. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while WANG (2023) provides systems and methods that use multiple machine learning models to efficiently detect and remove banding artifacts. Please see WANG (US 20210321020 A1), Abstract and Paragraph [0010] and WANG (US 20230131228 A1), Abstract.
Regarding claim 5, WANG in view of XU and in further view of SU and in further view of WANG (2023) explicitly teach the method of claim 4, WANG further teaches wherein at least one of the plurality of binary values corresponds to a particular pixel that is included in the at least one banding artifact, and wherein the predicted pixel location of the at least one banding artifact is identified by a location of the particular pixel in the image (Fig. 6. Paragraph [0042]-WANG discloses FIG. 6 illustrates an example method for generating banding maps. The banding map may be created for an image or video frame from the detected banding pixels. This process results in a binary map of the same size as the image or video frame being assessed, where the positions of detected banding pixels are marked as a first value (e.g., 1, true, etc.), and the rest of the map is marked as a second value (e.g., 0, false, etc.)).
Regarding claim 13, WANG in view of XU and in further view of SU explicitly teach the method of claim 1, WANG in view of XU and in further view of SU fail to explicitly teach wherein the neural network is trained based on an amount of loss between a target band size map associated with a training input image and a predicted band size map, wherein the predicted band size map is generated by the neural network based on the training input image.
However, WANG (2023) explicitly teaches wherein the neural network (Fig. 1. Paragraph [0043]-WANG discloses the ML models can be convolutional neural networks (CNNs). One of the machine-learning models is referred to herein as a BandingNet (and the first model) (wherein BandingNet is trained to receive an image as an input and output a banding score for the image) and the other machine-learning model is referred to as a DeBandingNet (and the second model) (wherein DeBandingNet is trained to receive an image as an input and output a banding score for the image)) is trained based on an amount of loss between a target band size map associated with a training input image and a predicted band size map, wherein the predicted band size map is generated by the neural network based on the training input image (Fig. 1. Paragraph [0106]-WANG discloses the technique 800 uses the banding edge Map, E, and the luminance plane Y of the first training image as input (i.e., [E, Y]) to the first model to train the first model. The first model is trained to output a banding score. The difference between the ground truth banding score and the banding score output during training can be used as a loss function, or as part of a loss function, that is used to optimize the parameter values of the BandingNet. Further in paragraph [0130]-WANG discloses a training input image 1402 is input to the DeBandingNet 1404. The DeBandingNet 1404 outputs a training debanded image 1406. The luminance plane of the training debanded image 1406A and a banding edge map are input to the BandingNet 1408, which outputs a banding score 1410 for the training debanded image 1406. An image difference 1414 is also obtained, using an operator 1412, as the difference between the training input image 1402 and the training debanded image 1406. A total loss value 1416, which combines the banding score 1410 and the image difference 1414 can be used to refine the parameters of the DeBandingNet 1404 during the training).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU and in further view of SU of having a method, comprising: receiving an input image having a first dynamic range, with the teachings of WANG (2023) of having wherein the neural network is trained based on an amount of loss between a target band size map associated with a training input image and a predicted band size map, wherein the predicted band size map is generated by the neural network based on the training input image.
Wherein having WANG’s method having wherein the neural network is trained based on an amount of loss between a target band size map associated with a training input image and a predicted band size map, wherein the predicted band size map is generated by the neural network based on the training input image.
The motivation behind the modification would have been to obtain a method that improves the accuracy and efficiency of banding artifact detection and removal, since both WANG and WANG (2023) systems and methods for image analysis in the context of banding artifacts. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while WANG (2023) provides systems and methods that use multiple machine learning models to efficiently detect and remove banding artifacts. Please see WANG (US 20210321020 A1), Abstract and Paragraph [0010] and WANG (US 20230131228 A1), Abstract.
Regarding claim 14, WANG in view of XU and in further view of SU and in further view of WANG (2023) explicitly teach the method of claim 13, WANG in view of XU and in further view of SU fail to explicitly teach wherein the target band size map is generated based on the training input image using a band size-finding algorithm that determines a distance from at least one pixel of the training input image to a nearest pixel that differs from the pixel by more than a threshold amount in at least one color channel.
However, WANG (2023) explicitly teaches wherein the target band size map is generated based on the training input image using a band size-finding algorithm that determines a distance from at least one pixel of the training input image to a nearest pixel that differs from the pixel by more than a threshold amount in at least one channel (Fig. 8. Paragraph [0092]-WANG discloses technique 800 generates a banding edge map for a first training image (wherein the map is computed using algorithm I). In paragraph [0095]-WANG discloses algorithm I converts the RGB data to the YUV color space, which includes a luminance (Y) channel and two chrominance (U and V) channels). In paragraph [0096]-WANG discloses algorithm I calculates the horizontal and vertical gradient components at each pixel location of the luminance. The gradient measures the amount of change in pixel values at a pixel location. Gradient information can be used to identify the boundaries between areas of the image, which are characterized by significant color changes. In paragraph [0105]-WANG discloses the banding edge map is calculated as the product of the weight map W with the gradient map G. Further at paragraph [0106]-WANG discloses the technique 800 uses the banding edge Map, E, and the luminance plane Y of the first training image as input to the first model to train the first model. The difference between the ground truth banding score and the banding score output during training can be used as a loss function that is used to optimize the parameter values of the BandingNet. Please also read paragraph [0110]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU and in further view of SU of having a method, comprising: receiving an input image having a first dynamic range, with the teachings of WANG (2023) of having wherein the target band size map is generated based on the training input image using a band size-finding algorithm that determines a distance from at least one pixel of the training input image to a nearest pixel that differs from the pixel by more than a threshold amount in at least one color channel.
Wherein having WANG’s method having wherein the target band size map is generated based on the training input image using a band size-finding algorithm that determines a distance from at least one pixel of the training input image to a nearest pixel that differs from the pixel by more than a threshold amount in at least one color channel.
The motivation behind the modification would have been to obtain a method that improves the accuracy and efficiency of banding artifact detection and removal, since both WANG and WANG (2023) systems and methods for image analysis in the context of banding artifacts. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while WANG (2023) provides systems and methods that use multiple machine learning models to efficiently detect and remove banding artifacts. Please see WANG (US 20210321020 A1), Abstract and Paragraph [0010] and WANG (US 20230131228 A1), Abstract.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over WANG et al. (US 20210321020 A1), hereinafter referenced as WANG in view of XU et al. (US 20160360202 A1), hereinafter referenced as XU and in further view of SU et al. (US 20160021391 A1), hereinafter referenced as SU and in further view of WANG et. al. (US 20230131228 A1), hereinafter referenced as WANG (2023) and in further view of SABUI et al. (US 11259040 B1), hereinafter referenced as SABUI.
Regarding claim 6, WANG in view of XU and in further view of SU and in further view of WANG (2023) explicitly teach the method of claim 4, WANG in view of XU and in further view of SU and in further view of WANG (2023) fail to explicitly teach wherein the given threshold band size is in a sequence of progressively increasing threshold band sizes associated with the band size map.
However, SABUI explicitly teaches wherein the given threshold band size is in a sequence of progressively increasing threshold band sizes associated with the band size map (Fig. 1. Col. [05], Line [24-27]-SABUI discloses the one or more machine learning modules 136 may determine a banding risk of segments of the video frames 132 (e.g., segments of a video file). Further at Col. [06], Line [04-28]-SABUI discloses video frame susceptible to banding or other artifacts may be assigned a banding index associated with the type of artifact. The one or more machine learning modules 218 may use multiple thresholds (e.g., for high, medium, and low risks) to classify segments of the video frames 214 based on the respective index of a frame in a respective video segment. Additionally, at Col. [06], Line [36-46]-SABUI discloses the machine learning modules 136 may assign a video segment having the video frame an index indicative of a higher banding visibility (e.g., a score between 0 and 1, with 1 being the highest risk of banding visibility) (wherein visibility may correspond to the size of a banding artifact)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU and in further view of SU and in further of WANG (2023) of having a method, comprising: receiving an input image having a first dynamic range, with the teachings of SABUI of having wherein the given threshold band size is in a sequence of progressively increasing threshold band sizes associated with the band size map.
Wherein having WANG’s method having wherein the given threshold band size is in a sequence of progressively increasing threshold band sizes associated with the band size map.
The motivation behind the modification would have been to obtain a method that improves the accuracy and efficiency of banding artifact detection and removal, since both WANG and SABUI systems and methods for image and banding artifact analysis. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while SABUI provides systems and methods that improve the performance of devices, encoding, transmission, storage, and playback of video by using a risk-based approach which accounts for the risk of banding and other compression artifacts in individual video frames or segments. Please see WANG (US 20210321020 A1), Abstract and Paragraph [0010] and SABUI (US 11259040 B1), Abstract and Col. [03], Line [27-53].
Claims 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over WANG et al. (US 20210321020 A1), hereinafter referenced as WANG in view of XU et al. (US 20160360202 A1), hereinafter referenced as XU and in further view of SU et al. (US 20160021391 A1), hereinafter referenced as SU and in further view of KARAM et al. (C. Karam, C. Chen and K. Hirakawa, "Stochastic bilateral filter for high-dimensional images," 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada, 2015, pp. 192-196, doi: 10.1109/ICIP.2015.7350786), hereinafter referenced as KARAM.
Regarding claim 11, WANG in view of XU and in further view of SU explicitly teach the method of claim 1, WANG in view of XU and in further view of SU fail to explicitly teach wherein the blur filter is a bilateral blur filter that samples one or more pixels of the input image.
However, KARAM explicitly teaches wherein the blur filter is a bilateral blur filter that samples one or more pixels of the input image (Page 192, Col. [01], Para. [01]-KARAM discloses a bilateral filter is a image smoothing filter. Moreover, at Page 192, Col. [02], Para. [02]-KARAM discloses Bilateral filtering is particularly suited for hyperspectral imaging, in the sense that the increased spectral samples collectively strengthen the evidence of edge presence (wherein a gaussian filter is used to model the distribution of pixels. Further at Page 193, Col. [02], Para. [03]-KARAM discloses we propose a novel approach called stochastic bilateral filtering (SBF) that replaces the deterministic range kernel ϕ(⋅,⋅) with a random convolutional kernel, whose rate of Monte-Carlo convergence does not depend on the color/spectral dimensionality of the edge image e(i).)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU and in further view of SU of having a method, comprising: receiving an input image having a first dynamic range, with the teachings of KARAM of having wherein the blur filter is a bilateral blur filter that samples one or more pixels of the input image.
Wherein having WANG’s method having wherein the blur filter is a bilateral blur filter that samples one or more pixels of the input image.
The motivation behind the modification would have been to obtain a method that improves the accuracy and efficiency of banding artifact detection and removal as well as the speed of filtering operations, since both WANG and KARAM systems and methods concern image analysis. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while KARAM provides systems and methods that provide a stochastic bilateral filter (SBF) that improves the speed of bilateral filtering (BF) for high dimensional image data. Please see WANG (US 20210321020 A1), Abstract and Paragraph [0010] and KARAM et al. (C. Karam, C. Chen and K. Hirakawa, "Stochastic bilateral filter for high-dimensional images," 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada, 2015, pp. 192-196, doi: 10.1109/ICIP.2015.7350786, Abstract.
Regarding claim 12, WANG in view of XU and in further view of SU explicitly teach the method of claim 11, WANG in view of XU and in further view of SU fail to explicitly teach wherein the bilateral blur filter is a stochastic bilateral blur filter that samples a subset of pixels of the input image, wherein each pixel in the subset is selected based on a probability of using the pixel, and wherein the probability of using the pixel is determined based on a random distribution.
However, KARAM explicitly teaches wherein the bilateral blur filter is a stochastic bilateral blur filter that samples a subset of pixels of the input image (Page 192, Col. [01], Para. [01]-KARAM discloses a bilateral filter is a image smoothing filter. Bilateral filter is a image smoothing filter. Image features such as edges and textures are preserved by the spatial kernel and range kernel that limit the averaging in bilateral filters to pixels that are spatially close and similar in intensity. Moreover, at Page 192, Col. [02], Para. [02]-KARAM discloses bilateral filtering is particularly suited for hyperspectral imaging, in the sense that the increased spectral samples collectively strengthen the evidence of edge presence), wherein each pixel in the subset is selected based on a probability of using the pixel, and wherein the probability of using the pixel is determined based on a random distribution (Page 193, Col. [02], Para. [03]-KARAM discloses we propose a novel approach called stochastic bilateral filtering (SBF) that replaces the deterministic range kernel ϕ(⋅,⋅) with a random convolutional kernel, whose rate of Monte-Carlo convergence does not depend on the color/spectral dimensionality of the edge image e(i).)). The significance of the lemma is that it replaces the range Gaussian kernel with the expected value of a random phenomenon. We may approximate the quantity in (8) by generating the normal random vector L times, evaluating cos (XTe) each time, and taking their ensemble average. This random approach will converge to the Gaussian function in the mean square error sense with at a rate upper-bounded by 2i).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU and in further view of SU of having a method, comprising: receiving an input image having a first dynamic range, with the teachings of KARAM of having wherein the bilateral blur filter is a stochastic bilateral blur filter that samples a subset of pixels of the input image, wherein each pixel in the subset is selected based on a probability of using the pixel, and wherein the probability of using the pixel is determined based on a random distribution.
Wherein having WANG’s method having wherein the bilateral blur filter is a stochastic bilateral blur filter that samples a subset of pixels of the input image, wherein each pixel in the subset is selected based on a probability of using the pixel, and wherein the probability of using the pixel is determined based on a random distribution.
The motivation behind the modification would have been to obtain a method that improves the accuracy and efficiency of banding artifact detection and removal as well as the speed of filtering operations, since both WANG and KARAM systems and methods concern image analysis. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while KARAM provides systems and methods that provide a stochastic bilateral filter (SBF) that improves the speed of bilateral filtering (BF) for high dimensional image data. Please see WANG (US 20210321020 A1), Abstract and Paragraph [0010] and KARAM et al. (C. Karam, C. Chen and K. Hirakawa, "Stochastic bilateral filter for high-dimensional images," 2015 IEEE International Conference on Image Processing
Claims 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over WANG et al. (US 20210321020 A1), hereinafter referenced as WANG in view of XU et al. (US 20160360202 A1), hereinafter referenced as XU and in further view of SU et al. (US 20160021391 A1), hereinafter referenced as SU and in further view of HUANG et. al. (US 20170103729 A1), hereinafter referenced as HUANG and in further view of IWAKI et. al. (US 20160286090 A1), hereinafter referenced as IWAKI.
Regarding claim 16, WANG in view of XU and in further view of SU explicitly teach the method of claim 1, WANG in view of XU and in further view of SU fail to explicitly teach wherein generating, based on the de-banded image, an output image comprises: increasing a saturation of one or more red, green, or blue values of at least one pixel of the output image.
However, HUANG explicitly teaches wherein generating, based on the de-banded image (Fig. 1. Paragraph [0065]-HUANG discloses the tone mapping module 12D includes a de-banding filter module 44, a mapping curve generator module 20B, a source map lookup table module 46, a color space conversion module 48, a saturation control module 40B, and a HRD-SDR mapping module 22C), an output image comprises:
increasing a saturation of one or more red, green, or blue values of at least one pixel of the output image (Fig. 1. Paragraph [0055]-HUANG discloses the saturation control module 40A controls saturation of red, green, blue values to prevent out of gamut values from being clipped and causing false colors in the image or video. A maximum saturation value L.sub.MaxSat for the red, green, blue values R′.sub.IN, G′.sub.IN, B′.sub.IN from the delay line buffer is calculated (wherein multiple equations are used to determine a maximum saturation level for each color and to map saturated colors to a standard dynamic range, including a catch all pseudocode statements indicating a saturation level any large number and/or having no limit). Moreover, in paragraph [0068]-HUANG discloses the de-banding filter module 44 filters banding noise in the image or video. Particularly, the de-banding filter module 44 applies a de-banding filter to red, green, blue values R′.sub.N, G′.sub.IN, B.sub.IN from the delay line buffer module 36 to obtain de-band filtered red, green, blue values R.sub.DB, G.sub.DB, B.sub.DB); and
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU and in further view of SU of having a method, comprising: receiving an input image having a first dynamic range, with the teachings of HUANG of having wherein generating, based on the de-banded image, an output image comprises: increasing a saturation of one or more red, green, or blue values of at least one pixel of the output image.
Wherein having WANG’s method having wherein generating, based on the de-banded image, an output image comprises: increasing a saturation of one or more red, green, or blue values of at least one pixel of the output image.
The motivation behind the modification would have been to obtain a method that improves the accuracy and efficiency of banding artifact detection and removal, since both WANG and HUANG systems and methods for image analysis in the context of banding artifacts. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while HUANG provides systems and methods that uses a de-banding filter in conjunction with compressing the number of RGB values and improving processing time. Please see WANG (US 20210321020 A1), Abstract and Paragraph [0010] and HUANG (US 20170103729 A1), Abstract and Paragraph [0050 and 0067].
WANG in view of XU and in further view of SU fail to explicitly teach reducing a chromaticity value of the at least one pixel of the output image to a reduced chromaticity value for which each Red, Green, and Blue (RGB) value is in a predetermined range.
However, IWAKI explicitly teaches reducing a chromaticity value of the at least one pixel of the output image to a reduced chromaticity value for which each Red, Green, and Blue (RGB) value is in a predetermined range (Fig. 1. Paragraph [0155]-IWAKI discloses the low-chromaticity color space transform unit 102 is an example of the chroma/chromaticity optimizing unit 84 illustrated in FIG. 1. After the input transform by the input transform unit 14, the low-chromaticity color space transform unit 102 performs transform processing of transforming input image data in an input color space, for example, input image data in an RGB color space, into image data in a first color space composed of brightness and chromaticity, such as a Ycbcr color space and an L*a*b* color space, and further transforming the first color space into a low-chromaticity color space in which the chromaticity of the first color space has decreased to acquire transformed image data. Please also read paragraph [0158 and 0161]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU and in further view of SU and in further view of HUANG of having a method, comprising: receiving an input image having a first dynamic range, with the teachings of IWAKI of having reducing a chromaticity value of the at least one pixel of the output image to a reduced chromaticity value for which each Red, Green, and Blue (RGB) value is in a predetermined range.
Wherein having WANG’s method having reducing a chromaticity value of the at least one pixel of the output image to a reduced chromaticity value for which each Red, Green, and Blue (RGB) value is in a predetermined range.
The motivation behind the modification would have been to obtain a method that improves the accuracy of the color space transform as well as banding artifact detection and removal, since both WANG and IWAKI systems and methods for image analysis. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while IWAKI provides systems and methods that improve the utilization rate of the input color space and thus to improve the accuracy of transform. Please see WANG et al. (US 20210321020 A1), Abstract and Paragraph [0010] and IWAKI et al. (US 20160286090 A1), Abstract and Paragraph [0077-0078].
Regarding claim 17, WANG in view of XU and in further view of SU and in further view of HUANG and in further view of IWAKI explicitly teach the method of claim 16, WANG in view of XU and in further view of SU and in further view of HUANG fail to explicitly teach wherein reducing the chromaticity value of the at least one pixel of the output image comprises converting the output image from Red Green Blue (RGB) color space to CIE-LAB or OkLab, or a different LAB color space, or Luminance Chroma Hue (LCH) color space associated with CIE-LAB or OkLab, or a different LAB color space.
However, IWAKI explicitly teaches wherein the chromaticity value of the at least one pixel of the output image comprises converting the output image from Red Green Blue (RGB) color space to CIE-LAB or OkLab, or a different LAB color space, or Luminance Chroma Hue (LCH) color space associated with CIE-LAB or OkLab, or a different LAB color space (Fig. 1. Paragraph [0155]-IWAKI discloses the low-chromaticity color space transform unit 102 is an example of the chroma/chromaticity optimizing unit 84 illustrated in FIG. 1. After the input transform by the input transform unit 14, the low-chromaticity color space transform unit 102 performs transform processing of transforming input image data in an input color space, for example, input image data in an RGB color space, into image data in a first color space composed of brightness and chromaticity, such as a Ycbcr color space and an L*a*b* color space, and further transforming the first color space into a low-chromaticity color space in which the chromaticity of the first color space has decreased to acquire transformed image data. Please also read paragraph [0158 and 0161]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of XU and in further view of SU and in further view of HUANG of having a method, comprising: receiving an input image having a first dynamic range, with the teachings of IWAKI of having wherein reducing the chromaticity value of the at least one pixel of the output image comprises converting the output image from Red Green Blue (RGB) color space to CIE-LAB or OkLab, or a different LAB color space, or Luminance Chroma Hue (LCH) color space associated with CIE-LAB or OkLab, or a different LAB color space.
Wherein having WANG’s method having wherein reducing the chromaticity value of the at least one pixel of the output image comprises converting the output image from Red Green Blue (RGB) color space to CIE-LAB or OkLab, or a different LAB color space, or Luminance Chroma Hue (LCH) color space associated with CIE-LAB or OkLab, or a different LAB color space.
The motivation behind the modification would have been to obtain a method that improves the accuracy of the color space transform as well as banding artifact detection and removal, since both WANG and IWAKI systems and methods for image analysis. Wherein WANG provides systems and methods for automated detection, assessment and removal of visual banding effect in digital images and video signals that balances accuracy and efficiency, while IWAKI provides systems and methods that improve the utilization rate of the input color space and thus to improve the accuracy of transform. Please see WANG et al. (US 20210321020 A1), Abstract and Paragraph [0010] and IWAKI et al. (US 20160286090 A1), Abstract and Paragraph [0077-0078].
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered pertinent to applicant`s disclosure.
AYDIN et al. (US 20220084170 A1)- The exemplary embodiments relate to converting Standard Dynamic Range (SDR) content to High Dynamic Range (HDR) content using a machine learning system. In some embodiments, the neural network is trained to convert an input SDR image into an HDR image using the encoded representation of a training SDR image and a training HDR image. In other embodiments, the neural network is trained to convert an input SDR image into an HDR image using a predefined set of color grading actions and the training images........................... Please see Fig. 1-4. Abstract.
STROM et al. (US 20190246123 A1)- A processing for a first pixel in a picture comprises obtaining a lower limit of a first color component of the first pixel in a first color space based on a distance between a color of the first pixel and a first distorted version of the color in a second color space. An upper limit of the first color component in the first color 5 space is obtained based on a distance between the color and a second distorted version of the color in the second color space. A filtered value is obtained of the first color component and which is equal to or larger than the lower limit and equal to or lower than the upper limit. The processing results in filtered values that are cheaper to encode but that are visibly undistinguishable from the original colors of the pixels......................... Please see Abstract and Fig. 1-6
WANG et al. (US 20210233259 A1)- No-reference (NR) quality assessment (VQA) of a test visual media input encoding media content is provided. The test video visual media input is decomposed into multiple-channel representations. Domain knowledge is obtained by performing content analysis, distortion analysis, human visual system (HVS) modeling, and/or viewing device analysis. The multiple-channel representations are passed into deep neural networks (DNNs) producing DNN outputs. The DNN outputs are combined using domain knowledge to produce an overall quality score of the test visual media input....................... Please see Fig. 1-5. Abstract.
YANG et al. (Yang, Y., Xiong, Y., Cao, Y. et al. Fast bilateral filter with spatial subsampling. Multimedia Systems 29, 435–446 (2023). https://doi.org/10.1007/s00530-022-01004-7)- The bilateral filter is a non-linear edge-preserving filter that can be adopted in a variety of tasks in computer photography. However, the naive bilateral filter is computationally expensive. Existing researches on the acceleration of bilateral filter mostly concentrate on range approximation. Nevertheless, the range kernel has more impact on the bilateral filter than the spatial kernel. Range approximation would have more side effects. In this paper, we propose a novel approximation of the bilateral filter with spatial subsampling, where the affinity matrix is estimated from a subset of it. We show that the main computational burden of our approximation is a large linear system, for which we propose an efficient iterative algorithm to solve. We have carried out both quantitative and qualitative experiments to evaluate our fast bilateral filter. Experimental results suggest that the proposed filter outperforms the state-of-the-art methods in approximation accuracy. The proposed filter is highly efficient; under a moderate sampling rate, i.e., it needs 0.29s to process a color image with 1 megapixel on an Intel i7-9700 CPU......................... Please see Abstract.
ZHANG et al. (US 20220067890 A1)- The exemplary embodiments relate to converting standard dynamic range (SDR) content to high dynamic range (HDR) content. An SDR image may be decomposed into a base layer of the SDR image that includes low frequency information from the SDR image and a detail layer of the SDR image that includes high frequency information from the SDR image. A base layer of an HDR image may be generated using the base layer of the SDR image and a detail layer of the HDR image may be generated using the detail layer of the SDR image. An HDR image is then generated using the base layer of the HDR image and the detail layer of the HDR image...................... Please see Fig. 2-5. Abstract.
GADJIL et al. (US 20210350511 A1)- Methods and systems for reducing banding artifacts when displaying high-dynamic-range images are described. Given an input image in a first dynamic range, and an input backward reshaping function mapping codewords from the first dynamic range to a second dynamic range, wherein the second dynamic range is equal or higher than the first dynamic range, statistical data based on the input image and the input backward reshaping function are generated to estimate the risk of banding artifacts in a target image in the second dynamic range generated by applying the input backward reshaping function to the input image. Separate banding alleviation algorithms are applied in the darks and highlights parts of the first dynamic range to generate a modified backward reshaping function, which when applied to the input image to generate the target image eliminates or reduces banding in the target image.......................... Please see Fig. 1-3. Abstract.
CHOU et al. (US 20160307298 A1)- An image signal processing system may include processing circuitry that may reduce banding artifacts in image data to be depicted on a display. The processing circuitry may receive a first pixel value associated with a first pixel of the image data and detect a first set of pixels located in a first direction along a same row of pixels or a same column of pixels with respect to the first pixel. The first set of pixels is associated with a first band. The processing circuitry may then interpolate a second pixel value based on an average of a first set of pixel values that correspond to the first set of pixels and a distance between the first pixel and a closest pixel in the first band. The processing circuitry may then output the second pixel value for the first pixel........................ Please see Fig. 1, 6 and 11. Abstract.
TU et al. (Z. Tu, J. Lin, Y. Wang, B. Adsumilli and A. C. Bovik, "Adaptive Debanding Filter," arXiv:2009.10804v1 [eess.IV] 22 Sep 2020)- Banding artifacts, which manifest as staircase-like color bands on pictures or video frames, is a common distortion caused by compression of low-textured smooth regions. These false contours can be very noticeable even on high-quality videos, especially when displayed on high-definition screens. Yet, relatively little attention has been applied to this problem. Here we consider banding artifact removal as a visual enhancement problem, and accordingly, we solve it by applying a form of content-adaptive smoothing filtering followed by dithered quantization, as a post-processing module. The proposed debanding filter is able to adaptively smooth banded regions while preserving image edges and details, yielding perceptually enhanced gradient rendering with limited bit-depths. Experimental results show that our proposed debanding filter outperforms state-of-the-art false contour removing algorithms both visually and quantitatively......................... Please see Fig. 2-3. Abstract.
KANNAN et al. (US 20220197505 A1)- One or more performance parameters associated with data stored at a storage device of a plurality of storage devices are received by a storage controller. A first number of blocks of the storage device to a high resiliency portion and a second number of blocks of the storage device to a low resiliency portion of the storage device are allocated based on the one or more performance parameters......................... Please see Abstract and Para. [0032, 0072, 0124, 0142, 0187, 0191, and 0196] with regard to claim 21.
THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to Aaron Bonansinga whose telephone number is (703) 756-5380 The examiner can normally be reached on Monday-Friday, 9:00 a.m. - 6:00 p.m. ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s
supervisor, Chineyere Wills-Burns can be reached by phone at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON TIMOTHY BONANSINGA/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights