Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed September 2nd, 2014 has been entered. Claims 1, 3-21, remain pending in the application.
Response to Arguments
Applicant’s arguments, see pages 7-12 of remarks/arguments, filed 7/28/2026, with respect to claims 1, 3-19 have been fully considered and are persuasive. The 35 USC 102(a)(1) and 103 of claims 1, 3-19 has been withdrawn.
It is agreed that the claims gain value is not analogous to Lashdan’s filter size, and that the amended language that ties determining a gain value in response to proportional surface area check against a proportional threshold value is not found in Lashdan, or in the combinations of Lashdan with the other named prior art. This places claims 1, 11, and their subsequent dependent claims in condition for allowance. As claim 20 lacks this limiting language referring to how a gain value is determined via the threshold check, a rejection was able to be made seen below.
Allowable Subject Matter
Claims 1-19 are allowed.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 20, 21 are rejected under 35 U.S.C. 103 as being unpatentable over Lashdan (WO 2017205010 A1) in view of Shen (Review of Postprocessing Techniques for Compression Artifact Removal) in view of Faroudja (US9445131B1) and further in view of Zhu (No-reference Video Quality Assessment and Applications).
Regarding claim 20,
Lashdan teaches:
A display device comprising: a controller configured to acquire a surface area of a flat area having few detail components on the basis of image data, (Lashdan ¶3 “In video processing, there may be instances in which contouring visual artifacts are observed in regions of a video image with very low texture. The contours formed in low texture regions by pixels with the same or similar level may be referred to as contouring artifacts, and more typically banding artifacts. In some instances, these banding artifacts may result from the quantization of regions or areas of a video image that have low gradients or ramps (e.g., gradients or ramps with small slopes). These regions or areas may be referred to as flat areas of a video image. For example, the quantization of low gradient areas to 8 bits may result in banding artifacts.” ¶106 “Returning to the first step of banding detection, as shown in Equation (1) below, a current filter size for a debanding filter (e.g., a low pass filter (LPF)) is applied to the original sample of a target pixel location (x) to produce a filtered sample (LPF(x)). | LPF(x) – x | < a (1) If the difference between the filtered sample and the original sample (e.g., the difference between the filtered pixel value and the original pixel value) is less than a threshold (α), then the first step is said to have been passed or met by the current size of the debanding filter. Passing the first step may indicate that the difference due to filtering is small and, therefore, due to truncation. Accordingly, passing the first step may indicate that the area is flat and potentially has banding, and that the current size of the debanding filter is good for that area.” Note: Lashdan teaches that flat areas in an image are found via a process of applying filters in an effort to find where contour noise/artifacts, also referred to as banding.) acquire a filter size for removing contour noise on the basis of the surface area of the flat area, wherein the surface area of the flat area is in the image area (Lashdan ¶107 “the second step of banding detection includes analyzing the area (e.g., groups of consecutive pixels in a row or column of a video image) being considered by the debanding filter to determine whether all non-zero gradients within a filter kernel have the same sign and are smaller than a threshold. A filter kernel may refer to a matrix or masking operation that is to be performed on pixels associated with a target pixel. In some instances, this threshold may be the same as the threshold ( ) used in the first step of banding detection (see e.g., Equation (1)), while in other instances it may be different. Additional aspects related to the second step or action of banding detection are provided below in more detail with respect to FIGs. 8A-10.” ¶114 “This scheme may involve both the banding detection (e.g., banding artifact detection) and the adaptive debanding filtering described above. In general, this scheme starts with a small filter size and iteratively increases the filter size until a largest, suitable filter size is identified for a particular pixel location (e.g., a target pixel location).”
PNG
media_image1.png
962
640
media_image1.png
Greyscale
Note: Lashdan compares the results of a filter against an original image and checks the difference against a threshold to determine if there is banding present in the area. If a filter passes the threshold, then we know the given filter size describes a flat area of that size, and we increase filter sizes until a filter fails the threshold check, in which case we default to the last largest filter.) and generate result image data on the basis of the filter size; (¶154 “At 1512, the method 1500 may include performing a first banding artifact correction in a first direction on a target pixel location in the video data based on a first debanding filter. For example, the vertical banding detection/filtering 712 in FIG. 7A may perform a first banding artifact correction (e.g., video debanding) in the vertical direction. In another example, the horizontal banding detection/filtering 732 in FIG. 7B may perform a first banding artifact correction (e.g., video debanding) in the horizontal direction.” Note: After the largest possible target pixel areas containing contour noise/banding are found, the artifacts are removed to produce a corrected image.) and a display configured to output an image on the basis of the result image data, (Lashdan ¶96 “The video debanding component 710 may perform various aspects described herein for video debanding that uses adaptive filter sizes and gradient based banding detection (also referred to as gradient based banding artifact detection). For example, the video debanding component 710 may receive image data in the form of decoded video images, where the image data may have pixel values of a first bit depth. In an example, the bit depth may be 8 bits since this is a typical number of bits used to represent colors and/or intensity levels in a pixel for display and/or storage purposes.” ¶43 “A destination device as described above may include an input interface, a video decoder, and a display device. A display device as described herein may display video data to a user, and may comprise any of a variety of display devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.” Note: Lashdan teaches a display device that can display the corrected output image.)
While Lashdan teaches finding the surface area of a flat area to it does not teach doing so on the basis of a high frequency component of a frequency domain converted from the image data. This is detailed in Shen which teaches and wherein the controller is configured to convert the image data into a frequency domain and acquire the surface area of the flat area on the basis of a high frequency component of the converted frequency domain. (Shen 3 “However, due to perceptual masking [27], the high frequency noise in the flat region is more visible to human eyes and thus requires special attention for perceived quality improvement.” Shen 5 “The coded bitstream is first dequantized. Each 8x8 block in an image is classified into two categories: flat block, and high activity block. There exist many classification rules. Let V, H, and D be three directional indices that correspond to different edge directions of the block in spatial domain … Let M be the maximum among three indices V, H, and D. If M is less than a threshold value, the block is classified as a flat block; otherwise, it is classified as a high activity block, which can be either a texture or an edge block. The threshold value T determines the sensitivity of classifier and it is selected from training images such that the probability of misclassification error is minimized. In our implementation, we chose T between 20 to 30.”
PNG
media_image2.png
480
442
media_image2.png
Greyscale
PNG
media_image3.png
722
872
media_image3.png
Greyscale
Note: Shen teaches a DCT, discrete cosine transform, is used on image data. DCT is a known common method of transforming or converting image data into a frequency domain. Shen 5 directly teaches that blocks in the frequency domain are determined to be flat areas or high frequency, non-flat areas, and that this is done because flat areas allow for noise to be more visually apparent. Aside from just identifying which areas are flat areas, the surface area of given flat areas is found as seen in Fig. 5 where flat areas are visualized as black blocks, showing the surface areas of flat areas are mapped for the entire image using a frequency domain. The flat areas are identified by being below a threshold the high frequency areas pass, and the ultimate surface areas of the flat areas is bounded by those high frequency areas, teaching that high frequency components are involved in determining the surface area of the flat areas. Fig. 1 is provided as a reference showing the original image before it has been converted into a frequency domain via DCT.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Lashdan with Shen where determining the surface area of flat areas is done using a high frequency data from a frequency domain converted from the image data.
There are several reasons that would motivate one to do so, determining whether or not there is noise present in a given area can be difficult since some areas contain more detail, making them higher frequency, than others, and thus the noise is harder to see and less important to correct. Identifying the areas that needs the most correction can be greatly simplified by using a frequency domain, as rather than having to evaluate the surface area one pixel at a time we can use the grouped blocks of pixels in the frequency domain to streamline the process.
While Lashdan teaches evaluating the amount of flat area it does not teach that the proportional amount of flat area to the total image area is found. Finding a proportional surface area of flat area to the total image is taught by Zhu wherein the surface area of the flat area is a proportion of an image area (Zhu p. 64 “In the Laplacian pyramid, the subband LK−1 in the highest level preserves the coarsest features of the image I. Meanwhile, the flat region is the lack of fine details, so almost all the features in the flat region are preserved in the highest level subband. Therefore, the image I has a high similarity with the subband LK−1 in flat regions. To identify flat regions based on the Laplacian pyramid, the SSIM index map between I and LK−1 is a reasonable option. When the SSIM index in a local region is greater than T0, the local region is considered to be a flat region. The smoothness of the frame is defined as the proportion of flat regions, which is given by:”)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Lashdan with Zhu where the surface area of the flat area is a proportion of an image area.
There are several reasons that would motivate one to do so, the present invention dictates correction amount by increasing or decreasing a gain value that controls the strength of a blurring filter, and has the means to correct different levels of banding. To apply the filter and correct as much as reasonably possible without over correcting and blurring regions that do not need blurring the strength of the filter could leverage the proportion of flat area as a reasonable measure of how much error is present.
As mentioned above, the present invention leverages a gain value that scales a blur/smoothing filter. While Lashdan teaches setting the filter size based on the flat area, which will cause more total area to be blurred, it does not change the intensity blurring filter being applied. This is taught in which teaches acquire a gain value for removing contour noise (Faroudja Col. 2 Line 36 “In one embodiment of the invention, a de-blocking and de-banding filter for a video signal processing is disclosed. This de-blocking and de-banding filter comprises: a smart filter strength control unit that receives a current gray level (Y) and at least one of a compression level (Qp) and a bitrate of a video signal to generate an adjustable filter strength level (T); and a de-blocking and de-banding unit containing a signal filter, wherein the de-blocking and de-banding unit receives the adjustable filter strength level (T) from the smart filter strength control unit, and wherein the de-blocking and de-banding unit is configured to reduce or remove a blocking artifact or a banding artifact with the signal filter at a variable filter strength controlled by the adjustable filter strength level (T).” Note: Faroudja teaches a filter strength level T that is adjustable and dictates the strength of a de-banding filter, analogous to the claims gain value. )
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Lashdan with Faroudja where the surface area of a flat area is used as the basis to acquire a gain value instead of a filter size.
There are several reasons that would motivate one to do so, as surface area is proportional it provides a good immediate measure of how much banding is present in an image and thus a good basis for how strong the correction applied should be.
Regarding claim 21,
The display device according to claim 20,
Lashdan does not teach that its image data is divided into blocks in order to determine which pixel grouped areas have a high frequency and which are flat. This is taught in Shen which teaches wherein the controller is configured to divide a screen into a plurality of blocks, extract blocks, in which the high frequency component is greater than a preset reference value, (Shen 5 “The coded bitstream is first dequantized. Each 8x8 block in an image is classified into two categories: flat block, and high activity block. There exist many classification rules. Let V, H, and D be three directional indices that correspond to different edge directions of the block in spatial domain … Let M be the maximum among three indices V, H, and D. If M is less than a threshold value, the block is classified as a flat block; otherwise, it is classified as a high activity block, which can be either a texture or an edge block. The threshold value T determines the sensitivity of classifier and it is selected from training images such that the probability of misclassification error is minimized. In our implementation, we chose T between 20 to 30.” Note: Shen teaches that once the image data has been transformed into a frequency domain and divided into a plurality of 8x8 blocks a threshold, or the claims “preset reference value” is checked to determine whether or not the block is high frequency or a flat area. If it is higher than the threshold then the block is classified as high frequency, and vice versa for flat areas.) from each of the plurality of blocks, and determine the surface area of the flat area on the basis of the number of extracted blocks. (Shen 5, cited above, teaches that blocks are determined to be flat if they are below a threshold, Fig. 5 cited above details visually how this process results in the surface area of the flat area is determined, and is made up of the flat blocks.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Lashdan with Shen where the screen is divided into a plurality of blocks from which the surface area of the flat area is determined based on the number of extracted blocks.
There are several reasons that would motivate on to do so, one of which is efficiency. Switching from block groups of pixels rather than individual pixels allows for more surface area to be checked at once, making the process of checking if an area is flat by extending horizontally/vertically quicker as there are fewer individual items to check.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALAN GREGORY HAKALA whose telephone number is (571)272-7863. The examiner can normally be reached 8:00am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571) 270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALAN GREGORY HAKALA/ Examiner, Art Unit 2617
/KING Y POON/ Supervisory Patent Examiner, Art Unit 2617
Prosecution Insights