Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-5, 10-13, 15, are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Lashdan (WO 2017205010 A1).
Regarding claim 1, 11,
Lashdan teaches:
A display device comprising: a controller configured to acquire a surface area of a flat area having few detail components on the basis of image data, (Lashdan ¶3 “In video processing, there may be instances in which contouring visual artifacts are observed in regions of a video image with very low texture. The contours formed in low texture regions by pixels with the same or similar level may be referred to as contouring artifacts, and more typically banding artifacts. In some instances, these banding artifacts may result from the quantization of regions or areas of a video image that have low gradients or ramps (e.g., gradients or ramps with small slopes). These regions or areas may be referred to as flat areas of a video image. For example, the quantization of low gradient areas to 8 bits may result in banding artifacts.” ¶106 “Returning to the first step of banding detection, as shown in Equation (1) below, a current filter size for a debanding filter (e.g., a low pass filter (LPF)) is applied to the original sample of a target pixel location (x) to produce a filtered sample (LPF(x)). | LPF(x) – x | < a (1)
If the difference between the filtered sample and the original sample (e.g., the difference between the filtered pixel value and the original pixel value) is less than a threshold (α), then the first step is said to have been passed or met by the current size of the debanding filter. Passing the first step may indicate that the difference due to filtering is small and, therefore, due to truncation. Accordingly, passing the first step may indicate that the area is flat and potentially has banding, and that the current size of the debanding filter is good for that area.” Note: Lashdan teaches that flat areas in an image are found via a process of applying filters in an effort to find where contour noise/artifacts, also referred to as banding.) acquire a gain value for removing contour noise on the basis of the surface area of the flat area, (Lashdan ¶107 “the second step of banding detection includes analyzing the area (e.g., groups of consecutive pixels in a row or column of a video image) being considered by the debanding filter to determine whether all non-zero gradients within a filter kernel have the same sign and are smaller than a threshold. A filter kernel may refer to a matrix or masking operation that is to be performed on pixels associated with a target pixel. In some instances, this threshold may be the same as the threshold ( ) used in the first step of banding detection (see e.g., Equation (1)), while in other instances it may be different. Additional aspects related to the second step or action of banding detection are provided below in more detail with respect to FIGs. 8A-10.” ¶114 “This scheme may involve both the banding detection (e.g., banding artifact detection) and the adaptive debanding filtering described above. In general, this scheme starts with a small filter size and iteratively increases the filter size until a largest, suitable filter size is identified for a particular pixel location (e.g., a target pixel location).”
PNG
media_image1.png
962
640
media_image1.png
Greyscale
Note: Lashdan compares the results of a filter against an original image and checks the difference against a threshold to determine if there is banding present in the area. If a filter passes the threshold, then we know the given filter size describes a flat area of that size, and we increase filter sizes until a filter fails the threshold check, in which case we default to the last largest filter. The size of the filter is the claims “gain value”, a value indicating how strong banding is for a given area. The application’s specification’s Fig. 9 provide an example of a gain value,
PNG
media_image2.png
306
524
media_image2.png
Greyscale
. As seen from the graph if the surface area of a flat area surpasses a certain threshold TH then the gain value is increased; the same concept found in Lashdan where if the surface area of a flat area surpasses a certain threshold the filter size is increased.) and generate result image data on the basis of the gain value;(¶154 “At 1512, the method 1500 may include performing a first banding artifact correction in a first direction on a target pixel location in the video data based on a first debanding filter. For example, the vertical banding detection/filtering 712 in FIG. 7A may perform a first banding artifact correction (e.g., video debanding) in the vertical direction. In another example, the horizontal banding detection/filtering 732 in FIG. 7B may perform a first banding artifact correction (e.g., video debanding) in the horizontal direction.” Note: After the largest possible target pixel areas containing contour noise/banding are found, the artifacts are removed to produce a corrected image.) and a display configured to output an image on the basis of the result image data. (Lashdan ¶96 “The video debanding component 710 may perform various aspects described herein for video debanding that uses adaptive filter sizes and gradient based banding detection (also referred to as gradient based banding artifact detection). For example, the video debanding component 710 may receive image data in the form of decoded video images, where the image data may have pixel values of a first bit depth. In an example, the bit depth may be 8 bits since this is a typical number of bits used to represent colors and/or intensity levels in a pixel for display and/or storage purposes.” ¶43 “A destination device as described above may include an input interface, a video decoder, and a display device. A display device as described herein may display video data to a user, and may comprise any of a variety of display devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.” Note: Lashdan teaches a display device that can display the corrected output image.)
Regarding claim 2, 12,
Lashdan teaches:
The display device according to claim 1, wherein, when the surface area of the flat area is less than a preset threshold value, the gain value is fixed. (Lashdan
PNG
media_image3.png
904
568
media_image3.png
Greyscale
Note: Fig. 11 depicts a workflow for the process described in Lashdan ¶106, ¶107, and ¶114, where we iteratively increase a filter size to check a larger and larger surface area. Lashdan ¶106 and ¶107 teach that a threshold is checked for each filter applied to determine if the new, larger area being checked contains banding. If it does not, as seen in 1130 to 1140 in Fig. 11, then we stop, making the gain value aka filter size now fixed and no longer increasing.)
Regarding claim 3,
Lashdan teaches:
The display device according to claim 1 display device according to wherein, when the surface area of the flat area is equal to or greater than a preset threshold value, the gain value is variable. (
PNG
media_image3.png
904
568
media_image3.png
Greyscale
Note: Referring again to Fig. 11, if the threshold is passed after applying a filter, then we do not stop and instead increase filter size. Unless we are at the maximum size then there are still remaining filters to check, which means the gain value aka filter size being checked, is still subject to change, aka variable.)
Regarding claim 4,
Lashdan teaches:
The display device according to claim 3, wherein the larger the surface area of the flat area, the higher the gain value. (Lashdan Fig. 11, ¶114, clearly teach the iterative process of checking increasingly larger surface areas to assign larger filter sizes, analogous to gain values, teaching that the larger the surface area the larger the filter size/gain value.)
Regarding claim 5, 13,
The display device according to claim 1, wherein the controller is configured to acquire motion information on the basis of the image data and acquire the gain value on the basis of the surface area of the flat area (Lashdan Fig. 11, ¶114, ¶106, ¶107, cited above teach acquiring gain value for surface areas of flat areas) and the motion information. (Lashdan ¶68 “The prediction processing unit 41 includes a motion estimation unit 42, a motion compensation unit 44, and an intra- prediction processing unit 46.” Note: Lashdan teaches that a motion estimation unit and compensation unit are present, teaching the presence of motion information for images evaluated in the video stream.)
Regarding claim 10, 15,
Lashdan teaches:
The display device according to claim 1, wherein the controller is configured to generate blur data by applying the gain value to the image data and generate the result image data by combining the image data with the blur data.( Lashdan ¶106 “Returning to the first step of banding detection, as shown in Equation (1) below, a current filter size for a debanding filter (e.g., a low pass filter (LPF)) is applied to the original sample of a target pixel location (x) to produce a filtered sample (LPF(x)). | LPF(x) – x | < a (1) If the difference between the filtered sample and the original sample (e.g., the difference between the filtered pixel value and the original pixel value) is less than a threshold (α), then the first step is said to have been passed or met by the current size of the debanding filter.” ¶155 “The first banding artifact correction may include performing banding artifact detection on the target pixel location, adapting, in response to the detection of a banding artifact, a filter size of the first debanding filter based on content in the video data, the filter size being adapted from a set of filter sizes, and applying, to a value of the target pixel location, the first debanding filter having the adapted filter size to produce a filtered value of the target pixel location.” ¶36 “A smoothing filter is then used in areas with banding artifacts to try to reconstruct the original information before it was truncated by quantization” Note: All of the filters of varying size that Lashdan leverages are low pass filters, known blur/smoothing filters for smoothing data. Lashdan clearly teaches that it is the applying of these debanding low pass filters that provide the noise correction resulting in the final output image.)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6, 7, are rejected under 35 U.S.C. 103 as being unpatentable over Lashdan (WO 2017205010 A1) in view of Viet (CA 2616875 A1).
Regarding claim 6,
Lashdan teaches:
The display device according to claim 5,
While Lashdan teaches compensating for motion compensation as part of its decoding process it does not state that the amount of motion can influence a gain value. This is taught in Viet which teaches wherein the gain value (Viet ¶20 “For noise correcting, the proposed apparatus and method are based on: i) minimization of the output noise variance for the temporal filter; ii) a shape adaptive local segmented window that considers only the similar intensity pixels to the current one for the local mean and local standard deviation estimations. For reliable window segmentation, a two-dimensional (2D) low pass filter is preferably required for the local adaptive windowing. The noise corrector further comprises a gain calculator in order to minimize the Mean Square Error (MMSE) for given local signal mean, local signal power and local additive noise power. The combination of local shape adaptive windowing and Minimum Mean Square Error constitutes a noise corrector working on all of the above-cited classified regions.” Note: Viet seeks to remove artifacts/noise from an image, while the noise is not contour noise as described in Lashdan and the application, it is a comparable process. Similarly to Lashdan, Viet identifies areas that need correction by applying a low pass filter) when an amount of motion depending on the motion information is large is less than the gain value when the amount of motion is small. (¶110 “Component-wise Image Differences are firstly calculated with three respective subtractions 431, 432 and 433 … The three resulted image differences are now squared up respectively by 434, 437 and 438. The squaring operator outputs are combined together with summation 439. The summation result 440 is provided to Low Pass Filter 441 which approximates the local signal mean value.” ¶111 “Low Pass Filter output 442 representing a variance signal s2lP is then provided to the Embedded Motion Soft Detection unit 460” ¶114 “receiving also temporal input noise variance σ2nT 104 yields the sum (s12 + σ2nT) … the calculated filter coefficient bmin for still parts of the picture.”
PNG
media_image4.png
734
800
media_image4.png
Greyscale
Note: As seen in ¶110 and ¶111, s2lP is the output of the lowpass filter and difference between a current frame and previous. This value is checked against the temporal noise variance, or in other words the noise variance that is caused over time from motion not from real artifacts/noise. If the difference, which indicates the presence of noise, is larger than what can be explained by motion variance we conclude 1, or true, meaning there is noise that needs removing. This binary value is used to obtain “no motion decision nm” which is a fractional from 0 to 1 acting as a weight. The gain value informs on how aggressively, or by how much, the noise should be removed. Thus, the indicator of noise bmin is multiplied by the weight in order to influence the gain value with the amount of motion, if the value is closer to 0 indicating there is motion present, then the value will be weakened.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Lashdan with Viet where the gain value is less when there is more motion, and more when there is less motion where the gain value defines how strongly contour noise is present in a given area in order to have it removed.
There are several reasons that would motivate one to do so, the quality of an original image could be damaged if contour noise that is not truly there is “corrected”, such a situation could occur if the system described conflates noise caused from motion with real contour noise, by having an inverse relationship between the gain value and motion present we can allow more confidence for still frames known to have no motion conflicting, and weaken the gain value when motion is present to avoid over correction.
Regarding claim 7,
Lashdan teaches:
The display device according to claim 5, maintain the acquired gain value when the surface area of the flat area is less than a preset threshold value, and increase in acquired gain value when the surface area of the flat area is equal to or greater than the threshold value. (Lashdan Fig. 11, ¶114, ¶106, cited previously, clearly teach this through the defined workflow where filters are applied to an image, a threshold to determine if that new higher level of noise is present is checked, and if passed a next larger filter is checked until the threshold check fails and we maintain the acquired filter size indicating strength of noise present, aka gain value.)
Lashdan does not however teach determining gain value on the basis of motion, this is taught by Viet wherein the controller is configured to acquire the gain value on the basis of the motion information,(Viet ¶110, ¶111, and the paragraphs contained in the screenshot, provided in claim 6, clearly teach that motion information is a factor in acquiring the final gain value and is rejected on the same basis.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Lashdan with Viet where a gain value that has acquired on the basis of motion can be increased by checking if the surface area of the given flat area is above a certain threshold, and stopping the increasing process and maintaining the current gain value if it fails the threshold.
There are several reasons that would motivate one to do so, once we have already acquired a gain value and adjusted it with respect to motion for a given area it may be the case that the level of noise is stronger than we know currently, this can be handled by increasing the gain value and checking a threshold to make sure we have not increased it too far, allowing for the noise to be removed more completely.
Claims 8, 9, 14, are rejected under 35 U.S.C. 103 as being unpatentable over Lashdan (WO 2017205010 A1) in view of Shen (Review of Postprocessing Techniques for Compression Artifact Removal).
Regarding claim 8, 14,
Lashdan teaches:
The display device according to claim 1, wherein the controller is configured acquire the surface area of the flat area (Lashdan ¶107 teaches finding the surface area of a flat area)
While Lashdan teaches finding the surface area of a flat area to it does not teach doing so on the basis of a high frequency component of a frequency domain converted from the image data. This is detailed in Shen which teaches to convert the image data into a frequency domain and acquire the surface area of the flat area on the basis of a high frequency component of the converted frequency domain. (Shen 3 “However, due to perceptual masking [27], the high frequency noise in the flat region is more visible to human eyes and thus requires special attention for perceived quality improvement.” Shen 5 “The coded bitstream is first dequantized. Each 8x8 block in an image is classified into two categories: flat block, and high activity block. There exist many classification rules. Let V, H, and D be three directional indices that correspond to different edge directions of the block in spatial domain … Let M be the maximum among three indices V, H, and D. If M is less than a threshold value, the block is classified as a flat block; otherwise, it is classified as a high activity block, which can be either a texture or an edge block. The threshold value T determines the sensitivity of classifier and it is selected from training images such that the probability of misclassification error is minimized. In our implementation, we chose T between 20 to 30.”
PNG
media_image5.png
480
442
media_image5.png
Greyscale
PNG
media_image6.png
722
872
media_image6.png
Greyscale
Note: Shen teaches a DCT, discrete cosine transform, is used on image data. DCT is a known common method of transforming or converting image data into a frequency domain. Shen 5 directly teaches that blocks in the frequency domain are determined to be flat areas or high frequency, non-flat areas, and that this is done because flat areas allow for noise to be more visually apparent. Aside from just identifying which areas are flat areas, the surface area of given flat areas is found as seen in Fig. 5 where flat areas are visualized as black blocks, showing the surface areas of flat areas are mapped for the entire image using a frequency domain. The flat areas are identified by being below a threshold the high frequency areas pass, and the ultimate surface areas of the flat areas is bounded by those high frequency areas, teaching that high frequency components are involved in determining the surface area of the flat areas. Fig. 1 is provided as a reference showing the original image before it has been converted into a frequency domain via DCT.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Lashdan with Shen where determining the surface area of flat areas is done using a high frequency data from a frequency domain converted from the image data.
There are several reasons that would motivate one to do so, determining whether or not there is noise present in a given area can be difficult since some areas contain more detail, making them higher frequency, than others, and thus the noise is harder to see and less important to correct. Identifying the areas that needs the most correction can be greatly simplified by using a frequency domain, as rather than having to evaluate the surface area one pixel at a time we can use the grouped blocks of pixels in the frequency domain to streamline the process.
Regarding claim 9,
Lashdan teaches:
The display device according to claim 8,
Lashdan does not teach that its image data is divided into blocks in order to determine which pixel grouped areas have a high frequency and which are flat. This is taught in Shen which teaches wherein the controller is configured to divide a screen into a plurality of blocks, extract blocks, in which the high frequency component is greater than a preset reference value, (Shen 5 “The coded bitstream is first dequantized. Each 8x8 block in an image is classified into two categories: flat block, and high activity block. There exist many classification rules. Let V, H, and D be three directional indices that correspond to different edge directions of the block in spatial domain … Let M be the maximum among three indices V, H, and D. If M is less than a threshold value, the block is classified as a flat block; otherwise, it is classified as a high activity block, which can be either a texture or an edge block. The threshold value T determines the sensitivity of classifier and it is selected from training images such that the probability of misclassification error is minimized. In our implementation, we chose T between 20 to 30.” Note: Shen teaches that once the image data has been transformed into a frequency domain and divided into a plurality of 8x8 blocks a threshold, or the claims “preset reference value” is checked to determine whether or not the block is high frequency or a flat area. If it is higher than the threshold then the block is classified as high frequency, and vice versa for flat areas.) from each of the plurality of blocks, and require the surface area of the flat area on the basis of the number of extracted blocks. (Shen 5, cited above, teaches that blocks are determined to be flat if they are below a threshold, Fig. 5 cited above details visually how this process results in the surface area of the flat area is determined, and is made up of the flat blocks.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Lashdan with Shen where the screen is divided into a plurality of blocks from which the surface area of the flat area is determined based on the number of extracted blocks.
There are several reasons that would motivate on to do so, one of which is efficiency. Switching from block groups of pixels rather than individual pixels allows for more surface area to be checked at once, making the process of checking if an area is flat by extending horizontally/vertically quicker as there are fewer individual items to check.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALAN GREGORY HAKALA whose telephone number is (571)272-7863. The examiner can normally be reached 8:00am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571) 270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALAN GREGORY HAKALA/Examiner, Art Unit 2617 /KING Y POON/Supervisory Patent Examiner, Art Unit 2617
Prosecution Insights