Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Rejections made under 35 U.S.C. 112(b) are withdrawn.
Applicant’s arguments with respect to claims 21-29 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 21-22 and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Knaus et al. (“DUAL-DOMAIN IMAGE DENOISING”, 2013 IEEE International Conference on Image Processing. IEEE, 2013.), (hereinafter, Knaus) in view of Shima (US 10,879,946 B1) and further in view of Yeh et al. (US 8696840 B2), (hereinafter Yeh).
Regarding claim 21, Knaus teaches a noise reduction circuit (Knaus, “Thus, a hybrid approach is taken by recent works. BM3D [3], shape-adaptive BM3D (SA-BM3D) [4], and BM3D with shape adaptive principal component analysis (BM3D-SAPCA) [5], sorted by increasing denoising quality, are considered state-of-the-art... We propose a method that is competitive in quality with BM3D, but is much simpler to implement.”, pg. 440, 1st column, 3rd and 4th paragraphs, “Thus, we implemented a C version using the FFTW library, which shortened the time to 40 seconds. Since the pixels are mutually independent, we achieved linear scalability using dual quad-core CPUs, reducing the time to 5 seconds. Finally, our GPU implementation on an NVIDIA GeForce GTX 470 cut the time down to one second.”, pg. 442, 1st column, 4th paragraph, lines 6-11, and 2nd column, lines 1-2, The system employs circuits (e.g., CPU and GPU) to perform the image denoising functions described therein. Accordingly, the recitation of circuit-based components is taught by Knaus.), comprising:
a kernel calculation circuit to: generate a bilateral kernel for at least one pixel of an image (Knaus, “In the first step, we calculate the denoised high-contrast value for
s
~
p
a pixel
p
using a joint bilateral filter [13]. Our joint bilateral filter uses the guide image to filter the noisy image
y
. We define the bilateral kernel over a square neighborhood window
N
p
centered around every pixel with window radius
r
.”, pg. 440, 2nd column, Section 2.1, lines 1-5, A bilateral kernel is applied to the noisy image to generate denoised a high-contrast image.);
a noise filtering circuit coupled to the kernel calculation circuit, the noise filtering circuit to:
perform noise filtering of the image to generate a first de-noised version of the image, and perform noise filtering of the image using the bilateral kernel to generate a second de-noised version of the image (Knaus, “We observe that spatial domain methods excel at denoising high-contrast images while transform domain methods excel at low-contrast images. We therefore separate the image into two layers, and denoise them separately… The high-contrast layer is the bilaterally filtered image, and the low-contrast layer is the residual image.”, pg. 440, 2nd column, lines 3-10, see sections 2.1 Spatial Domain: Bilateral filter, 2.2 Domain Transform, 2.3 Frequency Domain: Wavelet Shrinkage, and eqs. 3 and 8, A high-contrast image is generated via bilateral filtering, while a low-contrast image is generated through a short-term Fourier transform (SFTF) and wavelet shrinkage processes. These denoised images are defined as
s
~
and
S
~
, respectively.) and
a blending circuit coupled to the kernel calculation circuit and the noise filtering circuit, the blending circuit configured to blend each color component of the first de-noised version of the image with a corresponding color component of the second de-noised version of the image to generate a de-noised multi-color version of the image (Knaus, “The goal is to estimate the original image
x
from a noise-contaminated image
y
=
x
+
η
… The original image can thus be approximated by the sum of the two denoised layers as
x
~
=
s
~
+
S
~
, (1) where
s
~
and
S
~
are the denoised high- and low-contrast images.”, pg. 1, 2nd column, lines 13-16, “For color images, we make two modifications to the algorithm… Secondly, in the range kernel of the bilateral filter in Equation 4, we calculate the normalized Euclidean distance
(
g
p
-
g
q
)
2
σ
2
as the sum of the normalized distances of all channels.”, pg. 441, 1st column and 2nd column, Section 2.4 Color images, The final denoised image
x
~
is reconstructed by summing the separately denoised high- and low-contrast components
s
~
+
S
~
. This reconstruction process is extended to color images by applying the bilateral and transform domain filtering on each channel independently. Accordingly, the summation operation is performed per color component to generate the final denoised image.).
Knaus does not teach generate a machine learning (ML) kernel for at least one pixel of an image and performing noise filtering of the image using the ML kernel.
However, Shima teaches generating a machine learning (ML) kernel for at least one pixel of an image and performing noise filtering of the image using the ML kernel (Shima, “In this embodiment, the magnitude of the STFT 612 is input as a spectrogram 614 to a neural network 616. The neural network 616 can include a convolutional neural network (CNN) 620. The CNN can be implemented by the execution of instructions 328 by the processor 312. The output is a de-noising or output signal mask 624… The output signal mask 624 is overlaid onto the original 2D spectrogram 614, to provide an output spectrogram 630 in which the desired signals 124 present in the original spectrogram 614 are excised 628. An inverse transform 632, for example an inverse STFT where a Fourier transform was initially used to create the input datagram 614, is then performed on the excised signal 628, resulting in a de-noised time series output 636.”, columns 7 and 8, lines 53-67 and 1-6, respectively, “A signal detection network in accordance with embodiments of the present disclosure can therefore include a CNN 1020 to produce de-noised real and imaginary datagrams 1024, including but not limited to spectrograms, that can be used to produce a de-noised, time series output 1036 of a desired signal 124, instead of simply a binary mask.”, column 8, lines 37-42, The result of a STFT process is input to a CNN for denoised image generation. This process includes a machine learning kernel that learns and applies filtering.).
Knaus teaches performing STFT followed by wavelet shrinkage to generate a denoised low-contrast image in the transform domain (Knaus, see section. 2.3. Frequency Domain: Wavelet Shrinkage and eq. 8). Shima teaches using the real and imaginary components of an STFT-transformed signal as input to a CNN trained to generate a denoised output (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the transform domain denoising of Knaus by replacing the wavelet shrinkage with the CNN-based denoising method as taught by Shima, thereby producing separate images corresponding to the bilateral filtering and the CNN-based filtering, for image summation in the final denoised image reconstruct. The motivation for doing so would have been to apply conventional machine learning techniques to improve the sensitivity and accuracy of noisy signal detection over traditional energy detection methods (as suggested by Shima, “Embodiments of the present disclosure can provide an advance over conventional energy detection systems, which may miss weak signals. Moreover, whereas conventional energy detection systems may incorrectly indicate the presence of a signal 124 in response to the presence of interfering signals 128 in a noisy environment, embodiments of the present disclosure can more accurately indicate the presence of a desired signal 124.”, column 9, lines 48-55). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Knaus with Shima to obtain the invention as specified above.
Knaus in view of Shima does not teach generate alpha parameters for corresponding color components comprising luma and chroma components of the at least one pixel and blend each color component of the first de-noised version of the image with a corresponding color component of the second de-noised version of the image using the alpha parameters for each of the corresponding color components generated by the kernel calculation circuit.
However, Yeh teaches generate alpha parameters for corresponding color components comprising luma and chroma components of the at least one pixel and blend each color component of the first de-noised version of the image with a corresponding color component of the second de-noised version of the image using the alpha parameters for each of the corresponding color components generated by the kernel calculation circuit (Yeh, “In certain embodiments, a plurality of input graphics
images are iteratively blended in real time to provide a blended graphics image, which is then composited with other layers such as an input video image… Compositing may include an alpha blending based on alpha values for pixels of the images.”, column 3, lines 34-44, “At
block 743, the current pixel of the second image is blended with the current pixel of the first image to produce the output pixel according to the formula shown below: (see eq. ) where Lo and Co are the luma and chroma components of the output pixel, respectively; LS and Cs are the luma and chroma components of the current pixel of the second image, respectively: As is the alpha channel associated with the current pixel of the second image; Ld and Cd are the luma and chroma components of the first image, respectively.”, column 17 and 18, lines 67 and 1-13, respectively, Alpha blending is performed to combine two images. This includes blending luma and chroma components extracted from each image using respective alpha parameters.).
Knaus in view of Shima teaches image denoising using bilateral and machine learning kernels, which includes blending denoised images to produce a final denoised image (Knaus, pg. 1, 2nd column, lines 13-16, pg. 441, 1st column and 2nd column, Section 2.4 Color images). Knaus in view of Shima does not teach generating an alpha parameter for images or using the alpha parameters for image blending. Yeh teaches alpha blending of two images using luma and chroma pixel components with respective alpha parameters (see above). Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the image blending of Knaus in view of Shima to be based on alpha blending as taught by Yeh (Yeh, column 17 and 18, lines 67 and 1-13, see Fig. 7). The motivation for doing so would have been to enable independent control of luma and chroma information during image blending, thereby improving image quality. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Knaus in view of Shima with Yeh to obtain the invention in claim 21.
Regarding claim 22, Knaus in view of Shima and further in view of Yeh teaches the noise reduction circuit of claim 21, wherein the alpha parameters for the corresponding color components of the at least one pixel represent blending weights for the corresponding color components of the at least one pixel (Yeh, “At block 743, the current pixel of the second image is blended with the current pixel of the first image to produce the output pixel according to the formula shown below: (see eq.)”, column 18, lines 24-29, The alpha parameters define weights which controls the contribution of luma and chroma color components for corresponding pixels when blending images.).
Regarding claim 28, Knaus in view of Shima and further in view of Yeh teaches the noise reduction circuit of claim 21, wherein the kernel calculation circuit is further configured to generate the ML kernel by processing a corresponding patch of the image using at least one ML filter bank (Shima, “The received energy 404, as represented by the time-varying voltage 406, is transformed in a transform block 408 from a time series signal to a two-dimensional datagram 410, such as but not limited to a time-frequency gram. As an example, the transform block 408 may be implemented by the execution of instructions 328 by the processor 312, and may perform a fast Fourier transform (FFT), a short-time Fourier transform (STFT), a discrete Fourier transform (DFT), a Haar transform, a wavelet transform, a Hadamard transform, a discrete cosine transform, a Gabor transform, a Karhunen-Loeve transform, or the like of the time varying voltage 406 to create the two-dimensional data 410… In accordance with embodiments of the present disclosure, the real values are input to an input layer 412 of a neural network 416 as a first channel of information, and the imaginary values are input to the input layer 412 as a second channel of information. For instance, the real data, representing the magnitude of the input RF energy 404, is provided as a first two-dimensional data gram or spectrogram 414a to the first channel of the input layer 412, and the imaginary data, representing the phase of the input RF energy 404, is provided as a second two-dimensional data gram or spectrogram 414b to the second channel of the input layer 412. In addition, the first 414a and second 414b datagrams have the same dimensions as one another, and are provided as stacked inputs.”, column 6, 4-36, Transforms, such as STFT, are applied to the input image to decompose it into time-localized frequency sub bands. These sub bands are arranged in a spectrogram, representing the signal frequency over time. This process acts as a filter bank, where each frequency bin represents the output of a different bandpass filter. The resulting spectrogram serves as input to a CNN, where learned kernels are then able perform filtering over the time-frequency components.).
Claims 29 are rejected under 35 U.S.C. 103 as being unpatentable over Knaus et al. (“DUAL-DOMAIN IMAGE DENOISING”, 2013 IEEE International Conference on Image Processing. IEEE, 2013.), (hereinafter, Knaus) in view of Shima (US 10,879,946 B1) and further in view of Yeh et al. (US 8698840 B2) and Branco (WO 2022106554 A2).
Regarding claim 29, Knaus in view of Shima and further in view of Yeh teaches the noise reduction circuit of claim 21. Knaus in view of Shima and further in view of Yeh does not teach further comprising a combining circuit coupled to the blending circuit, the combining circuit to combine the image with the de-noised multi-color version of the image to generate a final de-noised multi-color version of the image.
However, Branco teaches further comprising a combining circuit coupled to the blending circuit, the combining circuit to combine the image with the de-noised multi-color version of the image to generate a final de-noised multi-color version of the image (Branco, “Further, the de-noised image data is combined with the image data, to which the de-noising process was applied (i.e., the initial image data or the initial image data after thresholding), in order receive combined image data. Two images or image data for two images are combined, for example, by multiplying them together (element by element). This results in brighter areas becoming brighter and darker areas becoming darker. The multiplication of the two images can be performed one time such combination step. Then, a smoothing filter is applied to the combined image data in order to receive smoothed image data.”, pg. 4, lines 24-29, “The processing system 132 may comprise any circuit or combination of circuits.”, pg. 16, lines 6-7, An initial image and a corresponding denoised image are combined via multiplication to produce an intermediate image, which is further processed to yield a final denoised image.).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified the noise reduction circuit of Knaus in view of Shima and further in view of Yeh to include circuitry which further processes the denoised color images by multiplying them with a corresponding initial image (as taught by Branco, pg. 4, lines 24-29). The motivation for doing so would have been to enhance contrast and preserve image details by combining the clarity of the denoised image with the sharp features of the original image (as suggested by Branco, “The technical features described aim and help to produce an OCT image that both reduces the speckle noise and improves image contrast of the morphological structures. Combining the respective images or image data sets allows the structures to be brighter and enhanced while also reducing the background speckle noise.”, pg. 6, lines 20-23). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the teachings of Knaus in view of Shima and further in view of Yeh with Branco to obtain the invention as specified in claim 29.
Allowable Subject Matter
Claims 1, 3-11, and 19 are allowed.
The following is an examiner’s statement of reason for allowance:
With respect to claims 1 and 19 (and dependent claims 3-11), in addition to other limitations in the claims the Prior art of Record fails to teach, disclose or render obvious the applicant’s invention as claimed, in particular the “A noise reduction circuit, comprising: a kernel calculation circuit configured to: generate a machine learning (ML) kernel for at least one pixel of an image and a bilateral kernel for the at least one pixel of the image; classify a corresponding patch of the image using a plurality of analysis kernels to obtain a plurality of classification weights for the at least one pixel of the image; calculate absolute values of the plurality of classification weights to generate a vector of absolute weights for the at least one pixel of the image; normalize the vector of absolute weights by a patch standard deviation for the at least one pixel to obtain a vector of normalized weights for the at least one pixel of the image; calculate a quality factor for the at least one pixel of the image using the vector of normalized weights; and compute, for the at least one pixel of the image, an alpha parameter for each color component of the image by inputting the quality factor into a look-up table (LUT) circuit for that color component; a noise filtering circuit coupled to the kernel calculation circuit, the noise filtering circuit configured to: perform noise filtering of the image using the ML kernel to generate a first de-noised version of the image, and perform noise filtering of the image using the bilateral kernel to generate a second de-noised version of the image; and a blending circuit coupled to the kernel calculation circuit and the noise filtering circuit, the blending circuit configured to blend each color component of the first de-noised version of the image with a corresponding color component of the second de-noised version of the image using the alpha parameter to generate a de-noised multi-color version of the image.” as recited in claims 1 and 19. Note that the bolded limitations above render the claims allowable.
Knaus teaches image denoising using bilateral filtering and transform domain processing, including the summation of denoised layers to produce a final denoised image. Shima teaches applying a machine learning kernel for transform domain image denoising. Yeh teaches performing alpha blending between luma and chroma image components. Branco teaches multiplying input images with denoised images. However, none of the references disclose the bolded limitations above.
Claims 23-27 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CONNOR LEVI HANSEN whose telephone number is (703)756-5533. The examiner can normally be reached Monday-Friday 9:00-5:00 (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CONNOR L HANSEN/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672