Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Disposition of Claims
In response to applicant’s amendment received on 7/27/2026, all requested changes to the claims and specification have been entered.
Claim(s) 1-20 were previously pending.
No Claim(s) have been added.
No Claim(s) have been cancelled.
Claim(s) 1-20 are currently pending.
Response to Amendment
Objections to the Specification
Applicant has amended ¶0085 to correct which side of Fig. 7 is treated as input and output. The objections to the specification have been withdrawn.
Rejections under 35 U.S.C. § 112(b)
Applicant has amended claims 6, 13 & 19 similarly recite “…loss between the denoised images and corresponding color channels for the ground truth”. The examiner believes that this amendment, in tandem with the amendments to claim 1, clarify what the corresponding color channels refer to. The rejections under 35 U.S.C. § 112(b) have been withdrawn.
Rejections under 35 U.S.C. § 103
Applicant's arguments filed 07/27/2026 have been fully considered but are not persuasive. Applicant has amended claims 1, 8 & 14 to now specify the color channel pixel groupings of typical non-Bayer CFA patterns such as Tetra (Quad-Bayer), Hexa-Deca, and Nona. While the examiner agrees with the applicant’s general assessment that neither Pan, nor Egorov teach the particular pixel groupings reflected in the amended claims, the examiner disagrees with applicant’s assessment of what Zeng teach.
Zeng verifiably demonstrate denoising images with Tetra CFA patterns following the claimed “each of the first color pixel groups has two or more adjacent pixels for one of the plurality of color channels”. The amended claims merely specify the denoising performed by the claims to non-Bayer image types with a particular color channel pixel grouping, which Zeng wholly teaches. Updated rejections under 35 U.S.C. § 103 are made below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4, 7-9, 11, 14-15 17 & 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pan et al (US 2021/0241429 A1), in view of Zeng et al (“Inheriting Bayer’s Legacy: Joint Remosaicing and Denoising for Quad Bayer Image Sensor, arXiv, 2023), hereinafter referred to as “Zeng”, further in view of Egorov et al (US 2021/0390658 A1), hereinafter referred to as “Egorov.
Regarding claim 1, Pan teach A method (the method used to train the network 500 [¶0036,41; Figs. 5 & 6]) comprising:
obtaining a noisy training image comprising a plurality of instances of a color filter array (CFA) pattern (Pan: a noisy multispectral filter arrays (MSFA) raw image is obtained in step 605 [¶0041; Fig. 6], the disclosed process can also be applicable to denoising CFA images as well [¶0034, 51-52]),
generating, using an artificial intelligence/machine learning (AI/ML)-based denoiser, a plurality of denoised images (Pan: a denoised image per channel C is produced via a deep learning model [0039-41; 44-48; Figs. 5 & 6]),
obtaining a ground truth image comprising the CFA pattern (Pan: a noise-free MSFA raw image (z) is obtained as the ground truth [¶0038-39]);
generating a noisy ground truth image from the ground truth image, (Pan: a noise-free (ground-truth) raw image can be augmented to have noise added [¶0038]). Pan however, fails to describe the CFA pattern comprising spatially-separated color channel pixel groups having two or more adjacent pixels of one color channel, indicative of a Quad-Bayer or otherwise Tetra CFA pattern, nor do they describe determining a loss based on the pixel groupings, and adapting the weights of the AI/ML-based denoiser based on this loss.
Zeng, on the other hand, is analogous art pertinent to the field of endeavor of the present application and disclose a method for denoising Quad Bayer images. More particularly, Zeng teach wherein the CFA pattern comprises multiple first spatially-separated color channel pixel groups for each of a plurality of color channels, and wherein each of the first color channel pixel groups has two or more adjacent pixels for one of the plurality of color channels (Zeng: Quad Bayer CFA patterns are denoised, wherein the Quad Bayer pattern comprises individual 2 x 2 cells of adjacent pixels of the same color (i.e. a first color channel pixel group) [Sec 3. Problem Formulation - ¶01-02; Fig. 2]); wherein each denoised image is based on the first color channel pixel groups of a respective color channel of the plurality of color channels (Zeng: each remosaiced and denoised image is based on the Quad Bayer CFA pattern of 2 x 2 pixel groups of matching colors [Sec 4.1 Overall Pipeline - ¶01-02; Fig. 3]); wherein the noisy ground truth image comprises multiple second spatially-separated color channel pixel groups for each of the plurality of color channels (Zeng: a ground truth (I) is also provided as a Quad Bayer pattern [Sec 4.1 – Overall Pipeline - ¶01-02; Eq. 3, Figs. 5 & 8]);
determining a loss based on a comparison between the denoised images and corresponding second color channel pixel groups of the noisy ground truth image; (Zeng: a loss is calculated between the remosaiced and denoised Quad Bayer image
I
^
and the ground truth I [Sec 4.1 – Overall Pipeline - ¶01-02; Eq. 3]).
Zeng states that their dual-head joint remosaicing and denoising network enables the conversion of Quad Bayer patterns without any loss is image resolution [Sec – Abstract]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to incorporate aspects of Zeng’s network specific to Quad Bayer denoising to the general denoising system disclosed by Pan that augments ground truth images with noise to arrive at the invention of the instant application.
While Zeng discloses adjusting weights of their denoiser, they fail to explicitly recite that these adjusted weights are based on the calculated loss. Egorov, per contra, is analogous art pertinent to the field of endeavor and disclose an image processing algorithm leveraging an neural network with adaptive weighting based on a loss function. More particularly, Egorov teach and
adapting weights of the AI/ML-based denoiser based on the loss (Egorov: adjusting filter weights of the neural network 103 to minimize a loss function across an enhanced (denoised) image 0002 and a high quality bayer image 0003 [0027-28; 62-63; Fig. 2]).
Egorov further teach that their neural network is directed to improve overall visual quality of CFM images prior to any downstream lossy image processing [¶0052]. Therefore, it would have been obvious to one of ordinary skill to combine the adaptive loss function weighting of Egorov to improve the loss optimization of Pan in view of Zeng for enhanced image quality, to arrive at the invention of the instant application.
As for claim 2, Pan in view Zeng, further in view of Egorov The method of claim 1 (as described previously), wherein:
the CFA pattern is one of: a Tetra CFA pattern (the input noisy image and corresponding ground truth are from a Quad Bayer mosaic – the examiner notes that Quad Bayer is identical to Tetra CFA patterns [Sec 4.3 Dual-head Joint Remosaic and Denoise - ¶03-05 & Sec 4.4 Bottleneck Data Mining - ¶01; Fig. 5]), a Hexa-Deca CFA pattern, or a Nona CFA pattern (Zeng: the examiner notes that given the use of the disjunctive “or”, only one of the listed limitations needs to be mapped to).
Zeng provides that their dual-head joint remosaicing and denoising network enables the conversion of Quad Bayer patterns without any loss is image resolution [Sec – Abstract]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to incorporate aspects of Zeng’s network specific to Quad Bayer to the denoising system disclosed by Pan in view Zeng, further in view of Egorov to arrive at the invention of the instant application.
With respect to claim 4, Pan in view of Zeng, further in view of Egorov teach The method of claim 1 (as described above), wherein generating the noisy ground truth image comprises adding zero-mean Gaussian noise to the ground truth image (Egorov: performing image degradation by adding gaussian noise to regular (ground-truth) images to generate noisy images [¶0060]).
Egorov further teach that their neural network is directed to improve overall visual quality of CFM images prior to any downstream lossy image processing [¶0052]. Therefore, it would have been obvious to one of ordinary skill to utilize the image degradation technique of adding Gaussian noise to the ground truth described by Egorov to improve the loss optimization of Pan in view of Zeng for enhanced image quality to arrive at the invention of the instant application.
As for claim 7, Pan in view Zeng, further in view of Egorov teach The method of claim 1 (as described previously), wherein the noisy training image comprises one of a plurality of noisy training images, each noisy training image having a different proportion of noise (Pan: the average PSNR (peak signal-to-noise ratio) for each set of experiments are calculated, with the PSNR from the noisy raw images included for reference, indicating a non-uniform degree of noise across the raw noisy images [¶0048; Table 1]).
Regarding claim 8, Pan teach An electronic device (computing system 700 [¶0056; Fig. 7]) comprising:
at least one processing device configured to train an artificial intelligence/machine learning (AI/ML)-based denoiser (CPU(s) 701 of the computing system that execute the training of the deep residual network [¶0056; Fig. 7]);
wherein, to train the AI/ML-based denoiser, the at least one processing device is configured to:
obtain a noisy training image comprising a plurality of instances of a color filter array (CFA) pattern (Pan: a noisy multispectral filter arrays (MSFA) raw image is obtained in step 605 [¶0041; Fig. 6], the disclosed process can also be applicable to denoising CFA images as well [¶0034, 51-52]),
generate, using an artificial intelligence/machine learning (AI/ML)-based denoiser, a plurality of denoised images (Pan: a denoised image per channel C is produced via a deep learning model [0039-41; 44-48; Figs. 5 & 6]),
obtain a ground truth image comprising the CFA pattern (Pan: a noise-free MSFA raw image (z) is obtained as the ground truth [¶0038-39]);
generate a noisy ground truth image from the ground truth image, (Pan: a noise-free (ground-truth) raw image can be augmented to have noise added [¶0038]). Pan however, fails to describe the CFA pattern comprising spatially-separated color channel pixel groups having two or more adjacent pixels of one color channel, indicative of a Quad-Bayer or otherwise Tetra CFA pattern, nor do they describe determining a loss based on the pixel groupings, and adapting the weights of the AI/ML-based denoiser based on this loss.
Zeng, on the other hand, is analogous art pertinent to the field of endeavor of the present application and disclose a method for denoising Quad Bayer images. More particularly, Zeng teach wherein the CFA pattern comprises multiple first spatially-separated color channel pixel groups for each of a plurality of color channels, and wherein each of the first color channel pixel groups has two or more adjacent pixels for one of the plurality of color channels (Zeng: Quad Bayer CFA patterns are denoised, wherein the Quad Bayer pattern comprises individual 2 x 2 cells of adjacent pixels of the same color (i.e. a first color channel pixel group) [Sec 3. Problem Formulation - ¶01-02; Fig. 2]); wherein each denoised image is based on the first color channel pixel groups of a respective color channel of the plurality of color channels (Zeng: each remosaiced and denoised image is based on the Quad Bayer CFA pattern of 2 x 2 pixel groups of matching colors [Sec 4.1 Overall Pipeline - ¶01-02; Fig. 3]); wherein the noisy ground truth image comprises multiple second spatially-separated color channel pixel groups for each of the plurality of color channels (Zeng: a ground truth (I) is also provided as a Quad Bayer pattern [Sec 4.1 – Overall Pipeline - ¶01-02; Eq. 3, Figs. 5 & 8]);
determine a loss based on a comparison between the denoised images and corresponding second color channel pixel groups of the noisy ground truth image; (Zeng: a loss is calculated between the remosaiced and denoised Quad Bayer image
I
^
and the ground truth I [Sec 4.1 – Overall Pipeline - ¶01-02; Eq. 3]).
Zeng states that their dual-head joint remosaicing and denoising network enables the conversion of Quad Bayer patterns without any loss is image resolution [Sec – Abstract]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to incorporate aspects of Zeng’s network specific to Quad Bayer denoising to the general denoising system disclosed by Pan that augments ground truth images with noise to arrive at the invention of the instant application.
While Zeng discloses adjusting weights of their denoiser, they fail to explicitly recite that these adjusted weights are based on the calculated loss. Egorov, per contra, is analogous art pertinent to the field of endeavor and disclose an image processing algorithm leveraging an neural network with adaptive weighting based on a loss function. More particularly, Egorov teach and
adapt weights of the AI/ML-based denoiser based on the loss (Egorov: adjusting filter weights of the neural network 103 to minimize a loss function across an enhanced (denoised) image 0002 and a high quality bayer image 0003 [0027-28; 62-63; Fig. 2]).
Egorov further teach that their neural network is directed to improve overall visual quality of CFM images prior to any downstream lossy image processing [¶0052]. Therefore, it would have been obvious to one of ordinary skill to combine the adaptive loss function weighting of Egorov to improve the loss optimization of Pan in view of Zeng for enhanced image quality, to arrive at the invention of the instant application.
As for claim 9, Pan in view Zeng, further in view of Egorov The electronic device of claim 8 (as described previously), wherein:
the CFA pattern is one of: a Tetra CFA pattern (the input noisy image and corresponding ground truth are from a Quad Bayer mosaic – the examiner notes that Quad Bayer is identical to Tetra CFA patterns [Sec 4.3 Dual-head Joint Remosaic and Denoise - ¶03-05 & Sec 4.4 Bottleneck Data Mining - ¶01; Fig. 5]), a Hexa-Deca CFA pattern, or a Nona CFA pattern (Zeng: the examiner notes that given the use of the disjunctive “or”, only one of the listed limitations needs to be mapped to).
Zeng provides that their dual-head joint remosaicing and denoising network enables the conversion of Quad Bayer patterns without any loss is image resolution [Sec – Abstract]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to incorporate aspects of Zeng’s network specific to Quad Bayer to the denoising system disclosed by Pan in view Zeng, further in view of Egorov to arrive at the invention of the instant application.
With respect to claim 11, Pan in view of Zeng, further in view of Egorov teach The electronic device of claim 8 (as described above), wherein generating the noisy ground truth image comprises adding zero-mean Gaussian noise to the ground truth image (Egorov: performing image degradation by adding gaussian noise to regular (ground-truth) images to generate noisy images [¶0060]).
Egorov further teach that their neural network is directed to improve overall visual quality of CFM images prior to any downstream lossy image processing [¶0052]. Therefore, it would have been obvious to one of ordinary skill to utilize the image degradation technique of adding Gaussian noise to the ground truth described by Egorov to improve the loss optimization of Pan in view of Zeng for enhanced image quality to arrive at the invention of the instant application.
With respect to claim 14, Pan teach A method (process 600 [¶0041; Fig. 6]) comprising:
obtaining a noisy captured image (a noisy multispectral filter arrays (MSFA) raw image is obtained in step 605 [¶0041; Fig. 6], the disclosed process can also be applicable to denoising CFA images as well [¶0034, 51-52]); and
denoising the noisy captured image using an artificial intelligence/machine learning (AI/ML)-based denoiser (a denoised image per channel C is produced via a deep learning model [0039-41; 44-48; Figs. 5 & 6]), wherein the AI/ML-based denoiser is trained by:
obtaining a noisy training image comprising a plurality of instances of a color filter array (CFA) pattern (Pan: a noisy multispectral filter arrays (MSFA) raw image is obtained in step 605 [¶0041; Fig. 6], the disclosed process can also be applicable to denoising CFA images as well [¶0034, 51-52]),
generating, using an artificial intelligence/machine learning (AI/ML)-based denoiser, a plurality of denoised images (Pan: a denoised image per channel C is produced via a deep learning model [0039-41; 44-48; Figs. 5 & 6]),
obtaining a ground truth image comprising the CFA pattern (Pan: a noise-free MSFA raw image (z) is obtained as the ground truth [¶0038-39]);
generating a noisy ground truth image from the ground truth image, (Pan: a noise-free (ground-truth) raw image can be augmented to have noise added [¶0038]). Pan however, fails to describe the CFA pattern comprising spatially-separated color channel pixel groups having two or more adjacent pixels of one color channel, indicative of a Quad-Bayer or otherwise Tetra CFA pattern, nor do they describe determining a loss based on the pixel groupings, and adapting the weights of the AI/ML-based denoiser based on this loss.
Zeng, on the other hand, teach wherein the CFA pattern comprises multiple first spatially-separated color channel pixel groups for each of a plurality of color channels, and wherein each of the first color channel pixel groups has two or more adjacent pixels for one of the plurality of color channels (Zeng: Quad Bayer CFA patterns are denoised, wherein the Quad Bayer pattern comprises individual 2 x 2 cells of adjacent pixels of the same color (i.e. a first color channel pixel group) [Sec 3. Problem Formulation - ¶01-02; Fig. 2]); wherein each denoised image is based on the first color channel pixel groups of a respective color channel of the plurality of color channels (Zeng: each remosaiced and denoised image is based on the Quad Bayer CFA pattern of 2 x 2 pixel groups of matching colors [Sec 4.1 Overall Pipeline - ¶01-02; Fig. 3]); wherein the noisy ground truth image comprises multiple second spatially-separated color channel pixel groups for each of the plurality of color channels (Zeng: a ground truth (I) is also provided as a Quad Bayer pattern [Sec 4.1 – Overall Pipeline - ¶01-02; Eq. 3, Figs. 5 & 8]);
determining a loss based on a comparison between the denoised images and corresponding second color channel pixel groups of the noisy ground truth image; (Zeng: a loss is calculated between the remosaiced and denoised Quad Bayer image
I
^
and the ground truth I [Sec 4.1 – Overall Pipeline - ¶01-02; Eq. 3]).
Zeng states that their dual-head joint remosaicing and denoising network enables the conversion of Quad Bayer patterns without any loss is image resolution [Sec – Abstract]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to incorporate aspects of Zeng’s network specific to Quad Bayer denoising to the general denoising system disclosed by Pan that augments ground truth images with noise to arrive at the invention of the instant application.
While Zeng discloses adjusting weights of their denoiser, they fail to explicitly recite that these adjusted weights are based on the calculated loss. Egorov, per contra, teach and
adapting weights of the AI/ML-based denoiser based on the loss (Egorov: adjusting filter weights of the neural network 103 to minimize a loss function across an enhanced (denoised) image 0002 and a high quality Bayer image 0003 [0027-28; 62-63; Fig. 2]).
Egorov further teach that their neural network is directed to improve overall visual quality of CFM images prior to any downstream lossy image processing [¶0052]. Therefore, it would have been obvious to one of ordinary skill to combine the adaptive loss function weighting of Egorov to improve the loss optimization of Pan in view of Zeng for enhanced image quality, to arrive at the invention of the instant application.
As for claim 15, Pan in view Zeng, further in view of Egorov The method of claim 14 (as described previously), wherein:
the CFA pattern is one of: a Tetra CFA pattern (the input noisy image and corresponding ground truth are from a Quad Bayer mosaic – the examiner notes that Quad Bayer is identical to Tetra CFA patterns [Sec 4.3 Dual-head Joint Remosaic and Denoise - ¶03-05 & Sec 4.4 Bottleneck Data Mining - ¶01; Fig. 5]), a Hexa-Deca CFA pattern, or a Nona CFA pattern (Zeng: the examiner notes that given the use of the disjunctive “or”, only one of the listed limitations needs to be mapped to).
Zeng provides that their dual-head joint remosaicing and denoising network enables the conversion of Quad Bayer patterns without any loss is image resolution [Sec – Abstract]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to incorporate aspects of Zeng’s network specific to Quad Bayer to the denoising system disclosed by Pan in view Zeng, further in view of Egorov to arrive at the invention of the instant application.
With respect to claim 17, Pan in view of Zeng, further in view of Egorov teach The method of claim 14 (as described above), wherein generating the noisy ground truth image comprises adding zero-mean Gaussian noise to the ground truth image (Egorov: performing image degradation by adding gaussian noise to regular (ground-truth) images to generate noisy images [¶0060]).
Egorov further teach that their neural network is directed to improve overall visual quality of CFM images prior to any downstream lossy image processing [¶0052]. Therefore, it would have been obvious to one of ordinary skill to utilize the image degradation technique of adding Gaussian noise to the ground truth described by Egorov to improve the loss optimization of Pan in view of Zeng for enhanced image quality to arrive at the invention of the instant application.
As for claim 20, Pan in view Zeng, further in view of Egorov teach The method of claim 14 (as described previously), wherein the noisy training image comprises one of a plurality of noisy training images, each noisy training image having a different proportion of noise (Pan: the average PSNR (peak signal-to-noise ratio) for each set of experiments are calculated, with the PSNR from the noisy raw images included for reference, indicating a non-uniform degree of noise across the raw noisy images [¶0048; Table 1]).
Claim(s) 3, 10 & 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pan et al (US 2021/0241429 A1), in view of Zeng et al (“Inheriting Bayer’s Legacy: Joint Remosaicing and Denoising for Quad Bayer Image Sensor, arXiv, 2023), hereinafter referred to as “Zeng”, further in view of Egorov et al (US 2021/0390658 A1), hereinafter referred to as “Egorov, further in view of Pan et al (“DeSmoke-LAP: improved unpaired image-to-image translation for desmoking in laproscopic surgery”, 2022, International Journal of Computer Assisted Radiology and Surgery), hereinafter referred to as “DeSmoke”.
Considering claim 3, Pan in view Zeng, further in view of Egorov teach The method of claim 1 (as described previously), wherein determining the loss comprises using a linear combination of a mean absolute error (L1) loss, a multi-scale structural similarity loss, (Egorov: the loss function optimizer 204 can train CNN 103 using a combination of different loss functions like an L1 loss function (mean absolute error), an MS-SSIM (multi-scale structural similarity loss) [¶0063; Fig. 2]).
Egorov further teach that their neural network is directed to improve overall visual quality of CFM images prior to any downstream lossy image processing [¶0052]. Therefore, it would have been obvious to one of ordinary skill to combine the adaptive loss function weighting of Egorov to improve the loss optimization of Pan for enhanced image quality to arrive at the invention of the instant application. Egorov, however, fails to teach utilizing an inter-channel loss.
DeSmoke, however, is analogous art pertinent to the field of endeavor of image correction and disclose a method using an inter-channel loss function for removing smoke from laparoscopic images while maintaining original scene illumination and semantics. DeSmoke teach and an inter-channel loss (DeSmoke: an inter-channel (IC) loss is utilized to describe the differences between channels of a pixel in an image [Sec – Proposed Method, Subsec – Inter-channel (IC) Loss; Eq. 5-8).
DeSmoke further teach that the difference in pixel values across an inter-channel loss accounts for the level of blur for a given pixel region [Sec – Proposed Method, Subsec – Inter-channel (IC) Loss]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to formulate a linear combination of the inter-channel loss taught by DeSmoke, in addition to the previously taught L1 and MS-SSIM loss of Pan in view Zeng, further in view of Egorov to control for blurring caused during the denoising process.
Turning to claim 10, Pan in view Zeng, further in view of Egorov teach The electronic device of claim 8 (as described previously), wherein determining the loss comprises using a linear combination of a mean absolute error (L1) loss, a multi-scale structural similarity loss, (Egorov: the loss function optimizer 204 can train CNN 103 using a combination of different loss functions like an L1 loss function (mean absolute error), an MS-SSIM (multi-scale structural similarity loss) [¶0063; Fig. 2]).
Egorov further teach that their neural network is directed to improve overall visual quality of CFM images prior to any downstream lossy image processing [¶0052]. Therefore, it would have been obvious to one of ordinary skill to combine the adaptive loss function weighting of Egorov to improve the loss optimization of Pan for enhanced image quality to arrive at the invention of the instant application. Egorov, however, fails to teach utilizing an inter-channel loss.
DeSmoke, however, teach and an inter-channel loss (DeSmoke: an inter-channel (IC) loss is utilized to describe the differences between channels of a pixel in an image [Sec – Proposed Method, Subsec – Inter-channel (IC) Loss; Eq. 5-8).
DeSmoke further teach that the difference in pixel values across an inter-channel loss accounts for the level of blur for a given pixel region [Sec – Proposed Method, Subsec – Inter-channel (IC) Loss]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to formulate a linear combination of the inter-channel loss taught by DeSmoke, in addition to the previously taught L1 and MS-SSIM loss of Pan in view Zeng, further in view of Egorov to control for blurring caused during the denoising process.
With respect to claim 16, Pan in view Zeng, further in view of Egorov teach The method of claim 14 (as described previously), wherein determining the loss comprises using a linear combination of a mean absolute error (L1) loss, a multi-scale structural similarity loss, (Egorov: the loss function optimizer 204 can train CNN 103 using a combination of different loss functions like an L1 loss function (mean absolute error), an MS-SSIM (multi-scale structural similarity loss) [¶0063; Fig. 2]).
Egorov further teach that their neural network is directed to improve overall visual quality of CFM images prior to any downstream lossy image processing [¶0052]. Therefore, it would have been obvious to one of ordinary skill to combine the adaptive loss function weighting of Egorov to improve the loss optimization of Pan for enhanced image quality to arrive at the invention of the instant application. Egorov, however, fails to teach utilizing an inter-channel loss.
DeSmoke, however, teach and an inter-channel loss (DeSmoke: an inter-channel (IC) loss is utilized to describe the differences between channels of a pixel in an image [Sec – Proposed Method, Subsec – Inter-channel (IC) Loss; Eq. 5-8).
DeSmoke further teach that the difference in pixel values across an inter-channel loss accounts for the level of blur for a given pixel region [Sec – Proposed Method, Subsec – Inter-channel (IC) Loss]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to formulate a linear combination of the inter-channel loss taught by DeSmoke, in addition to the previously taught L1 and MS-SSIM loss of Pan in view Zeng, further in view of Egorov to control for blurring caused during the denoising process.
Claim(s) 5, 12 & 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pan et al (US 2021/0241429 A1), in view of Zeng et al (“Inheriting Bayer’s Legacy: Joint Remosaicing and Denoising for Quad Bayer Image Sensor, arXiv, 2023), hereinafter referred to as “Zeng”, further in view of Egorov et al (US 2021/0390658 A1), hereinafter referred to as “Egorov, further in view of Smolic et al (EP 3913572 A1), hereinafter referred to as “Smolic”.
Considering claim 5, Pan in view Zeng, further in view of Egorov teach The method of claim 1 (as described previously), however they fail to explicitly teach using a noisy training image with a low exposure value.
Smolic, per contra, is analogous art pertinent to the field of endeavor of the present application and disclose neural network training method utilizing one or more loss functions for image correction. Smolic teach wherein the noisy training image comprises an exposure value zero (EV0) image or a lower exposure value image (Smolic: a dark/noisy image is input into the encoder-decoder architecture [¶0041,45; Fig. 1], such as the low exposure image 2100 [¶0045-47; Fig. 2]).
Smolic further teach that their approach provides a simple yet effective method for correcting noise introduced in images obtained from over or underexposure lighting conditions [¶0028], and that the frequency loss function they implement reduced overall image noise for smoother edges and accurate color capture [¶0045-47]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to utilize the low exposure images of Smolic to as training data for the denoising apparatus taught by Pan in view Zeng, further in view of Egorov to better enable to neural network to more effectively denoise images obtained under non-ideal lighting conditions.
Concerning claim 12, Pan in view Zeng, further in view of Egorov teach The electronic device of claim 8 (as described previously), however they fail to explicitly teach using a noisy training image with a low exposure value.
Smolic, per contra, teach wherein the noisy training image comprises an exposure value zero (EV0) image or a lower exposure value image (Smolic: a dark/noisy image is input into the encoder-decoder architecture [¶0041,45; Fig. 1], such as the low exposure image 2100 [¶0045-47; Fig. 2]).
Smolic further teach that their approach provides a simple yet effective method for correcting noise introduced in images obtained from over or underexposure lighting conditions [¶0028], and that the frequency loss function they implement reduced overall image noise for smoother edges and accurate color capture [¶0045-47]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to utilize the low exposure images of Smolic to as training data for the denoising apparatus taught by Pan in view Zeng, further in view of Egorov to better enable to neural network to more effectively denoise images obtained under non-ideal lighting conditions.
As for claim 18, Pan in view Zeng, further in view of Egorov teach The method of claim 14 (as described previously), however they fail to explicitly teach using a noisy training image with a low exposure value.
Smolic, per contra, teach wherein the noisy training image comprises an exposure value zero (EV0) image or a lower exposure value image (Smolic: a dark/noisy image is input into the encoder-decoder architecture [¶0041,45; Fig. 1], such as the low exposure image 2100 [¶0045-47; Fig. 2]).
Smolic further teach that their approach provides a simple yet effective method for correcting noise introduced in images obtained from over or underexposure lighting conditions [¶0028], and that the frequency loss function they implement reduced overall image noise for smoother edges and accurate color capture [¶0045-47]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to utilize the low exposure images of Smolic to as training data for the denoising apparatus taught by Pan in view Zeng, further in view of Egorov to better enable to neural network to more effectively denoise images obtained under non-ideal lighting conditions.
Claim(s) 6, 13 & 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pan et al (US 2021/0241429 A1), in view of Zeng et al (“Inheriting Bayer’s Legacy: Joint Remosaicing and Denoising for Quad Bayer Image Sensor, arXiv, 2023), hereinafter referred to as “Zeng”, further in view of Egorov et al (US 2021/0390658 A1), hereinafter referred to as “Egorov, further in view of Liu et al (“Joint Demosaicing and Denoising with Self Guidance”, 2020, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)), hereinafter referred to as “Liu”.
Regarding claim 6, Pan in view Zeng, further in view of Egorov teach The method of claim 1 (as described previously), but they fail to teach a total variation loss between denoised images and channels of the ground truth image.
Liu, contrastingly, disclose a joint demosaicing and denoising neural network that leverages a total variation loss to enhance edge-aware smoothness. In this regard, Liu teach wherein determining the loss comprises using a total variation loss between the denoised images and corresponding color channels for the ground truth image (Liu: a modified total variation loss (L) is calculated utilizing between a denoised output image and the green channel of the ground truth [Sec 3.4.2 – Edge-aware smoothness loss; Eq. 10]).
Liu additionally describe that total variation loss is implemented to smooth out noise and unexpected image artifacts, and their modified total variation loss prevents the over-smoothing of edge regions to maintain texture [Sec 3.4.2 – Edge-aware smoothness loss]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement the modified total variation loss utilized by Liu to provide a dynamic edge-ware smoothing function to the denoising neural network of Pan in view Zeng, further in view Egorov to effectively denoise images without a loss of texture and edges.
With respect to claim 13, Pan in view Zeng, further in view of Egorov teach The electronic device of claim 8 (as described previously), but they fail to teach a total variation loss between denoised images and channels of the ground truth image.
Liu, contrastingly teach wherein determining the loss comprises using a total variation loss between the denoised images and corresponding color channels for the ground truth image (Liu: a modified total variation loss (L) is calculated utilizing between a denoised output image and the green channel of the ground truth [Sec 3.4.2 – Edge-aware smoothness loss; Eq. 10]).
Liu additionally describe that total variation loss is implemented to smooth out noise and unexpected image artifacts, and their modified total variation loss prevents the over-smoothing of edge regions to maintain texture [Sec 3.4.2 – Edge-aware smoothness loss]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement the modified total variation loss utilized by Liu to provide a dynamic edge-ware smoothing function to the denoising neural network of Pan in view Zeng, further in view Egorov to effectively denoise images without a loss of texture and edges.
With respect to claim 19, Pan in view Zeng, further in view of Egorov teach The method of claim 14 (as described previously), but they fail to teach a total variation loss between denoised images and channels of the ground truth image.
Liu, contrastingly teach wherein determining the loss comprises using a total variation loss between the denoised images and corresponding color channels for the ground truth image (Liu: a modified total variation loss (L) is calculated utilizing between a denoised output image and the green channel of the ground truth [Sec 3.4.2 – Edge-aware smoothness loss; Eq. 10]).
Liu additionally describe that total variation loss is implemented to smooth out noise and unexpected image artifacts, and their modified total variation loss prevents the over-smoothing of edge regions to maintain texture [Sec 3.4.2 – Edge-aware smoothness loss]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement the modified total variation loss utilized by Liu to provide a dynamic edge-ware smoothing function to the denoising neural network of Pan in view Zeng, further in view Egorov to effectively denoise images without a loss of texture and edges.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael M. Sofroniou whose telephone number is (571)272-0287. The examiner can normally be reached M-F: 8:30 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John M. Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL M SOFRONIOU/Examiner, Art Unit 2661 /JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661
