DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-10 are pending.
Priority
Acknowledgement is made of Applicant’s claim of priority from JP2023-008181, filed January 23, 2023.
Response to Arguments
Applicant’s arguments, see p. 6, filed April 16, 2026, with respect to the Objection to the Title have been fully considered and are persuasive. The amendment to the Title has overcome the previous objection and it has therefore been withdrawn.
Applicant’s arguments, see p. 6-7, filed April 16, 2026, with respect to the 35 USC 101 abstract idea rejections have been fully considered and are persuasive. The newly added limitations have overcome the previous rejections because a person having ordinary skill in the art cannot mentally or manually perform “Retinex-theory-based dynamic range compressing using the illumination distribution”. Therefore, the claims recite significantly more than the abstract idea and the rejections have therefore been withdrawn.
Applicant’s arguments, see p. 7-9, filed April 16, 2026, with respect to the 35 USC 102 and 103 rejections have been fully considered but are moot because of the new grounds of rejection, presented in the sections below. The 35 USC 102 rejections have been withdrawn because the Lukac reference does not teach each limitation of claim 1. However, the newly proposed Zhu and Liu references are used to reject the newly added limitations of claim 1, as described in the 35 USC 103 rejections below. Therefore, the 35 USC 103 rejections of the claims are upheld, and consequently, THIS ACTION IS FINAL.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 7-10 are rejected under 35 U.S.C. 103 as being unpatentable over Lukac et al. (US 11,017,511 B2, filed February 13, 2019) in view of Yiwen Zhu (US 2024/0085317 A1, with priority to WO 2022168517 A1, published August 11, 2022, US PGPub used herein for translation and mapping purposes) further in view of Liu et al. (“Dehaze Enhancement Algorithm Based on Retinex Theory for Aerial Images Combined with Dark Channel”, included with Applicant’s IDS).
Regarding claim 1, Lukac teaches an image processing apparatus comprising:
one or more memories (Lukac, Col. 2, lines 37-57, the material disclosed herein also may be implemented as instructions stored on a machine-readable medium (i.e., memory), which may be read and executed by one or more processors); and
one or more processors (Lukac, Col. 2, lines 37-57, the material disclosed herein also may be implemented as instructions stored on a machine-readable medium (i.e., memory), which may be read and executed by one or more processors), wherein the one or more memories and the one or more processors are configured to:
calculate an atmospheric transmittance distribution based on an input image (Lukac, Col. Col. 8, lines 23-57, the process may include “estimate transmission distribution”. This process may include “determine minimum ratio between neighbor intensities and atmospheric light”, and particularly, by finding the minimum ratio between pixel intensities inside the neighborhood centered at a pixel location and the atmospheric light (i.e., calculate an atmospheric transmittance distribution based on an input image).
Although Lukac teaches determining an atmospheric transmittance distribution (Lukac, Col. Col. 8, lines 23-57), Lukac does not explicitly teach to “calculate an illumination distribution by inverting the atmospheric transmittance distribution to determine an intensity of illumination light that corresponds to each pixel position in the input image”. However, in an analogous field of endeavor, Zhu teaches the dark channel, which is the darkest pixel value among the color channels in each local region of the RGB image, is regarded as the inverted value of the transmittance of light (Zhu, Para. [0051]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the apparatus of Lukac with the teachings of Zhu by including calculating an illumination distribution by inverting the atmospheric transmittance distribution to determine the darkest pixel value among the color channels in each local region of the RGB image (i.e., the intensity of illumination light in each pixel position). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for improving visibility of an image reduced by scattering of natural light or the like due to fine particles of fog or haze, as recognized by Zhu.
Although Lukac in view of Zhu teaches determining an illumination distribution by inverting the transmittance distribution (Zhu, Para. [0051]), they do not explicitly teach to “sharpen the input image based on an illumination distribution that has been calculated based on the transmittance distribution by performing both dehazing processing using the atmospheric transmittance distribution and Retinex-theory-based dynamic range compression using the illumination distribution”. However, in an analogous field of endeavor, Liu teaches combining Retinex and DCP defogging process. Retinex can meet the dynamic range compression, enhance the details and maintain the color balance (Liu, pg. 7). Using the dark channel defogging algorithm, the atmospheric light value and transmittance of the fog incident light component and the fog reflected light component are re-estimated, respectively, to obtain new fog-free incident light component and fog-free reflected light component (Liu, pg. 11).\
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the apparatus of Lukac in view of Zhu with the teachings of Liu by sharpening the input image by performing dehazing processing using both atmospheric transmittance distribution and Retinex-theory-based dynamic range compression using the illumination distribution. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for generating a fog-free image, as recognized by Liu. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 7, Lukac in view of Zhu further in view of Liu teaches the image processing apparatus according to claim 1, further comprising:
an image capturing unit (Lukac, Col. 11, lines 48-48, the system has an image capture unit that may include camera hardware to capture images),
wherein the one or more memories and the one or more processors are further configured to generate the input image by performing development processing on a RAW image that is obtained by image capturing by the image capturing unit (Lukac, Col. 11, lines 48-48, the system has an image capture unit that may include camera hardware to capture images, or may include a camera module that receives raw image data from camera sensors for processing).
Regarding claim 8, Lukac in view of Zhu further in view of Liu teaches the image processing apparatus according to claim 1, wherein the one or more memories and the one or more processors are further configured to:
obtain, as the input image, an image captured by an image capturing apparatus (Lukac, Col. 11, lines 48-48, the system has an image capture unit that may include camera hardware to capture images).
Claim 9 recites a method with steps corresponding to the elements of the system recited in Claims 1. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding system claim. Additionally, the rationale and motivation to combine the Lukac, Zhu and Liu references, presented in rejection of Claim 1, apply to this claim.
Claim 10 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 1. Therefore, the recited programming instructions of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Lukac, Zhu and Liu references, presented in rejection of Claim 1, apply to this claim. Finally, the Lukac, Zhu, and Liu references discloses a computer readable storage medium (Lukac, Col. 2, lines 37-57, non-transitory article, such as a non-transitory computer readable medium).
Claims 2 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Lukac et al. (US 11,017,511 B2, filed February 13, 2019) in view of Yiwen Zhu (US 2024/0085317 A1, with priority to WO 2022168517 A1, published August 11, 2022, US PGPub used herein for translation and mapping purposes) further in view of Liu et al. (“Dehaze Enhancement Algorithm Based on Retinex Theory for Aerial Images Combined with Dark Channel”, included with Applicant’s IDS), as applied to claims 1 and 7-10 above, and further in view of Angamuthu Ganesan et al. (US 12,482,079 B2).
Regarding claim 2, Lukac in view of Zhu further in view of Liu teaches the image processing apparatus according to claim 1, wherein the one or more memories and the one or more processors are further configured to:
calculate the transmittance distribution based on the input image and the ambient light (Lukac, Col. Col. 8, lines 23-57, the process may include “estimate transmission distribution”. This process may include “determine minimum ratio between neighbor intensities and atmospheric light”, and particularly, by finding the minimum ratio between pixel intensities inside the neighborhood centered at a pixel location and the atmospheric light (i.e., ambient light)).
Although Lukac in view of Zhu further in view of Liu teaches obtaining the image airlight A by averaging several darkest pixels in the input image (Lukac, Col. 8, lines 7-22), they do not explicitly teach to “generate, based on the input image, a dark channel image whose pixel values are minimum color channel values of respective local regions in the input image” and “calculate ambient light based on the dark channel image”. However, in an analogous field of endeavor, Angamuthu Ganesan teaches obtaining the dark channel image corresponding to the low-resolution image by determining the color channel (out of Red, Green, and Blue color channels) of minimum intensity (or near zero intensity) for each local patch (or a pixel) of the low-resolution image (i.e., pixel values are minimum color channel values of respective local regions in the input image) (Angamuthu Ganesan, Col. 7, lines 24-40). The atmospheric light estimation module identifies the top 10% brightest pixels in the dark channel image. Further, the atmospheric light estimation module compares each of the identified pixels (e.g. top 10% brightest pixels) with corresponding pixels in the actual received input digital image to identify the pixel(s) with highest intensity in the input digital image. The identified pixel(s) are then selected as atmospheric light, and accordingly, the atmospheric light value A (i.e., ambient light) is identified as the intensity of the identified pixel(s) (Angamuthu Ganesan, Col. 7 line 56 – Col. 8 line 4).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the apparatus of Lukac in view of Zhu further in view of Liu with the teachings of Angamuthu Ganesan by including determining a dark channel image whose pixel values are minimum color channel values and using the dark channel image to calculate ambient light (i.e., atmospheric light value). One having ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to combine these references because doing so would allow for improving visibility in hazy videos/images, as recognized by Angamuthu Ganesan. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 6, Lukac in view of Zhu further in view of Liu teaches the image processing apparatus according to claim 1, as described above.
Although Lukac in view of Zhu further in view of Liu teaches correcting a hazed image (Lukac, Col. 10, lines 8-23), they do not explicitly teach to “perform gamma correction on the sharpened input image”. However, in an analogous field of endeavor, Angamuthu Ganesan teaches adjusting the brightness of the de-hazed output image (i.e., sharpened input image) by using gamma correction technique (Angamuthu Ganesan, Col. 9, lines 20-39).
The proposed combination as well as the motivation for combining the Lukac, Zhu, Liu and Angamuthu Ganesan references presented in the rejection of Claim 2, apply to Claim 6 and are incorporated herein by reference. Thus, the system recited in Claim 6 is met by Lukac in view of Zhu further in view of Liu and Angamuthu Ganesan.
Claims 3-5 are rejected under 35 U.S.C. 103 as being unpatentable over Lukac et al. (US 11,017,511 B2, filed February 13, 2019) in view of Yiwen Zhu (US 2024/0085317 A1, with priority to WO 2022168517 A1, published August 11, 2022, US PGPub used herein for translation and mapping purposes) further in view of Liu et al. (“Dehaze Enhancement Algorithm Based on Retinex Theory for Aerial Images Combined with Dark Channel”, included with Applicant’s IDS) and Angamuthu Ganesan et al. (US 12,482,079 B2), as applied to claims 2 and 6 above, and further in view of Chen et al. (“A High-Efficiency and High-Speed Gain Intervention Refinement Filter for Haze Removal”), Han et al. (US 2008/0037868 A1) and Shinnosuke Osawa (US 2021/0233218 A1).
Regarding claim 3, Lukac in view of Zhu further in view of Liu and Angamuthu Ganesan teaches the image processing apparatus according to claim 2, as described above.
Although Lukac in view of Zhu further in view of Liu and Angamuthu Ganesan teaches sharpening an input image based on an illumination distribution calculated from a transmittance distribution (Lukac, Col. 10, lines 8-23), they do not explicitly teach to “calculate a first gain based on the input image, the ambient light, an application amount of dehazing processing, and the transmittance distribution”. However, in an analogous field of endeavor, Chen teaches obtaining a refined transmission map through gain intervention (i.e., based on transmittance distribution), wherein the gain coefficient is calculated based on the total number of pixels, the difference of position between the minimum intensity of trichromatic components within each pixel in the image (i.e., based on the input image) and the dark channel intensity of the image (i.e., based on the ambient light and amount of dehazing processing) (Chen, pg. 755).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the apparatus of Lukac in view of Zhu further in view of Liu and Angamuthu Ganesan with the teachings of Chen by including a gain calculation based on a transmission map (i.e., transmittance distribution), the input image, the ambient light, and an application amount of dehazing processing. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for dehazing images, as recognized by Chen.
Although Lukac in view of Zhu further in view of Liu, Angamuthu Ganesan and Chen teaches refining a transmission map through gain intervention (Chen, pg. 755), they do not explicitly teach to “obtain a second gain based on the illumination distribution”. However, in an analogous field of endeavor, Han teaches determining an illumination level (i.e., illumination distribution) and providing a tone gain corresponding to the illumination level (Han, Para. [0093]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the apparatus of Lukac in view of Zhu further in view of Liu, Angamuthu Ganesan and Chen with the teachings of Han by including a second gain based on an illumination distribution (i.e., illumination level). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for improving the visibility of an image according to the luminance of the environment, as recognized by Han.
Although Lukac in view of Zhu further in view of Liu, Angamuthu Ganesan, Chen and Han teaches refining a transmission map through gain intervention (Chen, pg. 755), they do not explicitly teach to “sharpen the input image based on a third gain which has been obtained based on the first gain, the second gain, and the illumination distribution; on the input image; on the transmittance distribution; on the application amount; and on the ambient light”. However, in an analogous field of endeavor, Osawa teaches a third gain map generation unit applies based on a mixed ratio image, a value of the second gain map to a gradation area and a value of the first gain map to an area that is not the gradation area to generate a third gain map (i.e., third gain based on first and second gain). The gain processing unit performs gain processing on the input image supplied via the image input unit based on the third gain map generated by the third gain map generation unit. The image on which tone correction (tone compression) by the gain processing based on the third gain map has been performed by the gain processing unit is output as an output image (i.e., sharpen input image based on third gain) (Osawa, Paras. [0021]-[0022]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the apparatus of Lukac in view of Zhu further in view of Liu, Angamuthu Ganesan, Chen and Han with the teachings of Osawa by including a sharpening an input image using third gain based on the first and second gain (and based on illumination distribution, input image, transmittance distribution, application amount and on the ambient light because first and second gain are based on these values). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for performing gain processing on an optically blurred portion in an image, as recognized by Osawa. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Regarding claim 4, Lukac in view of Zhu further in view of Liu, Angamuthu Ganesan, Chen, Han and Osawa teaches the image processing apparatus according to claim 3, wherein the one or more memories and the one or more processors are further configured to:
use, as the application amount, a haze density that has been obtained based on a luminance value of the input image (Lukac, Col. 7, lines 39-50, The process then may include “obtain image statistics”, and particularly obtain the statistics, or the image data values, to be used to estimate the haze corruption. In the example used herein, pixel luminance data will be used (i.e., haze density based on luminance value)).
Regarding claim 5, Lukac in view of Zhu further in view of Liu, Angamuthu Ganesan, Chen, Han and Osawa teaches the image processing apparatus according to claim 3, wherein the one or more memories and the one or more processors are further configured to:
use, as the application amount, a value that has been set according to a user operation (Lukac, Col. 7, lines 23-37, process may include “detect haze” and this may be performed by known statistics and atmospheric scattering model analysis techniques or determined by the user such as by selecting an area by bounding box on a screen as one example (i.e., a value set according to a user)).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Emma Rose Goebel whose telephone number is (703)756-5582. The examiner can normally be reached Monday - Friday 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Emma Rose Goebel/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights