DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-11 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claim(s) 1-2 and 5-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kimura102 (US20190037102A1) in view of Gresset et al (US20210264579A1) and further in view of Kimura758 (US20180115758A1).
Regarding claims 1, 10 and 11, Kimura102 teaches an image processing apparatus, comprising: at least one memory configured to store instructions; and at least one processor in communication with the at least one memory and configured to execute the instructions to:
generate, based on an image, a first evaluation image for first tone mapping processing;
(Kimura102, "In step S602, reduction processing is stepwisely performed on the luminance image generated in step S601 to generate a first reduced image and a second reduced image.", "Accordingly, in step S602, a hierarchical image including a plurality of images different in frequency band from one another is generated based on the input image.", [0050]; Gresset, "The mini-image MPICf is a reduced, or subsampled, image having a number of pixels PMb less than the number of pixels of the image LOGY_IMAGEf.", [0056]; Kimura102 teaches first reduced image generated from luminance image of input image forming hierarchical image for tone mapping; Gresset teaches reduced/subsampled mini-image as evaluation image representative of intensity; together teach first evaluation image based on image for tone mapping)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of Gresset into the system or method of Kimura102 in order to improve the same tone-mapping device in the same way by simple substitution of one known reduced evaluation image for another known reduced evaluation image, the Gresset mini-image for the Kimura758 first reduced image, thereby reducing memory requirement and number of comparisons with predictable result that the first evaluation image still represents image content for first tone mapping processing, as both references teach reduction by averaging/bilinear filtering. The combination of Kimura102 and Gresset also teaches other enhanced capabilities.
generate, based on the image, a second evaluation image for second tone mapping processing, the second tone mapping processing being different from the first tone mapping processing;
(Kimura102, "The first reduced image and the second reduced image are different in image size from each other, and the second reduced image is an image created by further performing the reduction processing on the first reduced image.", [0050]; Kimura758, "wherein the gain map generation unit applies, for each subject region, the first tone characteristic and the second tone characteristic to the at least one of the one or more reduced images, and wherein the first tone characteristic and the second tone characteristic are mutually different tone characteristics associated with a respective subject region included in the input image.", [0007]; Kimura102 teaches second evaluation image distinct from first; Kimura758 expressly teaches first and second tone characteristics are mutually different; together teach second evaluation image for different second tone mapping processing)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of Kimura758 into the system or method of Kimura102 and Gresset in order to provide region-adaptive tone mapping by applying mutually different first and second tone characteristics associated with respective subject regions, improving local contrast and visibility for background and person regions with predictable use of known subject-region gain control in the hierarchical reduced evaluation image framework. The combination of Kimura102, Gresset and Kimura758 also teaches other enhanced capabilities.
The combination of Kimura102, Gresset and Kimura758 further teaches:
generate, based on the first evaluation image, a first gain map for the first tone mapping processing;
(Kimura102, "In step S603, a first tone characteristic is applied to the luminance image generated in step S601 and to the first reduced image and the second reduced image generated in step S602 to generate corresponding gain maps.", [0051]; Kimura758, "In step S203, the first gain conversion unit 104 applies a first tone characteristic to the hierarchized images (the image group formed by the input image, and the first reduced image and the second reduced image that are generated in step S202), and generates a gain map that corresponds to each image.", [0041]; both Kimura102 and Kimura758 teach first gain map from first reduced image using first tone characteristic)
apply the first tone mapping processing that is based on the first gain map to the second evaluation image;
(Kimura102, "In step S702, first gain processing is performed on the luminance image generated in step S701 with use of the first gain map generated in step S501 described above.", [0060]; Gresset, "for a second image of the succession of images, the modification of the second image according to the first mini-image in order to generate an output image.", [0017]; "According to an embodiment, each pixel OUTPUT_Pq of the output image is equal to OUTPUT_Pq=GMPq*Pq, where Pq represents the value of any of the channel pf the pixel, OUTPUT_Pq represents the value of the corresponding channel in the output image, and the value GMPq is a gain value dependent on the context value.", [0007]; Kimura102 lacks explicit teaching of applying first gain map specifically to second evaluation image before second gain generation; Gresset expressly teaches modifying second image according to first mini-image using gain GMPq; together Kimura102 and Gresset teach applying first tone mapping based on first gain map to second evaluation image; Incorporating Gresset into Kimura102 would use reduced evaluation image to efficiently drive tone mapping of second evaluation)
generate, based on the second evaluation image to which the first tone mapping processing has already been applied, a second gain map for the second tone mapping processing;
(Kimura102, "In step S703, a correction gain map is generated from the second tone characteristic based on the luminance image subjected to the first gain processing in step S702 described above.", [0061]; "A first exemplary embodiment of the disclosure is characterized in that a second tone characteristic that cancels a luminance change by a first tone characteristic is applied to a first gain map, which is generated from a reduced image of an input image and the first tone characteristic, to generate a second gain map, and gain processing is performed on the input image with use of the second gain map.", [0028]; correction/second gain map is generated from second tone characteristic based on image already subjected to first gain processing; teaching generating second gain map based on second evaluation to which first tone mapping already applied)
apply the first tone mapping processing that is based on the first gain map to the image; and
(Kimura102, "The present exemplary embodiment is characterized in that the first gain processing is performed on an input image with use of the first gain map, and then second gain processing is performed thereon", [0094]; Kimura758, "In step S207, the gain processing unit 108 performs processing for applying gain to the input image by using the composed gain map that was generated in step S206.", [0046]; Kimura102 teaches first gain processing performed on input image with first gain map; Kimura758 teaches applying gain to input image using composed gain map including first tone characteristic; together teach applying first tone mapping based on first gain map to image)
apply the second tone mapping processing that is based on the second gain map to the image to which the first tone mapping processing has already been applied.
(Kimura102, "In step S503, processing to apply a gain to the input image with use of the second gain map generated in step S502 is performed.", [0042]; "In step S703, a correction gain map is generated from the second tone characteristic based on the luminance image subjected to the first gain processing in step S702 described above.", [0061]; Gresset, "The values of the gain map are generated based on the image LOGY_IMAGEf and on the mini-image MPICf−1.", [0062]; Kimura102 teaches generating second gain map from image already subjected to first gain processing and applying second gain map to input image, which functionally applies second tone mapping to image to which first already applied; Gresset teaches gain map based on image and mini-image; incorporating Gresset into Kimura102 would use mini-image evaluation to drive sequential gain application)
Regarding claim 2, the combination of Kimura102, Gresset and Kimura758 teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 1, wherein each of the first tone mapping processing and the second tone mapping processing is one of brightness correction, contrast correction, and haze correction.
(Kimura102, "Performing the processing according to the present exemplary embodiment in the above-described manner makes it possible to achieve the contrast improvement effect of improving the contrast only in the high-frequency region where the luminance signal is varied and not changing brightness in the low-frequency region where the luminance signal is hardly varied.", [0082]; "According to an aspect of the embodiments, an apparatus includes an image generation unit configured to generate, from an input image, a hierarchical image including a plurality of images different in frequency band from one another, a first gain map generation unit configured to generate a first gain map with use of the hierarchical image and a first tone characteristic, a second gain map generation unit configured to generate a second gain map with use of the first gain map and a second tone characteristic", [0005]; tone processing is with different tone characteristics that achieve contrast improvement effect and brightness handling in low-frequency region; contrast improvement is contrast correction which is one of listed options; applying contrast correction as tone mapping is predictable; thus each of first and second tone mapping is one of brightness correction, contrast correction, haze correction)
Regarding claim 5, the combination of Kimura102, Gresset and Kimura758 teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 1, wherein the at least one processor executes the instructions to generate the first evaluation image based on the image and on a low-frequency image generated from the image.
(Kimura102, "In step S602, reduction processing is stepwisely performed on the luminance image generated in step S601 to generate a first reduced image and a second reduced image.", "Accordingly, in step S602, a hierarchical image including a plurality of images different in frequency band from one another is generated based on the input image.", [0050]; Gresset, "The mini-image MPICf is a reduced, or subsampled, image having a number of pixels PMb less than the number of pixels of the image LOGY_IMAGEf.", ; Kimura102 teaches reduction processing using bilinear method which is low-pass filtering generating low-frequency image, and hierarchical image different in frequency band based on input image; Gresset teaches reduced mini-image as low-frequency representative; together teach first evaluation image based on image and low-frequency image; incorporating Gresset into Kimura102 would reduce memory by using subsampled low-frequency image)
Regarding claim 6, the combination of Kimura102, Gresset and Kimura758 teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 1, wherein the at least one processor executes the instructions to generate the second evaluation image based on the image and on a low-frequency image generated from the image.
(Kimura102, "The first reduced image and the second reduced image are different in image size from each other, and the second reduced image is an image created by further performing the reduction processing on the first reduced image.", "Accordingly, in step S602, a hierarchical image including a plurality of images different in frequency band from one another is generated based on the input image.", [0050]; Gresset, "Each pixel of the mini-image is representative of the intensity of a plurality of pixels of the corresponding image LOGY_IMAGEf.", [0056]; Kimura102 teaches second reduced image as further low-frequency image and hierarchical image generation; Gresset teaches mini-image representative of intensity of plurality of pixels as low-frequency evaluation; together teach second evaluation image based on image and low-frequency image)
Regarding claim 7, the combination of Kimura102, Gresset and Kimura758 teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 1, wherein the image is a monochrome image.
(Gresset, "If the images of the succession of images are in monochrome, the greyness is not computed", [0153]; "The process can be used with greyscale images, color images, or hyperspectral images.", [0044]; processing applicable to monochrome images)
Regarding claim 8, the combination of Kimura102, Gresset and Kimura758 teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 1, wherein the image is a color image.
(Gresset, "The process can be used with greyscale images, color images, or hyperspectral images. In the case of color images or hyperspectral images, the process may be applied to an intensity image computed from the color or hyperspectral image.", [0044]; color image processing)
Regarding claim 9, the combination of Kimura102, Gresset and Kimura758 teaches an image capturing apparatus, comprising:
the image processing apparatus according to claim 1; and an image capturing sensor configured to generate the image.
(Kimura102, Gresset, Kimura758, see comments on Claim 1; Kimura102, "An optical system 101 includes a lens group including a zoom lens and a focus lens, a diaphragm adjustment device, and a shutter device.", "The imaging unit 102 is a photoelectric conversion device, such as a charge-coupled device (CCD) sensor and a complementary metal-oxide semiconductor (CMOS) sensor, that photoelectrically converts a light flux of an object that has passed through the optical system 101 into an electric signal.", [0030])
Claim(s) 3-4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kimura102 (US20190037102A1) in view of Gresset et al (US20210264579A1) and further in view of Kimura758 (US20180115758A1) and Li et al (US20180122051A1).
Regarding claim 3, the combination of Kimura102, Gresset and Kimura758 teaches its/their respective base claim(s).
The combination further teaches the image processing apparatus according to claim 2, wherein
the first tone mapping processing is the haze correction, and
to generate the first gain map, the at least one processor executes the instructions to generate a transmission map based on the first evaluation image.
(Kimura102, "In step S602, reduction processing is stepwisely performed on the luminance image generated in step S601 to generate a first reduced image and a second reduced image.", [0050]; Li, "In various embodiments, the medium transmission map may be determined, including determining dark channels of the input image; determining a base layer of the input image from the dark channels using the edge-preserving smoothing filter; and determining the medium transmission map based on the base layer.", [0018]; Li, "Various embodiments provide a method of processing an input image to generate a de-hazed image.", [0005]; Kimura102 teaches first reduced image as first evaluation image; Li teaches dark channels as evaluation image and transmission map based on base layer for de-hazing with amplification factor (1/t-1) as gain)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of Li into the system or method of Kimura102, Gresset and Kimura758 in order to provide known dark-channel transmission map for haze correction. The combination of Kimura102, Gresset, Kimura758 and Li also teaches other enhanced capabilities.
Regarding claim 4, the combination of Kimura102, Gresset and Kimura758 teaches its/their respective base claim(s).
The combination of Kimura102, Gresset, Kimura758 and Li further teaches the image processing apparatus according to claim 2, wherein
the second tone mapping processing is the haze correction, and
to generate the second gain map, the at least one processor executes the instructions to generate a transmission map based on the second evaluation image.
(Kimura102, "The first reduced image and the second reduced image are different in image size from each other, and the second reduced image is an image created by further performing the reduction processing on the first reduced image.", [0050]; Li, "In various embodiments, the medium transmission map may be derived from the base layer, for example, using values of the base layer as exponents of an exponentiation operation to determine the values of the medium transmission map.", [0021]; Kimura102 teaches second evaluation image as further reduced low-frequency image; Li teaches base layer as low-frequency evaluation image and transmission map derived from base layer for haze correction; incorporating Li into Kimura102 would provide haze correction from second evaluation image)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANXUN YANG whose telephone number is (571)272-9874. The examiner can normally be reached on MON-FRI: 8AM-5PM Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272- 1000.
/JIANXUN YANG/
Primary Examiner, Art Unit 2662 7/11/2026
Prosecution Insights