DETAILED ACTION
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 15 recites the limitation "the target illumination information" in claim 12. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-6, 11-17 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Kang et al (US 20220292635) .
The applied reference has a common assignee with the instant application. Based upon the earlier effectively filed date of the reference, it constitutes prior art under 35 U.S.C. 102(a)(2). This rejection under 35 U.S.C. 102(a)(2) might be overcome by: (1) a showing under 37 CFR 1.130(a) that the subject matter disclosed in the reference was obtained directly or indirectly from the inventor or a joint inventor of this application and is thus not prior art in accordance with 35 U.S.C. 102(b)(2)(A); (2) a showing under 37 CFR 1.130(b) of a prior public disclosure under 35 U.S.C. 102(b)(2)(B) if the same invention is not being claimed; or (3) a statement pursuant to 35 U.S.C. 102(b)(2)(C) establishing that, not later than the effective filing date of the claimed invention, the subject matter disclosed in the reference and the claimed invention were either owned by the same person or subject to an obligation of assignment to the same person or subject to a joint research agreement.
Regarding claim 1, Kang discloses a method performed by an electronic device (abstract method and apparatus with image correction is provided), comprising:
extracting, from an input image, original illuminant information representing an original illuminant color-cast of the input image (¶98 In operation 630, the image correction apparatus estimates a chromaticity and a mixture coefficient map for each illumination. For example, the image correction apparatus may identify the number of illuminant chromaticities that affect the input image. The image correction apparatus may generate the mixture coefficient maps based on the identified number), the extracting performed by applying an illuminant information extraction model to the input image (¶99 the image correction apparatus may extract the identified total number of sets of chromaticity information and the correspondingly identified total number of mixture coefficient maps selectively using a neural network model that had been previously trained with respect to a training total number of illuminants that is the same as the identified total number of illuminants, e.g., compared to another stored neural network model that had been previously trained with respect to a different training total number of illuminants than the identified total number of illuminants);
generating, from the original illuminant information, intermediate illuminant information representing an intermediate illuminant color-cast of the input image (¶101 In operation 640, the image correction apparatus estimates the illumination map. For example, the image correction apparatus may calculate a partial illumination map of each of illuminants based on a multiplication between the chromaticity information and the mixture coefficient maps, and calculate the illumination map by adding respective partial illumination maps of the illuminants), the generating performed by applying an illuminant information generation model to the original illuminant information (¶128 The image correction apparatus 1000 may be an image processing apparatus that uses deep image processing, artificial intelligence (AI) computing or machine learning, such as to generate the illumination map or to extract the chromaticity information and coefficient maps);
determining target illuminant information based on the extracted original illuminant information and based on the generated intermediate illuminant information (¶102 In operation 650, e.g., dependent on the estimation in operation 640, the image correction apparatus adjusts the chromaticity information based on an illuminant selected or set to be preserved. When the chromaticity information is adjusted, an illumination map for preserving information of a portion of the illuminants may ultimately be generated); and
generating an output image by applying the determined target illuminant information to the input image (¶103 In operation 660, the image correction apparatus generates an image by applying the generated illumination map to the input image. When the chromaticity information is adjusted, the image correction apparatus generates an image that preserves therein a portion of color casts. Thus, the image correction apparatus may perform white balancing separately for each illumination).
Regarding claim 2, Kang discloses the method of claim 1, wherein the determining of the target illuminant information comprises combining at least a portion of the original illuminant information and at least a portion of the intermediate illuminant information (¶102 In operation 650, e.g., dependent on the estimation in operation 640, the image correction apparatus adjusts the chromaticity information based on an illuminant selected or set to be preserved. When the chromaticity information is adjusted, an illumination map for preserving information of a portion of the illuminants may ultimately be generated; ¶103 .
Regarding claim 3, Kang discloses the method of claim 1, wherein the generating of the output image comprises: removing the original illuminant color-cast from the input image (¶109 the electronic apparatus 700 may apply white balance selectively to a portion of illuminants, and generate a white-balanced image in various ways based on a user's selection while ensuring vividness of a scene) and adding a color-cast of the target illuminant information to the input image (¶108 the electronic apparatus 700 may obtain a first relighted image 710 by applying, to the white-balanced image 790, a relighting map that is generated by substituting second chromaticity information of a second illuminant with white information (e.g., a vector (1,1,1)) in response to a user input 701. The electronic apparatus 700 may also obtain a second relighted image 720 by applying, to the white-balanced image 790, a relighting map that is generated by substituting first chromaticity information of a first illuminant with white information in response to a user input 702. Thus, the electronic apparatus 700 may perform white balancing selectively based on a user's preference).
Regarding claim 4, Kang discloses the method of claim 1, wherein the determining of the target illuminant information comprises: obtaining a user input and according thereto controlling contribution of the original illuminant information or the intermediate illuminant information to the target illuminant information (¶109 the image correction apparatus may change chromaticity information of each illuminant in response to a user input).
Regarding claim 5, Kang discloses the method of claim 1, further comprising: outputting an embedding vector produced by the illuminant information extraction model in extracting the original illuminant information from the input image (¶60 Chromaticity information may include a chromaticity vector that defines a chromaticity of an illuminant based on a color space), wherein the intermediate illuminant information is generated by applying the illuminant information generation model to the embedding vector of the input image and to the original illuminant information (¶64 As represented by Equation 4, the input image I.sub.ab may be construed as being a result that could be obtained by applying an illumination map L.sub.ab to the white-balanced image Î.sub.ab through an element-wise multiplication).
Regarding claim 6, Kang discloses the method of claim 1, further comprising: determining whether to apply the illuminant information extraction model and the illuminant information generation model to the input image, based on color diversity of the input image (¶95 In operation 620, the image correction apparatus determines whether a plurality of illuminations are present in the image. For example, the image correction apparatus may identify a type of chromaticities of illuminants that induce color casts affecting pixels of the image; ¶96 In operation 621, when only a color cast by a single illuminant is present in the image, the image correction apparatus adjusts a white balance in a first way).
Regarding claim 11, Kang discloses a non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configured the one or more processors to perform the method of claim 1 (¶125).
Regarding claim(s) 12-17 (drawn to a device):
The rejection/proposed combination of Kang, explained in the rejection of method claim(s) 1-6, anticipates/renders obvious the steps of the device of claim(s) 12-17 because these steps occur in the operation of the proposed combination as discussed above. Thus, the arguments similar to that presented above for claim(s) 1-6 is/are equally applicable to claim(s) 12-17.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kang as applied to claim 6 above, and further in view of Lim et al (US 20220201203).
Regarding claim 8, Kang discloses the method of claim 6, but fails to teach where Lim wherein the determining of whether to apply the illuminant information extraction model and the illuminant information generation model to the input image based on the color diversity of the input image comprises: determining a color diversity score of the input image by applying a color diversity determination model to the input image (¶178 the processor 120 or 260 may extract color information from the image and may calculate a color score based on the diversity of colors; ¶192 based on the scene information of the original image, the image effect may include automatic image quality correction (e.g., auto enhance)).
Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the invention to have implemented the teaching of determining of whether to apply the illuminant information extraction model and the illuminant information generation model to the input image based on the color diversity of the input image comprises: determining a color diversity score of the input image by applying a color diversity determination model to the input image from Lim into the method as disclosed by Kang. The motivation for doing this is to improve automatic image quality correction.
Claim(s) 9 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kang as applied to claim 1 and 12 above, and further in view of Dinerstein et al (US Patent 9838660 B1).
Regarding claim 9, Kang discloses the method of claim 1, but fails to teach where Dinerstein teaches determining whether to apply the illuminant information extraction model and the illuminant information generation model to the input image, based on a score for a gray pixel corresponding to an achromatic-color object in the input image (col 6 lines 56-63 FIG. 3C illustrates another extension of a method of determining white balancing gains, taking into account pixels that have a saturation of zero or close to zero. Pixels having a saturation of zero or close to zero means that the pixels are gray or almost gray. These pixels may affect the optimization, since they weakly constraint their corresponding angles. Thus, the method of FIG. 3C models the pixels that are gray or almost gray separately; col 7 lines 14-30 the joint segmentation and AWB circuit 1205 extracts pixels that are gray or “close to gray”, which forms a set of M pixels; At S325, the joint segmentation and AWB circuit 1205 performs an optimization step on the sets of N1-NK pixels and M gray pixels).
Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the invention to have implemented the teaching of determining whether to apply the illuminant information extraction model and the illuminant information generation model to the input image, based on a score for a gray pixel corresponding to an achromatic-color object in the input image from Dinerstein into the method as disclosed by Kang. The motivation for doing this is to improve imaging techniques including white balancing.
Regarding claim(s) 19 (drawn to a device):
The rejection/proposed combination of Kang and Dinerstein, explained in the rejection of method claim(s) 9, anticipates/renders obvious the steps of the device of claim(s) 19 because these steps occur in the operation of the proposed combination as discussed above. Thus, the arguments similar to that presented above for claim(s) 9 is/are equally applicable to claim(s) 19.
Allowable Subject Matter
Claim 7, 10, 18 and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claim 7, and similarly regarding claim 18, the prior art of record, alone or in combination, fails to teach at least wherein the determining of whether to apply the illuminant information extraction model and the illuminant information generation model to the input image based on the color diversity of the input image comprises: performing linear regression on pixel values of pixels of the input image; and determining a color diversity score based on a difference between a result of the linear regression and the pixel values, the color diversity score indicating the color diversity.
At best, Kang (US 20220292635) teaches in ¶95 In operation 620, the image correction apparatus determines whether a plurality of illuminations are present in the image. For example, the image correction apparatus may identify a type of chromaticities of illuminants that induce color casts affecting pixels of the image, and in ¶96 In operation 621, when only a color cast by a single illuminant is present in the image, the image correction apparatus adjusts a white balance in a first way
Regarding claim 10, and similarly regarding claim 20, the prior art of record, alone or in combination, fails to teach at least determining candidate gray pixels from among pixels of the input image based on a pixel value and a gray locus mapped to an achromatic-color object; detecting a fake gray pixel corresponding to a chromatic-color object from among the determined candidate gray pixels; and based on the detected fake gray pixel, either generating the original illuminant information, generating the intermediate illuminant information, determining the target illuminant information, or generating the output image.
At best, Dinerstein et al (US Patent 9838660 B1) teaches in col 6 lines 56-63 FIG. 3C illustrates another extension of a method of determining white balancing gains, taking into account pixels that have a saturation of zero or close to zero. Pixels having a saturation of zero or close to zero means that the pixels are gray or almost gray. These pixels may affect the optimization, since they weakly constraint their corresponding angles. Thus, the method of FIG. 3C models the pixels that are gray or almost gray separately.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEVIN KY whose telephone number is (571)272-7648. The examiner can normally be reached Monday-Friday 9-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEVIN KY/Primary Examiner, Art Unit 2671
Prosecution Insights