DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1 – 3, 10 – 13 and 19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Landy; Harvey et al. (US 20220311981 A1; hereinafter simply referred to as Landy).
Regarding independent claim 1, Landy teaches:
A method for encoding color lookup tables (LUTs) (See ¶ 19, 29 wherein the LUTs are used for color transformations (color LUTs) and wherein they are encoded into 3-dimensional cubes with each axis of the cube representing a particular color parameter)
determining a first identifier corresponding to a first LUT of a plurality of LUTs, each LUT of the plurality of LUTs comprising mappings from input color values to output color values (See ¶ 28, 44, 45, 31, 19, Figure 1, wherein the LUT database, ‘120’ in figure 1, stores a plurality of LUTs, wherein each LUT corresponds to a specific type of scene (identifier) and wherein each of the plurality of LUTs contains color mappings from input color values to output color values)
providing, to a trained machine learning model, the first identifier and an input lattice, the trained machine learning model having been jointly trained on the plurality of LUTs and identification information corresponding to each LUT in the plurality of LUTs (See ¶ 35 – 39, 28, Figure 1, wherein the colorization logic (machine learning model), ‘102’ in figure 1, receives as input the LUT (the LUT corresponding to a specific scene, representing the identifier) and an input lattice (input images ‘104’ in figure 1, see applicant’s specification ¶ 12 wherein the input lattice is an input image) and wherein the trained machine learning model is trained on the plurality of LUTs and their associated identifiers)
obtaining, from the trained machine learning model, an output lattice corresponding to the first LUT (See ¶ 28 - 31 wherein the trained machine learning models (colorization unit ‘102’ in figure 1) outputs an output lattice (output images ‘108’ in figure 1, see applicant’s specification ¶ 12 wherein the output lattice is an output image that has undergone color transformation) corresponding to the first LUT (received/retrieved LUT))
and performing color manipulation on at least one input image using the output lattice. (See ¶ 35 – 39, 28 - 31 wherein the colorization logic uses the output images (output lattice) as training for a machine learning model that takes in input images and performs color transformation/manipulation on the input image).
Regarding dependent claim 2, Landy teaches:
The input lattice comprises a regularly spaced grid, wherein the output lattice comprises the first LUT, (See ¶ 20, 28 – 30, wherein the input lattice is an input image, ‘104’ in figure 1, (the input image necessarily being made up of a regularly spaced grid of pixels organized in regularly spaced out columns and rows) wherein the output lattice comprises the first LUT (generated LUTs output and displayed in GUI as three dimensional cubes))
and wherein the performing of the color manipulation on the at least one input image comprises applying the first LUT, obtained from the trained machine learning model, to the at least one input image. (See ¶ 35, 36, 28 wherein the machine learning model is trained on the plurality of generated LUTs (including the first LUT) which is then used on the input image to perform the color manipulation).
Regarding dependent claim 3, Landy teaches
The input lattice comprises the at least one input image, wherein the output lattice comprises at least one color-manipulated image based on the first LUT, the at least one color-manipulated image corresponding to the at least one input image, (See ¶ 28, 44, 45, 31, 35, 19, Figure 1 wherein the input lattice is an input image, ‘104’ in figure 1, and wherein the output lattice comprises a color manipulated/adjusted image, ‘108’ in figure 1, based on the first LUT (received/retrieved LUT) corresponding to the input image)
and wherein the performing of the color manipulation on the at least one input image comprises providing the at least one color-manipulated image obtained from the trained machine learning model. (See ¶ 35 – 39, 44 – 45, and 28 wherein a machine learning model is trained on the generated output images (provided one color-manipulated image) and then the model is used to perform the color manipulation on the input image).
Regarding dependent claim 10, Landy teaches:
Providing, to the trained machine learning model, a second identifier and the input lattice, the second identifier indicating the first LUT and a second LUT of the plurality of LUTs; (See ¶ 34 - 35 wherein a plurality of LUTs (including a first and second LUT) are provided to the colorization logic (machine learning model) wherein an identifier is used to indicate the plurality of LUTs (including the first and second LUT) wherein the identifier is the specific television series the LUTs belong to)
and obtaining, from the trained machine learning model, another output lattice corresponding to a combination of the first LUT and the second LUT. (See ¶ 34 – 35 wherein the movie/series comprising updated/adjusted output lattices (adjusted frame images that make up a series/movie) and are obtained based on the plurality of LUTs (including the first and second LUTs) used in combination to train the machine learning model).
Regarding independent claim 11, claim 11 is an apparatus claim corresponding to claim 1. Please see the discussion of claim 1 above. Furthermore, Landy teaches of an apparatus for encoding color lookup tables comprising a memory storing instructions; and one or more processors coupled to the memory configured to execute the instructions. (See ¶ 40 – 45, wherein a device/apparatus comprises a processor that is used to execute instructions stored in memory for encoding color lookup tables. Please see the discussion of claim 1 above for further explanation).
Regarding dependent claim 12, claim 12 is an apparatus claim corresponding to claim 2. Please see the discussion of claim 2 above. Furthermore, Landy teaches of an apparatus for encoding color lookup tables comprising a memory storing instructions; and one or more processors coupled to the memory configured to execute the instructions. (See ¶ 40 – 45, wherein a device/apparatus comprises a processor that is used to execute instructions stored in memory for encoding color lookup tables. Please see the discussion of claim 2 above for further explanation).
Regarding dependent claim 13, claim 13 is an apparatus claim corresponding to claim 3. Please see the discussion of claim 3 above. Furthermore, Landy teaches of an apparatus for encoding color lookup tables comprising a memory storing instructions; and one or more processors coupled to the memory configured to execute the instructions. (See ¶ 40 – 45, wherein a device/apparatus comprises a processor that is used to execute instructions stored in memory for encoding color lookup tables. Please see the discussion of claim 3 above for further explanation).
Regarding independent claim 19, claim 19 is a non-transitory computer-readable storage medium claim corresponding to claim 1. Please see the discussion of claim 1 above. Furthermore, Landy teaches of a non-transitory computer-readable storage medium storing instructions for encoding color lookup tables by an apparatus that is executed by a processor (See ¶ 40 – 45, wherein a non-transitory computer-readable medium storage device is disclosed that stores instructions executed by a processor to encode color lookup tables. Furthermore, please see claim 1 above for further explanation).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Landy; Harvey et al. (US 20220311981 A1; hereinafter simply referred to as Landy) in view of Conde; Marcos et al. (NILUT: Conditional Neural Implicit 3D Lookup Tables for Image Enhancement).
Regarding dependent claim 4, Landy does not explicitly disclose:
The input lattice comprises a Hald image at a first resolution level, and wherein the output lattice comprises the first LUT at the first resolution level.
However, Conde teaches of the input lattice comprises a Hald image at a first resolution level, and wherein the output lattice comprises the first LUT at the first resolution level. (See Page 7 Left Column Paragraph 1, Page 4 Left Column Paragraph 3, Figure 8, and Figure 4, wherein the machine learning model is trained on the RGB maps (input lattice Hald image) wherein the LUTs are learned (output) wherein the resolution of the output LUT and input Hald image are the same as only the color is manipulated).
As taught by Conde the NILUT model they introduce using input Hald images allows for multiple styles to be encoded into a single network. (See Page 1 right column Contribution Section, wherein the NILUT model allows for the encoding of multiple styles into a single network.) As both the teachings of Landy and Conde deal with the technical field of image processing regarding LUTs for color manipulations, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Landy with Conde to teach of the input lattice comprises a Hald image at a first resolution level, and wherein the output lattice comprises the first LUT at the first resolution level in order to allow for multiple styles to be encoded into a single network
Regarding dependent claim 14, claim 14 is an apparatus claim corresponding to claim 4. Please see the discussion of claim 4 above. Furthermore, Landy teaches of an apparatus for encoding color lookup tables comprising a memory storing instructions; and one or more processors coupled to the memory configured to execute the instructions. (See ¶ 40 – 45, wherein a device/apparatus comprises a processor that is used to execute instructions stored in memory for encoding color lookup tables. Please see the discussion of claim 4 above for further explanation).
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Landy; Harvey et al. (US 20220311981 A1; hereinafter simply referred to as Landy) in view of Batur; Aziz et al. (US 20150302561 A1; hereinafter simply referred to as Batur).
Regarding dependent claim 5, Landy teaches:
A first resolution level of the output lattice matches a second resolution level of the input lattice (See ¶ 23- 24, wherein the resolutions of the output lattice and input lattice is unchanged and kept at high resolutions, wherein the input lattice and output lattice are the input images ‘104’ in figure 1, and output images ‘108’, in figure 1 respectively).
Landy does not explicitly disclose the first resolution level of the output lattice is different from a third resolution level of the first LUT.
However, Batur disclose of the first resolution level of the output lattice is different from a third resolution level of the first LUT. (See ¶ 28 wherein the LUT (first LUT) and the output lattice (output composite image) have similar resolutions, wherein similar does not equate to the same and therefore the two resolutions of the LUT and the output composite image have resolutions that are at least somewhat different).
As taught by Batur the first resolution level of the output lattice being different from a third resolution level of the first LUT allows for in each entry of the LUT for there to be coordinates from where to fetch the input pixels. (See ¶ 28 wherein coordinates are given for each entry of the LUT wherein the coordinates indicated where an input pixel can be fetched to generate the output pixel). As both the teachings of Landy and Batur deal with the technical field of image processing regarding the use of lookup tables, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Landy with Batur to teach of the first resolution level of the output lattice being different from a third resolution level of the first LUT in order to allow for in each entry of the LUT for there to be coordinates from where to fetch the input pixels to generate the output pixels.
Regarding dependent claim 15, claim 15 is an apparatus claim corresponding to claim 5. Please see the discussion of claim 5 above. Furthermore, Landy teaches of an apparatus for encoding color lookup tables comprising a memory storing instructions; and one or more processors coupled to the memory configured to execute the instructions. (See ¶ 40 – 45, wherein a device/apparatus comprises a processor that is used to execute instructions stored in memory for encoding color lookup tables. Please see the discussion of claim 5 above for further explanation).
Allowable Subject Matter
Claims 6 – 9, 16 – 18 and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indications of allowable subject matter:
Regarding clams 6 and 20, the reason of allowable subject matter is that the prior art fails to teach or reasonably suggest the limitations of claims 1 and 19 respectively, further comprising selecting a plurality of random input colors from an input color space based on a predetermined distribution; normalizing the plurality of random input colors to a predetermined range of values; computing a plurality of target colors by applying the normalized plurality of random input colors to a machine learning model; normalizing the plurality of target colors to the input color space; determining a reconstruction error based on the plurality of target colors; and adjusting weights of the machine learning model based on the reconstruction error to obtain the trained machine learning model.
Regarding claim 16, the reason of allowable subject matter is that the prior art fails to teach or reasonably suggest the limitations of claim 11, further comprising select a plurality of random input colors from an input color space based on a predetermined distribution; normalize the plurality of random input colors to a predetermined range of values; compute a plurality of target colors by applying the normalized plurality of random input colors to a machine learning model; normalize the plurality of target colors to the input color space; determine a reconstruction error based on the plurality of target colors; and adjust weights of the machine learning model based on the reconstruction error to obtain the trained machine learning model, wherein the predetermined distribution comprises a uniform distribution of colors across the input color space.
Prior Art Made of Record
The prior art made of record and not relied upon is considered pertinent to Applicant’s disclosure and is as follows:
U.S. Patent Application No. US 20180288381 A1 (He) discloses the use of a lookup table (LUT) that is estimated by an encoder machine learning model that executes the color conversion of videos and images. (He [0041])
Furthermore please See attached PTO-892.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEJANDRO HERNANDEZ whose telephone number is (703)756-1876. The examiner can normally be reached M-F 10 am - 6 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John M Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEJANDRO HERNANDEZ/Examiner, Art Unit 2661
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661
Prosecution Insights