DETAILED ACTION
This office action is in response to applicant’s amendment/response filed
09/19/2025, which has been entered and made of record. Claims 1-8 and 10-18 are currently
pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
With respect to the objection to claims 9 and 11 due to minor grammatical informalities, claim 9 has been cancelled and claim 11 has been amended to correct the objected issue. Therefore, the objection to claims 9 and 11 has been withdrawn.
With respect to the rejection of claim 10 under 35 U.S.C. § 112(b), claim 10 has been amended to resolve the antecedent basis issue in the original claim. Therefore, the rejection of claim 10 has been withdrawn.
With respect to the provisional rejection of claims 1, 5-9, 13, 17, and 19 on the ground of nonstatutory double patenting, both the current claims and the claims of application 18/483,227 have since been amended such that the differences between the current claims and the claims of application 18/483,227 are more than trivial. Therefore, the provisional rejection of claims 1, 5-9, 13, 17, and 19 on the ground of nonstatutory double patenting has been withdrawn.
Applicant’s arguments, filed 09/19/2025, with respect to the rejection(s) of claim(s) 1, 11, and 13 under 35 U.S.C. § 103 have been fully considered and are persuasive. Applicant describes how independent claims 1, 11, and 13 have been amended to require an input image that contains both color representation and a line drawing. Primary reference “Mou” teaches both color palette and line drawing inputs, but not within the same image, and the separate inputs are processed before being combined. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Chugh et al. (US 20250117978 A1, hereinafter "Chugh"). Chugh teaches a diffusion-based method of generating an image based on a single input image, which includes a line drawing and color “hints” added to the line drawing.
Since the new grounds of rejection are necessitated by the applicant’s amendments to the claims, the present action is made final.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5-8, 11, 13, 17, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mou et al. (“T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models”. arXiv preprint (20 Mar 2023). https://arxiv.org/abs/2302.08453; hereinafter “Mou”) in view of Chugh (US 20250117978 A1) and Once Upon an Algorithm (“Guide: What Denoising Strength Does and How to Use It in Stable Diffusion”).
Regarding claim 1, Mou discloses an apparatus comprising: at least one processor assembly configured to:
input (i) a color palette representation (figs. 1, 8, and 10 show color palette input along with text and optionally other guidance, pg. 4 section 3.3 “In this paper, we design a spatial color palette to roughly control the hue and color distribution of the generated images.”) and at least one line representing an object (fig. 10, sketch drawing and color palette inputs are combined to produce an output with the shape of the sketch and the coloration of the palette) to a stable diffusion (SD) model (pg. 3 section 3.1 “In this paper, we implement our method based on the recent text-to-image diffusion model (i.e., Stable Diffusion (SD) [32])”, section 1 explains how the model discussed in the paper is an adapter built on top of Stable Diffusion and similar models);
input text to the SD model (figs. 1, 8, and 10 show text input along with other guidance); and
present an image output by the SD model responsive to the input text conforming to the input color palette (figs. 1, 8, and 10 show image output based on the text and palette input, fig. 1 caption “We propose T2I-Adapter, a simple and small model that can provide extra guidance to pre-trained text-to-image (T2I) models while not affecting their original network topology and generation ability… Various guidance such as color, depth, sketch, semantic segmentation, and keypose can be used.”).
Mou does not explicitly teach a single input image that includes both a color palette representation and at least one line representing an object.
Chugh teaches an input image that includes a color palette representation and at least one line representing an object (fig. 3 shows an example color hint 310 being added directly to the input line drawing image, where the color hints are selected from, and representative of, color palette 320; [0002] “Embodiments of the present disclosure include a colorization apparatus configured to generate a synthesized image based on an outline image and color hints. Color hints are color additions provided by a user on top of the outline image. Embodiments include an outline encoder configured to encode the outline image and the color hints to produce a conditional embedding that an image generator uses as a basis for generating the synthesized image.”).
Mou and Chugh are both analogous to the claimed invention because they are in the same field of image generation using diffusion models. Both accept image input in the form of colors and line drawings, as well as text input. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Mou to incorporate the teachings of Chugh to accept both line drawing and color palette input in a single image. The motivation would have been to improve the ease of use/convenience for a user.
The combination of Mou in view of Chugh does not explicitly disclose that the input image is input with (ii) a strength parameter to the Stable Diffusion model.
Once Upon an Algorithm teaches an image being input with a strength parameter to Stable Diffusion (“When a user asks Stable Diffusion to generate an output from an input image, whether that is through image-to-image (img2img) or InPaint, it initiates this process by adding noise to that input based on a seed. The amount of noise it adds is controlled by Denoising Strength, which can be a minimum of 0 and a maximum of 1… The practical implication of this setting is that Denoising Strength helps to determine how closely your output image will be influenced by your input image”).
Once Upon an Algorithm teaches that denoising strength is a standard parameter for inputting an image to a diffusion model. Furthermore, Once Upon an Algorithm and the combination of Mou in view of Chugh are both analogous to the claimed invention because they are in the same field of image generation using Stable Diffusion-based models. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the generative model of Mou in view of Chugh to incorporate the teachings of Once Upon an Algorithm to add the option to accept the denoising strength as an input parameter. The motivation would have been to allow the user to adjust how closely the output image resembles the input image.
Regarding claim 5, the combination of Mou in view of Chugh and Once Upon an Algorithm teaches the apparatus of Claim 1, wherein the color palette representation has a resolution of at least four pixels (Mou figs. 1, 5, and 8, input color palette has an 8x8 resolution).
Regarding claim 6, the combination of Mou in view of Chugh and Once Upon an Algorithm teaches the apparatus of Claim 1, wherein the color palette representation has a resolution of at least eight pixels by eight pixels (Mou figs. 1, 5, and 8, input color palette has an 8x8 resolution).
Regarding claim 7, the combination of Mou in view of Chugh and Once Upon an Algorithm teaches the apparatus of Claim 6, wherein the color palette representation has a resolution of no more than one hundred twenty eight by one hundred twenty eight (128x128) pixels (Mou figs. 1 and 8, input color palette has an 8x8 resolution).
Regarding claim 8, the combination of Mou in view of Chugh and Once Upon an Algorithm teaches the apparatus of Claim 1.
The combination of Mou in view of Chugh and Once Upon an Algorithm does not explicitly teach that the strength parameter is at least 0.9. However, Once Upon an Algorithm teaches that “higher Denoising Strength increases variation and reduces the influence of your input image on your output image, which makes high values useful for significant modifications.” One of ordinary skill in the art would have reasonably come to the conclusion that if the input image is a line drawing without proper coloration, then “significant modification” would likely be required to generate an acceptable output image, and it would have been obvious to set the denoising strength to a high value, including trying values in the highest decile of 0.9 to 1. Therefore, it would have been obvious to one of ordinary skill in the art to modify the invention of Mou in view of Chugh and Once Upon an Algorithm to obtain the invention as specified in claim 8.
Regarding claim 11, it is rejected using the same references, rationales, and motivations to combine described in the rejection of claims 1 and 8, since it reiterates elements from claims 1 and 8 in the form of a method.
Regarding claims 13 and 18, they are rejected using the same references, rationales, and motivations to combine described in the rejections of claims 1 and 8 respectively, with the additional limitation of at least one computer medium that is not a transitory signal and that comprises instructions executable by at least one processor assembly (implied by Mou pg. 2 “They are lightweight with ∼ 77 M parameters and ∼ 300 M storage space”).
Regarding claim 17, it is rejected using the same references, rationales, and motivations to combine described in the rejections of claims 5 and 7, with the additional limitation of at least one computer medium that is not a transitory signal and that comprises instructions executable by at least one processor assembly (implied by Mou pg. 2 “They are lightweight with ∼ 77 M parameters and ∼ 300 M storage space”).
Claim(s) 2, 4, 14, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mou (“T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models”) in view of Chugh (US 20250117978 A1) and Once Upon an Algorithm (“Guide: What Denoising Strength Does and How to Use It in Stable Diffusion”) as applied to claims 1 and 13 above, and further in view of TouchDesigner ("Random Pixel Sampling").
Regarding claim 2, the combination of Mou in view of Chugh and Once Upon an Algorithm teaches the apparatus of Claim 1, wherein the processor assembly is configured to: establish the input color palette representation at least in part by converting a color palette into a smaller pixelated patch color palette representation (Mou section 3.3 “The original condition input has the resolution of 512 × 512”, “Spatial color palette” section describes how 64x downsampling of the original input is used as part of the process of generating an input color palette, resulting in an 8x8 palette image).
The combination of Mou in view of Chugh and Once Upon an Algorithm does not explicitly teach converting a blocked color palette into a smaller pixelated patch color palette representation.
TouchDesigner teaches converting a blocked color palette (input is a 1x50 image sorted by color) into a smaller pixelated palette (output is 1x10 and randomized).
TouchDesigner and the combination of Mou in view of Chugh and Once Upon an Algorithm are both analogous to the claimed invention because they are in the same field of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Mou in view of Chugh and Once Upon an Algorithm to incorporate the teachings of TouchDesigner to add the ability to convert a blocked color palette into a smaller, randomized palette. One of ordinary skill in the art would have found it obvious to adapt the teachings of TouchDesigner to match the required input resolutions of Mou, converting a 512x512 input palette to 8x8 rather than converting a 1x50 input palette to 1x10. The motivation would have been to make it easier for a user to input their own customized color palette image, rather than generating it from a preexisting image as taught by Mou, while still maintaining the appearance of randomness of a palette derived from a preexisting image. The user could quickly draw large solid blocks of color, and the randomization would be performed automatically.
Regarding claim 4, the combination of Mou in view of Chugh and Once Upon an Algorithm and TouchDesigner teaches the apparatus of claim 2, wherein the processor assembly is configured to: convert the blocked color palette into the color palette representation at least in part by sampling a random pixel in the blocked color palette for every pixel in the color palette representation being created to create a random patch of colors with no spatial dependencies (TouchDesigner “First Input is your 1x50 image (in the example of the screenshot a ramp with some colors) and the second input a 1x10 Noise TOP set to random… The Remap TOP will use the value from the pixels in the second input as a lookup into the first input returning in this instance a random selection of points”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Mou in view of Chugh and Once Upon an Algorithm to incorporate the teachings of TouchDesigner to add a method of implementing the color palette conversion described previously.
Regarding claims 14 and 16, they are rejected using the same references, rationales, and motivations to combine described in the rejections of claims 2 and 4, with the additional limitation of at least one computer medium that is not a transitory signal and that comprises instructions executable by at least one processor assembly (implied by Mou pg. 2 “They are lightweight with ∼ 77 M parameters and ∼ 300 M storage space”).
Claim(s) 3 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mou (“T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models”) in view of Chugh (US 20250117978 A1) and Once Upon an Algorithm (“Guide: What Denoising Strength Does and How to Use It in Stable Diffusion”) as applied to claims 1 and 13 above, and further in view of ControlNet T2I-Adapter Models (hereinafter "ControlNet").
Regarding claim 3, the combination of Mou in view of Chugh and Once Upon an Algorithm teaches the apparatus of claim 1.
The combination of Mou in view of Chugh and Once Upon an Algorithm does not teach wherein the color palette representation comprises a 16x16 pixelated “patch” color palette.
ControlNet teaches wherein the color palette representation comprises a 16x16 pixelated “patch” color palette (first attached image).
ControlNet and the combination of Mou in view of Chugh and Once Upon an Algorithm are both analogous to the claimed invention because they are in the same field of image generation using Stable Diffusion-based models. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Mou in view of Chugh and Once Upon an Algorithm to incorporate the teachings of ControlNet to double the size of the color palette in order to increase the level of detail in the output image.
In addition to the example taught by ControlNet, it would have been obvious to one of ordinary skill in the art to modify the invention of Mou in view of Chugh and Once Upon an Algorithm to increase the level of detail in the input images by doubling the standardized input resolution from 512x512 to 1024x1024. Mou teaches that its 8x8 color palettes are the result of 64x downsampling of 512x512 input images (section 3.3 “The original condition input has the resolution of 512 × 512”, “Spatial color palette” section describes how 64x downsampling of the original input is used as part of the process of generating an input color palette). If one of ordinary skill in the art followed the same procedure taught by Mou to generate a color palette using a 1024x1024 input image, the resulting palette would be 16x16.
Regarding claim 15, it is rejected using the same references, rationales, and motivations to combine described in the rejections of claim 3, with the additional limitation of at least one computer medium that is not a transitory signal and that comprises instructions executable by at least one processor assembly (implied by Mou pg. 2 “They are lightweight with ∼ 77 M parameters and ∼ 300 M storage space”).
Claim(s) 10 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mou (“T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models”) in view of Chugh (US 20250117978 A1) and Once Upon an Algorithm (“Guide: What Denoising Strength Does and How to Use It in Stable Diffusion”) as applied to claims 1 and 11 above, and further in view of Wang et al. (CN 116824004 A, hereinafter "Wang").
Regarding claim 10, the combination of Mou in view of Chugh and Once Upon an Algorithm teaches the apparatus of Claim 1.
The combination of Mou in view of Chugh and Once Upon an Algorithm does not explicitly teach wherein the presented image comprises an emblem.
Wang teaches a method of generating icons based on Stable Diffusion ([n0043] “The sample icons of the same style and their corresponding prompts are combined into
sample data, and the sample data is used to train a low-rank adaptation model (LoRA) to obtain an icon generation model. Among them, the LoRA model can be regarded as a model that locally integrates (fine-tuned) the diffusion model, with small storage (storage), and the LoRA model can be customized and can obtain the characteristics of the specified object data through learning. The LoRA model is trained using sample data consisting of sample icons and prompts, and an icon generation model can eventually be obtained. For example, the LoRA model can be trained based on Stable-diffusionv1.5”)
Wang and the combination of Mou in view of Chugh and Once Upon an Algorithm are both analogous to the claimed invention because they are in the same field of image generation using Stable Diffusion-based models. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Mou in view of Chugh and Once Upon an Algorithm with the teachings of Wang to apply it towards generating emblems, in order for the invention to be marketed toward graphic design and UI design.
Regarding claim 12, it is rejected using the same references, rationales, and motivations to combine described in the rejection of claim 10, since it reiterates elements from claim 10 in the form of a method.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENJAMIN STATZ whose telephone number is (571)272-6654. The examiner can normally be reached Mon-Fri 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571)272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BENJAMIN TOM STATZ/Examiner, Art Unit 2611
/TAMMY PAIGE GODDARD/Supervisory Patent Examiner, Art Unit 2611