DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election without traverse of claims 1 – 11 and 17 – 20 in the reply filed on 25 February 2025 is acknowledged.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1 – 11 and 17 – 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gong et al. (US 2025/0336122) in view of Chen et al. (US 2025/0322570).
Regarding independent claim 1, Gong teaches <<does not teach>> a method (Figures 1 and 2) comprising:
obtaining an image generation prompt comprising a text to be displayed in a synthetic image (paragraph 21: the predictability of the output generated by the AI system can be improved by converting prompts into a standardized image prompt and using that standardized image prompt as input to the model that generates the visual effect; Figure 5: prompt containing word “FRUIT” 314 in visual effect 330);
generating<<, using a first image generation model,>> a first image feature based on the image generation prompt, wherein the first image feature represents the text (paragraph 65 and Figure 5: a pixelated version 506 of the display phrase in the chosen font is created in order to define boundaries within which an image may be generated); and
generating, using a second image generation model, the synthetic image based on the image generation prompt and the first image feature (paragraph 66: after the boundaries have been determined, an image or images are generated by the image generating model 320 subject to the constraint that they must fit within the boundaries of the pixelated version 506 of the display phrase 304), wherein the synthetic image includes the text (Figure 5: prompt containing word “FRUIT” 314 in visual effect 330).
Gong does not expressly disclose generating, using a first image generation model, a first image feature based on the image generation prompt, however Gong does disclose image generating models are models that have been trained, for example, on a large data set images each associated with a label or caption (paragraph 58). Chen discloses vision generative model of shape-adaptive generation model (first and second quadrants) and refinement model of the shape-adaptive generative model (third and fourth quadrants) (paragraph 38 and Figure 1B), as shown in the first quadrant (i.e., the upper left-hand comer), the font mask 153 has smooth edges around the character R in a rectangular canvas to define an irregular canvas to iteratively fill in the salient content (paragraph 39), and moving towards the third quadrant (i.e., the lower left-hand comer), the consistent font visual effect generation pipeline first applies the text-to-image model 126a in a depth-to-image process 163 to refine the edge details of the styled reference character image 155 into the refined image of the reference character 164 with a more natural appearance (e.g., some croissant edges naturally protruding outside the font shape) through regeneration (paragraph 43). It would have been obvious for one of ordinary skill in the art at the time of the invention (pre-AIA ) or at the time of the effective filing date of the application (AIA ) to achieve a predictable result of having a first image generation model generating a first image feature by modifying Gong's system that the process involving the image generation model 320 utilizes a generated pixelated/mask of the word in the text prompt by adding another similar image generation model that splits said process to be performed by the added image generation model based on the teaching of Chen that contains multiple models of the shape-adaptive generative models that utilize different model for generating a font mask and another different model for generating the styled reference character image, and the result would have been predictable.
Regarding dependent claim 2, Gong teaches generating, using a language generation model, a layout description based on the image generation prompt, wherein the first image feature is generated based on the layout description (paragraph 61: The input text 302 (including the display phrase 304 and additional information 306) may be placed in a template which is fed into the language model 310 to produce the candidate prompt 312).
Regarding dependent claim 3, Gong teaches generating a text mask based on the layout description, wherein the first image feature is generated based on the text mask (paragraph 66: after the boundaries have been determined, an image or images are generated by the image generating model 320 subject to the constraint that they must fit within the boundaries of the pixelated version 506 of the display phrase 304).
Regarding dependent claim 4, Gong teaches wherein obtaining the text comprises: extracting, using a language generation model, the text based on the image generation prompt (paragraph 61: The input text 302 (including the display phrase 304 and additional information 306) may be placed in a template which is fed into the language model 310 to produce the candidate prompt 312; paragraph 65: A pixelated version 506 of the display phrase in the chosen font is created in order to define boundaries within which an image may be generated).
Regarding dependent claim 5, Gong teaches generating, using a language generation model, a custom image generation prompt based on the image generation prompt (paragraph 61: The input text 302 (including the display phrase 304 and additional information 306) may be placed in a template which is fed into the language model 310 to produce the candidate prompt 312).
Regarding dependent claim 6, Gong teaches generating a plurality of layer-specific intermediate image features at a plurality of layers of the first image generation model, respectively; and providing the plurality of layer-specific intermediate image features to a plurality of layers of the second image generation model, respectively (paragraph 67: the image generator 320 may produce many images. These images are pre-generated in that they are generated before receipt of a particular text input 302, a particular candidate prompt 312, and a particular image prompt 314. An embedding of each of the images may also be determined prior to receiving the current image prompt 314. When the image prompt 314 is received, an embedding of the image prompt 314 is determined and a similarity is calculated between the embedding of the image prompt and the embedding of each of the pre-generated images. All of the pre-generated images which have a similarity greater than a threshold value may be returned as an image 322 for use with the visual effect. Alternatively, a specified number of pre-generated images, ranked by their similarity values with the image prompt 314, may be returned as an image 322 for use with the visual effect. By generating the images and their embeddings in advance, it will be possible to save resources by only calling upon the image generating model 320 once to produce a group of images and then using the similarity comparison with the embedding of the image prompt 314 to select fewer than all the images. In this instance one measure of the quality of the image 322 is the similarity calculated between the embedding of the image 322 and the embedding of the image prompt 314).
Regarding dependent claim 7, Gong teaches wherein generating the synthetic image comprises: generating, using the second image generation model, a second image feature; and adding the first image feature and the second image feature element-wise (paragraph 66: after the boundaries have been determined, an image or images are generated by the image generating model 320 subject to the constraint that they must fit within the boundaries of the pixelated version 506 of the display phrase 304).
Regarding dependent claim 8, Gong teaches obtaining a reference image and a bounding box indicating a region of the reference image, wherein the synthetic image depicts the reference image with the text in the region indicated by the bounding box (paragraph 65: An alternative way of thinking is to use the pixelated version 506 of the display phrase 304 as a mask with boundaries. The pixelated display phrase 506 uses patches of a certain pixel size to define a mask in the shape of the display phrase 304. The size of the patches may be varied as part of optimizing the generated image).
Regarding dependent claim 9, the combination of Gong’s and Chen’s systems teaches wherein: the first image feature is generated using a first diffusion process; and the synthetic image is generated using a second diffusion process (Gong, paragraph 40: an image generating model 320 may include a generative adversarial network, a variational autoencoder, an autoregressive model, a convolutional neural network, a large transformer model, flow-based models, diffusion models and other models for producing images from a text input; Chen, Figure 1: multiple models of the image-adaptive generation model).
Regarding dependent claim 10, Gong teaches wherein: the image generation prompt indicates a design category of the synthetic image (paragraph 63: The image prompt 314 is submitted to the image generating model 320 to produce an image 322. In this example, the image 322 generated is a stylized set of several daisy blossoms which is used to provide a visual effect to be applied to text, e.g., as a background image; paragraph 65: part of the text input 302 is the phrase "Fruit in the shape of the word 'FRUIT"').
Regarding dependent claim 11, the combination of Gong’s and Chen’s systems teaches wherein: the first image generation model is trained to generate text structure images; and the second image generation model is trained to generate text design images (Gong, paragraph 40: A model may be trained prior to use as an image generating model 320. The training data may include many images with captions or labels describing the image in a text format. The training of such a model may be quite resource intensive, but the use of the image generating model 320 after it has been trained is significantly less resource intensive. The constraints on use are then the number of images requested or the number of text phrases 302 as inputs; Chen, Figure 1: multiple models of the image-adaptive generation model).
Regarding claims 21 - 25, claims 21 - 25 are similar in scope as to claims 1 - 4 and 6, thus the rejections for claims 1 - 4 and 6 hereinabove are applicable to claims 21 - 25. Gong teaches a non-transitory computer readable medium storing code for image processing, the code comprising instructions that, when executed by at least one processor, cause the at least one processor to perform operations (paragraphs 69 and 70).
Regarding independent claim 17, Gong teaches <<does not teach>> an apparatus (Figures 1 and 6) comprising: at least one processor; at least one memory storing instructions executable by the at least one processor (paragraphs 69 and 70);
<<a first image generation model>> comprising parameters stored in the at least one memory (paragraph 76: The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources) and trained (paragraph 40: A model may be trained prior to use as an image generating model 320. The training data may include many images with captions or labels describing the image in a text format) to generate a first image feature based on an image generation prompt comprising a text to be displayed in a synthetic image (paragraph 21: the predictability of the output generated by the AI system can be improved by converting prompts into a standardized image prompt and using that standardized image prompt as input to the model that generates the visual effect; Figure 5: prompt containing word “FRUIT” 314 in visual effect 330), wherein the first image feature represents the text (paragraph 65 and Figure 5: a pixelated version 506 of the display phrase in the chosen font is created in order to define boundaries within which an image may be generated); and
a second image generation model comprising parameters stored in the at least one memory (paragraph 76: The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources) and trained (paragraph 40: A model may be trained prior to use as an image generating model 320. The training data may include many images with captions or labels describing the image in a text format) to generate the synthetic image based on the image generation prompt and the first image feature (paragraph 66: after the boundaries have been determined, an image or images are generated by the image generating model 320 subject to the constraint that they must fit within the boundaries of the pixelated version 506 of the display phrase 304), wherein the synthetic image includes the text (Figure 5: prompt containing word “FRUIT” 314 in visual effect 330).
Gong does not expressly disclose a first image generation model to generate a first image feature based on an image generation prompt comprising a text to be displayed in a synthetic image, however Gong does disclose image generating models are models that have been trained, for example, on a large data set images each associated with a label or caption (paragraph 58). Chen discloses vision generative model of shape-adaptive generation model (first and second quadrants) and refinement model of the shape-adaptive generative model (third and fourth quadrants) (paragraph 38 and Figure 1B), as shown in the first quadrant (i.e., the upper left-hand comer), the font mask 153 has smooth edges around the character R in a rectangular canvas to define an irregular canvas to iteratively fill in the salient content (paragraph 39), and moving towards the third quadrant (i.e., the lower left-hand comer), the consistent font visual effect generation pipeline first applies the text-to-image model 126a in a depth-to-image process 163 to refine the edge details of the styled reference character image 155 into the refined image of the reference character 164 with a more natural appearance (e.g., some croissant edges naturally protruding outside the font shape) through regeneration (paragraph 43). It would have been obvious for one of ordinary skill in the art at the time of the invention (pre-AIA ) or at the time of the effective filing date of the application (AIA ) to achieve a predictable result of having a first image generation model generating a first image feature by modifying Gong's system that the process involving the image generation model 320 utilizes a generated pixelated/mask of the word in the text prompt by adding another similar image generation model that splits said process to be performed by the added image generation model based on the teaching of Chen that contains multiple models of the shape-adaptive generative models that utilize different model for generating a font mask and another different model for generating the styled reference character image, and the result would have been predictable.
Regarding dependent claim 18, Gong teaches a language generation model configured to generate the text, a layout description, or a custom image generation prompt (paragraph 61: The input text 302 (including the display phrase 304 and additional information 306) may be placed in a template which is fed into the language model 310 to produce the candidate prompt 312).
Regarding dependent claim 19, Gong teaches a layout component configured to generate a layout based on a layout description (paragraph 66: after the boundaries have been determined, an image or images are generated by the image generating model 320 subject to the constraint that they must fit within the boundaries of the pixelated version 506 of the display phrase 304).
Regarding dependent claim 20, the combination of Gong’s and Chen’s systems teaches wherein: the first image generation model comprises a first diffusion model; and the second image generation model comprises a second diffusion model (Gong, paragraph 40: an image generating model 320 may include a generative adversarial network, a variational autoencoder, an autoregressive model, a convolutional neural network, a large transformer model, flow-based models, diffusion models and other models for producing images from a text input; Chen, Figure 1: multiple models of the image-adaptive generation model).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEFFREY J CHOW whose telephone number is (571)272-8078. The examiner can normally be reached 11AM-7PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona Faulk can be reached at 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEFFREY J CHOW/Primary Examiner, Art Unit 2618
Prosecution Insights