DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 12/22/2025 has been entered. Claims 1-20 remain pending in the application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-6 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Saharia U.S. Patent Application 20230377226 in view of Serrano U.S. Patent Application 20100067793, and further in view of Hsu U.S. Patent 8762836.
Regarding claim 1, Saharia discloses a method for image generation, comprising:
obtaining a prompt that includes a description text (paragraph [0064]: the system 100 can receive a text prompt (custom-character) 102 describing a scene. The text prompt 102 can be a text sequence that includes multiple text tokens);
encoding the prompt to obtain a prompt encoding (paragraph [0065]: The text encoder 110 is configured to process the text prompt 102 to generate a set of contextual embeddings (u) of the text prompt 102); and
generating, using an image generation network, an image that includes the text with the typographic characteristic based on the prompt encoding (paragraph [0067] The system 100 processes the contextual embeddings 104 through the sequence 121 to generate an output image 106 at a high resolution... provided by any GNN (generative neural networks) 120 in the sequence 121; see fig. 1A and fig. 7, fig. 7 image 108-1 is generated with the text prompt 102-1: “Sprouts in the shape of text ‘Imagen’ coming out of a fairytale book”),
wherein the image generation network is trained to use a denoising process to generate images having the typographic characteristics using training data (paragraph [0027]: each GNN (generative neural networks) in the sequence can be independently optimized by the training engine to impart certain properties to the GNN, e.g., particular output resolutions, fidelity, perceptual quality, efficient decoding (or denoising), fast sampling, reduced artifacts, etc.; fig. 7 image 108-1 is generated with the text prompt 102-1: “Sprouts in the shape of text ‘Imagen’ coming out of a fairytale book”).
Saharia discloses all the features with respect to claim 1 as outlined above. However, Saharia fails to disclose a prompt that includes a description of a typographic characteristic of text explicitly, training data including typographic information generated for a training image, and wherein the typographic information includes a plurality of typographic characteristics including the typographic characteristic of the text.
Serrano discloses training data including typographic information generated for a training image, and wherein the typographic information includes a plurality of typographic characteristics including the typographic characteristic of the text (paragraph [0038]: a set of computer-generated images (training examples) 14 of the query string is automatically rendered using different computer typographic fonts (typographic information); paragraph [0095]: An improvement in mAP can be achieved when a set of font types are used to train the model; paragraph [0063]: various combinations of font types are used, to identify a group of fonts which yield optimum results).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia’s to generate font information as taught by Serrano, to recognize handwritten words in document images more efficiently.
Saharia as modified by Serrano discloses all the features with respect to claim 1 as outlined above. However, Saharia as modified by Serrano fails to disclose a prompt that includes a description of a typographic characteristic of text explicitly.
Hsu discloses a prompt that includes a description of a typographic characteristic of text (col. 6 line 51-61: In step 302, a design program user manually inputs a target font definition which is accepted by the computer system 200. The user can manually input a target font by entering font properties for a target font in a prompt provided in the design program interface... a user can use any font description language to describe a target font having any arbitrary weight, style, or stretch; Hsu’s prompt for text typographic characteristic can be used in Saharia and Serrano’s device, such that to obtain a prompt including a description of a typographic characteristic of text).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia and Serrano’s to input font via a prompt as taught by Hsu, to allow users to apply design font to text in the design.
Regarding claim 2, Saharia as modified by Serrano and Hsu discloses the method of claim 1, wherein generating the image further comprises:
obtaining a noise image (Saharia’s paragraph [0004]: a noise input sampled from a noise distribution, a pre-existing image, an embedding of a pre-existing image, a video, an embedding of a video, a numeric representation of a desired object category for the image); and
removing noise from the noise image based on the prompt encoding to obtain the image (Saharia’s paragraph [0027]: each GNN (generative neural networks) in the sequence can be independently optimized by the training engine to impart certain properties to the GNN, e.g., particular output resolutions, fidelity, perceptual quality, efficient decoding (or denoising), fast sampling, reduced artifacts, etc.; paragraph [0093]: the GNN 120 can use a fixed signal-to-noise ratio (e.g., about 3 to 5), representing a small amount of augmentation, which aids in removing artifacts in the output image).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia’s to generate font information as taught by Serrano, to recognize handwritten words in document images more efficiently; and combine Saharia and Serrano’s to input font via a prompt as taught by Hsu, to allow users to apply design font to text in the design.
Regarding claim 3, Saharia as modified by Serrano and Hsu discloses the method of claim 1, wherein: the typographic characteristic comprises at least one of a font, a text size, a text justification, and a color (Hsu’s col. 6 line 51-61: In step 302, a design program user manually inputs a target font definition which is accepted by the computer system 200. The user can manually input a target font by entering font properties for a target font in a prompt provided in the design program interface... a user can use any font description language to describe a target font having any arbitrary weight, style, or stretch).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia’s to generate font information as taught by Serrano, to recognize handwritten words in document images more efficiently; and combine Saharia and Serrano’s to input font via a prompt as taught by Hsu, to allow users to apply design font to text in the design.
Regarding claim 4, Saharia as modified by Serrano and Hsu discloses the method of claim 1, wherein: the prompt comprises a visual description of the image and a description of a location of the text within the image (Saharia’s fig. 7 image 108-1 is generated with the text prompt 102-1: “Sprouts in the shape of text ‘Imagen’ coming out of a fairytale book”).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia’s to generate font information as taught by Serrano, to recognize handwritten words in document images more efficiently; and combine Saharia and Serrano’s to input font via a prompt as taught by Hsu, to allow users to apply design font to text in the design.
Regarding claim 5, Saharia as modified by Serrano and Hsu discloses the method of claim 1, wherein:
the image generation network is trained using a training image and a training description of a text element of the training image (Saharia’s paragraph [0007]: the generative neural networks in the sequence have been trained jointly on a set of training examples that each include: (i) a respective training text prompt, and (ii) a respective ground truth image that depicts a scene described by the respective training text prompt; paragraph [0082]: provide the GNNs 120 a means of combining, mixing, and compressing information from different images such that the sequence 121 can generate new instances of images that are ostensibly unlike anything appearing in the training sets; fig. 7 image 108-1 is generated with the text prompt 102-1: “Sprouts in the shape of text ‘Imagen’ coming out of a fairytale book”).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia’s to generate font information as taught by Serrano, to recognize handwritten words in document images more efficiently; and combine Saharia and Serrano’s to input font via a prompt as taught by Hsu, to allow users to apply design font to text in the design.
Regarding claim 6, Saharia as modified by Serrano and Hsu discloses the method of claim 1, wherein:
the image generation network comprises a diffusion model conditioned on the prompt that includes the description of the typographic characteristic of the text (Saharia’s paragraph [0008]: each generative neural network in the sequence is a diffusion-based generative neural network; Hsu’s col. 6 line 51-61: In step 302, a design program user manually inputs a target font definition which is accepted by the computer system 200. The user can manually input a target font by entering font properties for a target font in a prompt provided in the design program interface... a user can use any font description language to describe a target font having any arbitrary weight, style, or stretch).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia’s to generate font information as taught by Serrano, to recognize handwritten words in document images more efficiently; and combine Saharia and Serrano’s to input font via a prompt as taught by Hsu, to allow users to apply design font to text in the design.
Regarding claim 13, Saharia discloses a system for image generation, comprising:
a user interface (graphical user interface) configured to obtain a prompt that includes a description of text (paragraph [0034]: Users can interact with the image generation system, e.g., by providing inputs to the image generation system by way of an interface, e.g., a graphical user interface, or an application programming interface (API). In particular, a user can provide an input that includes: (i) a request to generate an image, and (ii) a prompt (e.g., a text prompt) describing the contents of the image to be generated; paragraph [0064]: the system 100 can receive a text prompt (custom-character) 102 describing a scene. The text prompt 102 can be a text sequence that includes multiple text tokens);
an image generation network trained to use a denoising process to generate images having the typographic characteristics using training data (paragraph [0027]: each GNN (generative neural networks) in the sequence can be independently optimized by the training engine to impart certain properties to the GNN, e.g., particular output resolutions, fidelity, perceptual quality, efficient decoding (or denoising), fast sampling, reduced artifacts, etc.; paragraph [0067] The system 100 processes the contextual embeddings 104 through the sequence 121 to generate an output image 106 at a high resolution... provided by any GNN (generative neural networks) 120 in the sequence 121; fig. 7 image 108-1 is generated with the text prompt 102-1: “Sprouts in the shape of text ‘Imagen’ coming out of a fairytale book”),
wherein the mage generation network is trained using the training image and a training description comprising a text element of the training image (paragraph [0007]: the generative neural networks in the sequence have been trained jointly on a set of training examples that each include: (i) a respective training text prompt, and (ii) a respective ground truth image that depicts a scene described by the respective training text prompt; paragraph [0082]: provide the GNNs 120 a means of combining, mixing, and compressing information from different images such that the sequence 121 can generate new instances of images that are ostensibly unlike anything appearing in the training sets).
Saharia discloses all the features with respect to claim 13 as outlined above. However, Saharia fails to disclose processors and memory, a prompt that includes a description of a typographic characteristic of text explicitly, and parameters stored in the one or more memory components and trained to generate images having the typographic characteristic, training data including typographic information generated for a training image, and wherein the typographic information includes a plurality of typographic characteristics including the typographic characteristic of the text.
Serrano discloses one or more processors (processor 66);
one or more memory components (memory 64) coupled with the one or more processors (paragraph [0059]: A processor 66, such as a central processing unit, or separate processors for each component, in communication with the memory 64, executes the software instructions for performing analysis and markup of the document(s) 18); and
parameters stored in the one or more memory components and trained to generate images having the typographic characteristic (paragraph [0071]: Various types of models 16 can be trained for generating a representation of the extracted features of the training images, such as hidden Markov models (HMMs), support vector machines, neural networks, or the like; paragraph [0038]: At S104, given a typed query, multiple samples 14 are generated by varying only the font. Specifically, a set of computer-generated images (training examples) 14 of the query string is automatically rendered using different computer typographic fonts; paragraph [0042]: At S112, documents 18 which contain one or more of the matching samples may be labeled and output; paragraph [0077]: This step may be performed before the query word is input and the GMM parameters stored in memory to be used for all input query words),
training data including typographic information generated for a training image, and wherein the typographic information includes a plurality of typographic characteristics including the typographic characteristic of the text (paragraph [0038]: a set of computer-generated images (training examples) 14 of the query string is automatically rendered using different computer typographic fonts (typographic information); paragraph [0095]: An improvement in mAP can be achieved when a set of font types are used to train the model; paragraph [0063]: various combinations of font types are used, to identify a group of fonts which yield optimum results).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia’s to generate font information as taught by Serrano, to recognize handwritten words in document images more efficiently.
Saharia as modified by Serrano discloses all the features with respect to claim 13 as outlined above. However, Saharia as modified by Serrano fails to disclose a prompt that includes a description of a typographic characteristic of text explicitly.
Hsu discloses a prompt that includes a description of a typographic characteristic of text (col. 6 line 51-61: In step 302, a design program user manually inputs a target font definition which is accepted by the computer system 200. The user can manually input a target font by entering font properties for a target font in a prompt provided in the design program interface... a user can use any font description language to describe a target font having any arbitrary weight, style, or stretch; Hsu’s prompt for text typographic characteristic can be used in Saharia’s device, such that to obtain a prompt including a description of a typographic characteristic of text).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia and Serrano’s to input font via a prompt as taught by Hsu, to allow users to apply design font to text in the design.
Claim 7-9 are rejected under 35 U.S.C. 103 as being unpatentable over Saharia U.S. Patent Application 20230377226 in view of Serrano U.S. Patent Application 20100067793, and further in view of Hoffmann U.S. Patent Application 20230315532.
Regarding claim 7, Saharia discloses a method for image generation, comprising:
obtaining training data comprising a training image, a training description comprises a description of a characteristic of the training image (paragraph [0007]: the generative neural networks in the sequence have been trained jointly on a set of training examples that each include: (i) a respective training text prompt, and (ii) a respective ground truth image that depicts a scene described by the respective training text prompt; paragraph [0082]: provide the GNNs 120 a means of combining, mixing, and compressing information from different images such that the sequence 121 can generate new instances of images that are ostensibly unlike anything appearing in the training sets);
generating a synthetic image based on training description using a denoising process (paragraph [0027]: each GNN (generative neural networks) in the sequence can be independently optimized by the training engine to impart certain properties to the GNN, e.g., particular output resolutions, fidelity, perceptual quality, efficient decoding (or denoising), fast sampling, reduced artifacts, etc.; paragraph [0067] The system 100 processes the contextual embeddings 104 through the sequence 121 to generate an output image 106 at a high resolution... provided by any GNN (generative neural networks) 120 in the sequence 121; paragraph [0082]: provide the GNNs 120 a means of combining, mixing, and compressing information from different images such that the sequence 121 can generate new instances of images that are ostensibly unlike anything appearing in the training sets; fig. 7 image 108-1 is generated with the text prompt 102-1: “Sprouts in the shape of text ‘Imagen’ coming out of a fairytale book”);
training, based on the synthetic image, an image generation network to generate images having the typographic characteristics (paragraph [0027]: each GNN (generative neural networks) in the sequence can be independently optimized by the training engine to impart certain properties to the GNN, e.g., particular output resolutions, fidelity, perceptual quality, efficient decoding (or denoising), fast sampling, reduced artifacts, etc.; paragraph [0144]: Training engine jointly trains the generative neural networks on the respective set of contextual embeddings and ground truth output images of each training example (440); fig. 7 image 108-1 is generated with the text prompt 102-1: “Sprouts in the shape of text ‘Imagen’ coming out of a fairytale book”).
Saharia discloses all the features with respect to claim 7 as outlined above. However, Saharia fails to disclose training data comprising typographic information, wherein the typographic information includes a plurality of typographic characteristics; generating a training description based on the typographic information, wherein the training description comprises a description of a typographic characteristic of the training image from the plurality of typographic characteristics.
Serrano discloses training data comprising typographic information, wherein the typographic information includes a plurality of typographic characteristics (paragraph [0038]: a set of computer-generated images (training examples) 14 of the query string is automatically rendered using different computer typographic fonts; paragraph [0071]: Various types of models 16 can be trained for generating a representation of the extracted features of the training images, such as hidden Markov models (HMMs), support vector machines, neural networks, or the like);
a training description based on the typographic information, wherein the training description comprises a description of a typographic characteristic of the training image from the plurality of typographic characteristics (paragraph [0038]: a set of computer-generated images (training examples) 14 of the query string is automatically rendered using different computer typographic fonts (typographic information); paragraph [0095]: An improvement in mAP can be achieved when a set of font types (training description) are used to train the model; paragraph [0063]: various combinations of font types are used, to identify a group of fonts which yield optimum results).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia’s to generate font information as taught by Serrano, to recognize handwritten words in document images more efficiently.
Saharia as modified by Serrano discloses all the features with respect to claim 7 as outlined above. However, Saharia as modified by Serrano fails to disclose generating a training description explicitly.
Hoffmann discloses generating a training description (paragraph [0058]: the multimodal model may generate text representing, describing (e.g., captioning), or otherwise characterizing an image or audio input; paragraph [0108]: Training system obtains the target amount of training data for training the machine learning model (240)... retrieve the selected training data for use in training the machine learning model; Hoffmann’s teaching of generating training description can be used in Saharia and Serrano’s device, such as to generate training description based on the typographic information).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia and Serrano’s to use multimodal as taught by Hoffmann, to process data using machine learning models.
Regarding claim 8, Saharia as modified by Serrano discloses all the features with respect to claim 7 as outlined above. However, Saharia as modified by Serrano fails to disclose training description is generated using a multimodal text generation model based on the training image.
Hoffmann discloses training description is generated using a multimodal text generation model based on the training image (paragraph [0058]: the multimodal model may generate text representing, describing (e.g., captioning), or otherwise characterizing an image or audio input).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia and Serrano’s to use multimodal as taught by Hoffmann, to process data using machine learning models.
Regarding claim 9, Saharia as modified by Serrano and Hoffmann discloses the method of claim 8, wherein the training description is generated by generating an intermediate description using the multimodal text generation model and generating the training description using a text combination model based on the intermediate description (Hoffmann’s paragraph [0004]: a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers that each apply a non-linear transformation to a received input to generate an output; paragraph [0058]: the multimodal model may generate text representing, describing (e.g., captioning), or otherwise characterizing an image or audio input).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia and Serrano’s to use multimodal as taught by Hoffmann, to process data using machine learning models.
Claim 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Saharia U.S. Patent Application 20230377226 in view of Serrano U.S. Patent Application 20100067793, in view of Hoffmann U.S. Patent Application 20230315532, and further in view of Li U.S. Patent Application 20220391755.
Regarding claim 10, Saharia as modified by Serrano and Hoffmann discloses font and multimodal model (Serrano’s paragraph [0038]: a set of computer-generated images (training examples) 14 of the query string is automatically rendered using different computer typographic fonts; paragraph [0071]: Various types of models 16 can be trained for generating a representation of the extracted features of the training images, such as hidden Markov models (HMMs), support vector machines, neural networks, or the like; Hoffmann’s paragraph [0058]: the multimodal model may generate text representing, describing (e.g., captioning), or otherwise characterizing an image or audio input). However, Saharia as modified by Serrano and Hoffmann fails to disclose encoding text data to obtain a text encoding using a encoder, wherein the multimodal text generation model takes the encoding as an input.
Li discloses encoding text data to obtain a text encoding using a encoder, wherein the multimodal text generation model takes the encoding as an input (paragraph [0019]: The multimodal encoder is configured to receive the encoding of the image input and the encoding of the text input).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia, Serrano and Hoffmann’s to use multimodal encoder as taught by Li, to improve vision and language representation learning.
Regarding claim 11, Saharia as modified by Serrano, Hoffmann and Li discloses the method of claim 8, further comprising: encoding text data to obtain a text style encoding using a text style encoder, wherein the multimodal text generation model takes the text style encoding as an input (Li’s paragraph [0019]: The multimodal encoder is configured to receive the encoding of the image input and the encoding of the text input).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia, Serrano and Hoffmann’s to use multimodal encoder as taught by Li, to improve vision and language representation learning.
Claim 14-15 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Saharia U.S. Patent Application 20230377226 in view of Serrano U.S. Patent Application 20100067793, in view of Hsu U.S. Patent 8762836, and further in view of Hoffmann U.S. Patent Application 20230315532.
Regarding claim 14, Saharia as modified by Serrano, Hsu and Hoffmann discloses the system of claim 13, further comprising: a multimodal text generation model trained to generate the training description of the training image based on the training image (Hoffmann’s paragraph [0058]: the multimodal model may generate text representing, describing (e.g., captioning), or otherwise characterizing an image or audio input).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia, Serrano and Hsu’s to use multimodal as taught by Hoffmann, to process data using machine learning models.
Regarding claim 15, Saharia as modified by Serrano, Hsu and Hoffmann discloses the system of claim 13, further comprising: a text recognition component configured to perform text recognition on the training image to obtain text data, wherein the training description is generated based on the text data (Serrano’s paragraph [0057]: the tagged documents 20 may be automatically output for further processing, such as OCR recognition or the like; paragraph [0095]: a single image 14 was generated for each keyword in the selected font type; Hoffmann’s paragraph [0058]: the multimodal model may generate text representing, describing (e.g., captioning), or otherwise characterizing an image or audio input; Saharia’s paragraph [0007]: the generative neural networks in the sequence have been trained jointly on a set of training examples that each include: (i) a respective training text prompt, and (ii) a respective ground truth image that depicts a scene described by the respective training text prompt).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia, Serrano and Hsu’s to use multimodal as taught by Hoffmann, to process data using machine learning models.
Regarding claim 19, Saharia as modified by Serrano, Hsu and Hoffmann discloses the system of claim 13, further comprising: a text combination model configured to generate the training description based on a description of the training image and a description of the text element (Hoffmann’s paragraph [0058]: the multimodal model may generate text representing, describing (e.g., captioning), or otherwise characterizing an image or audio input; Saharia’s paragraph [0007]: the generative neural networks in the sequence have been trained jointly on a set of training examples that each include: (i) a respective training text prompt, and (ii) a respective ground truth image that depicts a scene described by the respective training text prompt).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia, Serrano and Hsu’s to use multimodal as taught by Hoffmann, to process data using machine learning models.
Claim 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Saharia U.S. Patent Application 20230377226 in view of Serrano U.S. Patent Application 20100067793, in view of Hsu U.S. Patent 8762836, in view of Hoffmann U.S. Patent Application 20230315532, and further in view of Li U.S. Patent Application 20220391755.
Regarding claim 16, Saharia as modified by Serrano, Hsu, Hoffmann and Li discloses the system of claim 15, further comprising: a font encoder configured to encode the text data to obtain a font encoding, wherein the training description is generated based on the font encoding (Li’s paragraph [0019]: The multimodal encoder is configured to receive the encoding of the image input and the encoding of the text input; Serrano’s paragraph [0038]: a set of computer-generated images (training examples) 14 of the query string is automatically rendered using different computer typographic fonts; Hoffmann’s paragraph [0058]: the multimodal model may generate text representing, describing (e.g., captioning), or otherwise characterizing an image or audio input).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia, Serrano, Hsu and Hoffmann’s to use multimodal encoder as taught by Li, to improve vision and language representation learning.
Regarding claim 17, Saharia as modified by Serrano, Hsu, Hoffmann and Li discloses the system of claim 15, further comprising: a text style encoder configured to encode the text data to obtain a text style encoding, wherein the training description is generated based on the text style encoding (Li’s paragraph [0019]: The multimodal encoder is configured to receive the encoding of the image input and the encoding of the text input; Hoffmann’s paragraph [0058]: the multimodal model may generate text representing, describing (e.g., captioning), or otherwise characterizing an image or audio input).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia, Serrano, Hsu and Hoffmann’s to use multimodal encoder as taught by Li, to improve vision and language representation learning.
Claim 18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Saharia U.S. Patent Application 20230377226 in view of Serrano U.S. Patent Application 20100067793, in view of Hsu U.S. Patent 8762836, and further in view of Kreis U.S. Patent Application 20240005604.
Regarding claim 20, Saharia as modified by Serrano and Hsu discloses all the features with respect to claim 13 as outlined above. However, Saharia as modified by Serrano and Hsu fails to disclose the image generation network is a text-guided diffusion model.
Kreis discloses the image generation network is a text-guided diffusion model (paragraph [0056]: the CLIP encodings obtained from a text encoder can be used to condition the latent diffusion models on text prompts).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia, Serrano and Hsu’s to use diffusion model as taught by Kreis, to adapted to various tasks such as multimodal voxel or text guided synthesis.
Regarding claim 18, Saharia as modified by Serrano, Hsu and Kreis discloses the system of claim 13, further comprising: an object detection component configured to detect an object included in the training image and to generate an object encoding based on the object, wherein the training description is generated based on the object encoding (Kreis’s paragraph [0150]: execute machine learning model(s) (for example, neural networks, such as CNNs) for segmentation, reconstruction, object detection, feature detection, classification, and/or other inferencing tasks; paragraph [0032]: feature points, representative of the object, encoded into a latent space; paragraph [0153]: generating a visualization, or generating text to summarize findings; Saharia’s paragraph [0007]: the generative neural networks in the sequence have been trained jointly on a set of training examples that each include: (i) a respective training text prompt, and (ii) a respective ground truth image that depicts a scene described by the respective training text prompt; paragraph [0082]: provide the GNNs 120 a means of combining, mixing, and compressing information from different images such that the sequence 121 can generate new instances of images that are ostensibly unlike anything appearing in the training sets).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Saharia, Serrano and Hsu’s to use diffusion model as taught by Kreis, to adapted to various tasks such as multimodal voxel or text guided synthesis.
Allowable Subject Matter
Claim 12 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Claim 12 is about computing a loss function for the image generation network based on the synthetic image, wherein the training is based on the loss function.
Saharia 20230377226, Serrano 20100067793, Hoffmann 20230315532, and Gopalkrishna 20230281963 combined cannot teach these features perfectly. These limitations when read in light of the rest of the limitations in the claim and the claims to which it depends make the claim allowable subject matter.
Response to Arguments
Applicant's arguments filed 12/22/2025, page 8 - 11, with respect to the rejection(s) of claim(s) 1 and 13 under 103, have been fully considered and are moot upon a new ground(s) of rejection made under 35 U.S.C. 103 as being unpatentable over Saharia U.S. Patent Application 20230377226 in view of Serrano U.S. Patent Application 20100067793, and further in view of Hsu U.S. Patent 8762836, as outlined above.
page 11 - 12, with respect to the rejection(s) of claim(s) 7 under 103, have been fully considered and are moot upon a new ground(s) of rejection made under 35 U.S.C. 103 as being unpatentable over Saharia U.S. Patent Application 20230377226 in view of Serrano U.S. Patent Application 20100067793, and further in view of Hoffmann U.S. Patent Application 20230315532, as outlined above.
Applicant argues on page 8-11 that Saharia and Hsu do not teach or suggest the image generation network is trained using training data including typographic information generated for a training image, wherein the typographic information includes a plurality of typographic characteristics including the typographic characteristic of the text.
In reply, the rejection is based on Saharia, Serrano and Hsu combined. Saharia discloses using both text and training image for image generation (paragraph [0144]: Training engine jointly trains the generative neural networks on the respective set of contextual embeddings and ground truth output images of each training example (440)).
Serrano discloses training data including typographic information generated for a training image, and wherein the typographic information includes a plurality of typographic characteristics including the typographic characteristic of the text (paragraph [0038]: a set of computer-generated images (training examples) 14 of the query string is automatically rendered using different computer typographic fonts (typographic information); paragraph [0095]: An improvement in mAP can be achieved when a set of font types are used to train the model; paragraph [0063]: various combinations of font types are used, to identify a group of fonts (generate font information) which yield optimum results). From cited prior art, we can conclude both the font type information and font images are used in training.
Applicant argues on page 11-12 that Saharia and Serrano do not teach or suggest obtaining training data comprising typographic information for a training image, wherein the typographic information includes a plurality of typographic characteristics, and generating a training description based on the typographic information, wherein the training description comprises a description of a typographic characteristic of the training image from the plurality of typographic characteristics as recited in claim 7.
In reply, this argument is explained in detail as discussion for claim 7 rejection (Saharia, Serrano and Hoffmann combined) above.
Hoffmann discloses generating a training description (paragraph [0058]: the multimodal model may generate text representing, describing (e.g., captioning), or otherwise characterizing an image or audio input; paragraph [0108]: Training system obtains the target amount of training data for training the machine learning model (240)... retrieve the selected training data for use in training the machine learning model; Hoffmann’s teaching of generating training description can be used in Saharia and Serrano’s device, such as to generate training description based on the typographic information).
Applicant argues on page 12-13 that There is no prima facie case that claim 1 is obvious in view of Saharia and Hsu, nor that claims 7 and 13 are obvious in view of Saharia and Serrano.
In reply, Saharia and Serrano can be combined to recognize handwritten words in document images more efficiently; and Saharia, Serrano can be combined with Hsu, to allow users to apply design font to text in the design. It’s obvious for the ordinary skill in the art to combine known technique to achieve reasonable and useful result.
Applicant argues on page 14-15 about the dependent claims.
In reply, the applicant can consider to incorporate dependent claims or allowable subject matter into independent claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Yi Yang whose telephone number is (571)272-9589. The examiner can normally be reached on Monday-Friday 9:00 AM-6:00 PM EST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached on 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/YI YANG/
Primary Examiner, Art Unit 2616
Prosecution Insights