DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed June 19th 2026 has been entered. Claims 1-20 are pending in the application. Applicant’s amendments to the Claims 1, 2, 5, 8, 10, 11, 12, and 17 have overcome the rejections previously set forth in the Final Office Action mailed March 19th 2026. Further search has been performed to address the material amended in the aforementioned claims. Newly found reference Cao (NPL: DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models) was used for the amended claim limitations.
Response to Arguments
The Examiner appreciates the Applicant’s thorough review of the previous Final Office Action and the clarifications provided during the interview on May 26th 2026. Applicant’s arguments with respect to claims 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 3, 4, 5, 6, 7, 8, 10, 17, 18, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Gal (NPL: An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion) in view of Burgess (NPL: Viewpoint Textual Inversion: Unleashing Novel View Synthesis with Pretrained 2D Diffusion Models) and Cao (NPL: DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models).
Regarding claim 1:
Gal teaches:
A method comprising:
obtaining a text prompt and an attribute value (Gal: “A painting of S∗ sitting next to a serene pond”, Figure 11, Pg. 20; see Note 1A);
embedding the text prompt to obtain a text embedding in a text embedding space (Gal: Typical text encoder models, […] begin with a text processing step (Figure 2, left). First, each word or sub-word in an input string is converted to a token […] Each token is then linked to a unique embedding vector […] In our work, we choose this embedding space as the target for inversion, Pg. 5, Text Embeddings, par. 1-2);
embedding, using a model, the attribute value to obtain an attribute embedding in the text embedding space (Gal: we designate a placeholder string, S∗, to represent the new concept we wish to learn. We intervene in the embedding process and replace the vector associated with the tokenized string with a new, learned embedding v∗, in essence “injecting” the concept into our vocabulary. In doing so, we can then compose new sentences containing the concept, just as we would with any other word, Pg. 5, Text embeddings, par. 2); and
generating, using an image generation model, a synthetic image based on a text embedding of a text prompt (Gal: each word or sub-word in an input string is converted to a token, which is an index in some pre-defined dictionary. Each token is then linked to a unique embedding vector that can be retrieved through an index-based lookup, Pg. 5, Text embeddings, par. 1; see Note 1B) and the attribute embedding (Gal: We represent a new embedding vector with a new pseudo-word (Rathvon, 2004) which we denote by S∗. This pseudo-word is then treated like any other word, and can be used to compose novel textual queries for the generative models, Pg. 2, Section 1: Introduction, par. 5), wherein the synthetic image depicts the continuous attribute based on the attribute value (see Note 1C).
Note 1A: The specification of the present application recites: “In one aspect, the continuous attribute input includes a 3-dimensional characteristic of the element such as orientation, illumination direction, nonrigid shape transformation, object pose, zoom effect, etc.” [0026].
Gal teaches in Figure 11, a prompt of the form: “A painting of S* sitting next to a serene pond”. The text prompt of Gal includes a placeholder string, which may be considered an attribute with a corresponding embedding: “we introduce both a universal placeholder, S∗, and an additional placeholder unique to each image, {Si}n i=1, associated with a unique embedding vi. We then compose sentences of the form “A photo of S∗ with Si”, where every image is matched to sentences containing its own, unique string.” (Pg. 12, par. 4)
Note 1B: Gal teaches that each word in a string may be used to generate a text embedding. In Figure 2 on Pg. 4, Gal further shows that “the embedding vectors are transformed into a single conditioning code cθ(y) which guides the generative model.”
Note 1C: Gal showcases a plurality of examples where the text-to-image model generates a synthetic image on Pg. 22-25. For example, on Pg. 22, Gal showcases a synthetic image labelled “The Scream in the style of S*” based on the attribute embedding obtained from the “pseudo word” S*. Given that the attribute embedding was generated based on a continuous vector (as shown in Note 1B), it is reasonable to conclude that Gal teaches generating synthetic images based on the attribute value.
Gal fails to explicitly teach:
obtaining a text prompt and an attribute value, wherein the text prompt describes an element and the attribute value comprises a numerical value that indicates a change to a semantic attribute of the element, wherein the semantic attribute comprises a shape or an illumination of the element;
embedding, using a continuous control model, the attribute value to obtain an attribute embedding in the text embedding space, wherein the continuous control model is trained to generate text embeddings for a plurality of values of the semantic attribute; and
generating, using an image generation model, a synthetic image based on the text embedding and the attribute embedding, wherein the synthetic image depicts the continuous element with the change to the semantic attribute of the element based on indicated by the attribute value.
Burgess teaches:
obtaining a text prompt (Burgess: We create a caption for each image. Pg. 3, Figure 3) and an attribute value (Burgess: The caption contains a camera-specific token SRi that is different for each view, Pg. 3, Figure 3),
wherein the text prompt describes an element (Burgess: The caption contains […] an object token, So, Pg. 3, Figure 3) and the attribute value comprises a numerical value for a continuous attribute of the element (Burgess: … in Fig. 20, we show NVS results where the camera is parameterized by spherical coordinates. We assume a central object is fixed at the origin, and that the camera is at a fixed radius with variable polar and azimuth angles, (θ,φ), Pg. 14, Section G: Validation of spherical coordinate parameterization, par. 1; see also Note 1E);
embedding the text prompt to obtain a text embedding in a text embedding space (Burgess: The prompt is passed through the CLIP text encoder, Pg. 3, Figure 3);
embedding, using a continuous control model (Burgess: We parameterize Mv as a 2-layer MLP with 64 dimensions, Pg. 5, par. 1), the attribute value to obtain an attribute embedding in the text embedding space (Burgess: Right: textual inversion training of our neural mappers, Mv and Mo. The mappers predict the word embeddings for SRi and So respectively. Pg. 3, Figure 3), wherein the continuous control model is trained to generate text embeddings (Burgess: We optimize the weights of Mv and Mo using the loss in Eq. (1), except we replace D with DMV′. Intuitively, we are learning text space latents, Pg. 5, Section 4.2: Single-Scene Novel View Synthesis) for a plurality of values of the attribute (Burgess: We propose an architecture for predicting text embeddings that control camera viewpoint, Pg. 2, Section 2: Related work, par. 2; see Note 1F); and
generating, using an image generation model (Burgess: It produces views with photorealistic details for real-world objects that are in the massive 2D training distribution of 2D diffusion models like Stable Diffusion, Pg. 2, col. 2, par. 1), a synthetic image based on the text embedding and the attribute embedding, wherein the synthetic image depicts the continuous attribute of the element based on the attribute value (Burgess: Right: our viewpoint control mechanism enables leveraging pretrained 2D diffusion models for novel view synthesis from only a single view; we can learn from more views if available, Pg. 1, Figure 1; Burgess: ViewNeTI controls viewpoint in text-to-image generation by composing the view-mapper text encoder, represented by Ri, with novel text prompts, Pg. 8, Figure 6).
Note 1E: The specification of the present application teaches that: “The continuous attribute, such [as] […] an apparent camera view of the scene can be difficult to describe precisely using text. For example, it can include one or more numerical parameters such as distance and angle (e.g., the distance between an object and the viewpoint, or angles describing the relationship between an object and a light source). […] The attribute can be described in terms of one or more continuous variables such as 3D position coordinates, Euler angles, or orientation angles such as yaw, pitch and roll.”
Burgess teaches: “in Fig. 20, we show NVS results where the camera is parameterized by spherical coordinates. We assume a central object is fixed at the origin, and that the camera is at a fixed radius with variable polar and azimuth angles, (θ,φ);” (Pg. 14, Section G: Validation of spherical coordinate parameterization, par. 1). In other words, Burgess teaches a camera view of the scene parameterized by angles.
Note 1F: The specification of the current application states: “the continuous control model includes a multilayer perceptron (MLP), where the MLP is able to receive a continuous input (e.g., the attribute value) and generate a continuous output (e.g., the attribute embedding)” [0075].
Burgess teaches: “We pass the text encoder a prompt like, ‘SR. A photo of a So’, where SR has word embedding vR and So has word embedding vo. The embedding for SR controls the viewpoint for the image that is been generated.” Furthermore, Burgess teaches that “the neural mappers [] learn a predictor of word-embedding space that is sufficiently sensitive to small changes in camera parameters; that is, it can represent high frequency changes in word embedding space.”
Therefore, the Examiner understands that the encoder taught by Burgess accepts an input (attribute value) and outputs a continuous output (attribute embedding). Burgess also teaches: “Following the base architecture of [1], the encoding is passed through an MLP with two blocks”. (Pg. 15, Section I: ViewNeTI implementation details) Therefore, the encoder parsing the prompts is an MLP, similarly to what is described in the specification at paragraph [0075]. It follows that Burgess teaches the continuous control model of claim 1.
Before the effective filing date of the present application, it would be obvious to one of ordinary skill in the art to combine the teachings of Burgess with Gal. Embedding the attribute value to obtain an attribute embedding in the text embedding space, as in Burgess, would benefit the Gal teachings by enabling controlled modification of the subject and camera angle in a generated image: “The pretrained ViewNeTI mapper generalizes to novel scenes, enabling synthesis of novel views far from the input views with little data; it can even do [novel view synthesis (NVS)] from a single image. Compared to existing single-image NVS methods, ViewNeTI has several advantages, especially in single-view NVS. It produces views with photorealistic details for real-world objects that are in the massive 2D training distribution of 2D diffusion models like Stable Diffusion. Once trained, it can generate diverse predictions under uncertainty in close to real time” (Burgess, Pg. 2, col. 2, par. 1).
Gal in view of Burgess still fails to teach:
obtaining a text prompt and an attribute value, wherein the text prompt describes an element and the attribute value comprises a numerical value that indicates a change to a semantic attribute of the element, wherein the semantic attribute comprises a shape or an illumination of the element;
generating, using an image generation model, a synthetic image based on the text embedding and the attribute embedding, wherein the synthetic image depicts the continuous element with the change to the semantic attribute of the element based on indicated by the attribute value.
Cao teaches:
obtaining a text prompt and an attribute value (Cao: Our network takes as input a text prompt and SMPL parameters, Pg. 5, Fig. 3), wherein the text prompt describes an element (see Note 1G) and the attribute value comprises a numerical value that indicates a change to a semantic attribute of the element (see Note 1H), wherein the semantic attribute comprises a shape (Cao: As illustrated in Fig. 3, our proposed framework takes as input a text prompt and SMPL parameters, defining the target shape and pose in the posed space, Pg. 4, Section 3.2: DreamAvatar) or an illumination of the element;
generating, using an image generation model, a synthetic image based on the text embedding and the attribute embedding, wherein the synthetic image depicts the continuous element with the change to the semantic attribute of the element based on indicated by the attribute value (Cao: Figure 2. Avatar generation with different poses. Our method can handle and control the 3D generation with any pose, Pg. 3, Fig. 2; see Note 1I).
Note 1G: Cao showcases different prompts in Fig. 4 and Fig. 5 on Pg. 6, such as “Crystal Maiden as in Dota2” and “Buddhist Monk”.
Note 1H: Cao teaches that the SMPL parameters are used to obtain a 3D SMPL human model: “Formally, by assembling pose parameters ξ and shape parameters β, we can obtain the 3D SMPL human model by [Equations 4 and 5]” (Pg. 4, col. 2, SMPL [4, 44] 3D parametric human model). Because the SMPL parameters are used in mathematical equations, the Examiner submits that the SMPL parameters are numerical values that indicate a change to a base SMPL model.
Note 1I: Cao showcases that in Figure 2 on Pg. 3, an image based on a prompt (such as “Spiderman”) may be used alongside SMPL shape and pose parameters to generate an image of a posed model. Contrast with Figure 1 on Pg. 2, where the SMPL is not assigned a pose. Furthermore, the semantic attribute may be modified to change properties of the element: “Our method can generate 3D avatars with different shapes based on the input SMPL shape parameters, e.g., “short”, “fat”, “thin”, “tall” as above from left to right.” (Cao, Pg. 7, Fig. 7)
Before the effective filing date of the present application, it would be obvious to one of ordinary skill in the art to combine the teachings of Cao with Gal in view of Burgess. Utilizing a semantic attribute describing a shape, as in Cao, would benefit the Gal in view of Burgess teachings by producing realistic outputs based on the input prompt and attribute: “human bodies, which are the primary focus of this paper, exhibit a complex articulated structure, with head, arms, hands, trunk, legs, feet, etc., each capable of posing in various ways. As a result, while Dream Fusion [46] and subsequent methods (e.g., Magic3D [29], ProlificDreamer [58], Fantasia3D [8]) produce impressive results, they lack the proper constraints to enforce consistent 3D human structure” (Cao, Pg. 1, col. 2, par. 2)
Regarding claim 2:
Gal in view of Burgess and Cao teaches:
The method of claim 1 (as shown above), wherein:
the semantic attribute comprises a 3-dimensional characteristic of the element (Cao: In this paper, we present DreamAvatar, a novel frame work for generating high-quality 3D human avatars from text prompts and shape priors, Pg. 1, col. 2, par. 3).
Regarding claim 3:
Gal in view of Burgess and Cao teaches:
The method of claim 1 (as shown above), wherein embedding the text prompt comprises:
dividing the text prompt into a plurality of tokens (Gal: Here, an input string is first converted to a set of tokens, Pg. 2, Section 1: Introduction, par. 4); and
embedding each of the plurality of tokens using a text embedding model (Gal: Each token is then replaced with its own embedding vector, and these vectors are fed through the downstream model, Pg. 2, Section 1: Introduction, par. 4).
Regarding claim 4:
Gal in view of Burgess and Cao teaches:
The method of claim 1 (as shown above), wherein: the text prompt includes a nonce token corresponding to the attribute value (see Note 4A).
Note 4A: The specification of the present application teaches: “In some cases, the text prompt includes a nonce token that corresponds to the attribute. For example, the text prompt states “A <V*> photo of a horse,” where <V*> represents the nonce token” [0041]. Similarly, Burgess teaches: “We pass the text encoder a prompt like, ‘SR. A photo of a So’, where SR has word embedding vR and So has word embedding vo.” (Pg. 5, par. 2) Therefore, as best understood by the Examiner, SR is a nonce token. Additionally, SR was previously discussed to correspond to the continuous attribute in the rejection of claim 1.
Regarding claim 5:
Gal in view of Burgess and Cao teaches:
The method of claim 1 (as shown above), wherein:
the text prompt includes a word corresponding to the semantic attribute (Burgess: The caption contains a camera-specific token SRi that is different for each view, and an object token, So, which is common across views, Pg. 3, Figure 3; see Note 5A and Note 5B).
Note 5A: Burgess teaches that SR and So have word embeddings: “We pass the text encoder a prompt like, ‘SR. A photo of a So’, where SR has word embedding vR and So has word embedding vo.” (Pg. 5, par. 2). Therefore, the Examiner understands SR and So to be analogous to words. Additionally, SR was previously discussed to correspond to the continuous attribute in the rejection of claim 1.
Note 5B: Cao teaches: “Our method can generate 3D avatars with different shapes based on the input SMPL shape parameters, e.g., “short”, “fat”, “thin”, “tall” as above from left to right.” (Pg. 7, Figure 7). As shown in the figure, the text indicating a shape of the 3D model corresponds to the parameters applied to the output.
Regarding claim 6:
Gal in view of Burgess and Cao teaches:
The method of claim 1 (as shown above), further comprising:
encoding the text embedding and the attribute embedding to obtain guidance information for the image generation model (Gal: the embedding vectors are transformed into a single conditioning code cθ(y) which guides the generative model, Pg. 4, Figure 2; see Note 18A), wherein the synthetic image is generated based on the guidance information (Gal: Figure 2; see Note 6A).
Note 6A: Figure 2 of Gal depicts the conditioning code (cθ(y)) being sent from the Text Encoder to the Generator, which produces the synthetic image (Specifically, the output synthetic image is depicted in the figure as left of the other clock image labelled “Input sample”).
Regarding claim 7:
Gal in view of Burgess and Cao teaches:
The method of claim 1 (as shown above), wherein generating the synthetic image comprises:
performing a diffusion process on a noise input to obtain the synthetic image (Gal: At inference time, a random noise tensor is sampled and iteratively denoised to produce a new image latent, z0. Finally, this latent code is transformed into an image through the pre-trained decoder x0 = D(z0), Pg. 5, Latent Diffusion Models, par. 3).
Regarding claim 8:
Gal in view of Burgess and Cao teaches:
The method of claim 1 (as shown above), wherein:
the image generation model is trained using a training set (Burgess: We do diffusion model training on this dataset while optimizing only Mv and Mo (this is textual inversion training [1,13], Pg. 3, Figure 3) including the plurality of training images depicting an object depicting an object (Burgess: we have a small multi-view dataset, DMV containing images, xi, Pg. 3, Figure 3) with a plurality of values of the semantic attribute, respectively (Burgess: Examples of novel view synthesis using ViewNeTI where the input camera parameters are In spherical coordinate system. We do single-scene NVS using an input dataset of nine multiview images of a Shape Net car with random forward-facing poses, Pg. 21, Figure 20).
Regarding claim 10:
Gal in view of Burgess and Cao teaches:
The method of claim 1 (as shown above), further comprising:
obtaining an additional attribute value corresponding to an additional continuous attribute, wherein the synthetic image is generated to depict the additional attribute value (Gal: Let {xi}ni=1 be the set of input images. Rather than optimizing a single word vector shared across all images, we introduce both a universal placeholder, S∗, and an additional placeholder unique to each image, {Si}ni=1, associated with a unique embedding vi. We then compose sentences of the form “A photo of S∗ with Si”, where every image is matched to sentences containing its own, unique string, Pg. 12, Section 5.2: Evaluation steps, par. 4 (emphasis added); see Note 10A).
Note 10A: Gal teaches that instead of using just one attribute value S*, an indefinite amount of attribute values may be added to a prompt for image generation.
Regarding claim 17:
Claim 17 is substantially similar to claim 1, and is therefore rejected for similar reasons (over Gao in view of Burgess and Cao). Claim 17 contains the following notable differences:
Claim 17 claims an apparatus instead of a method. Gal teaches a apparatus, as Gal teaches executing the method on a graphical processing unit (GPU) (Gal: Our experiments were conducted using 2×V100 GPUs with a batch size of 4, Pg. 5, Implementation details, par. 1) which inherently comprises a memory and processor.
Claim 1 contains an obtaining step not present in Claim 17.
Regarding claim 18:
Gal in view of Burgess and Cao teaches:
The apparatus of claim 17 (as shown above), further comprising:
a text encoder comprising parameters (Gal: Here, cθ is realized through a BERT (Devlin et al., 2018) text encoder, with y being a text prompt, Pg. 5, Latent Diffusion Models, par. 5) stored in the at least one memory (see Note 1B) and configured to encode the text embedding and the attribute embedding to obtain guidance information for the image generation model (Gal: the embedding vectors are transformed into a single conditioning code cθ(y) which guides the generative model, Pg. 4, Figure 2; see Note 18A).
Note 18A: Gal teaches that the embedding vectors generated from the tokens may be used to generate a “conditioning code” cθ that guides the generative model, and therefore, it is reasonable to call the conditioning code “guidance information.”
Regarding claim 19:
Gal in view of Burgess and Cao teaches:
The apparatus of claim 17 (as shown above), wherein: the continuous control model comprises a multilayer perceptron (MLP) (Burgess: We parameterize Mv as a 2-layer MLP with 64 dimensions, Pg. 5, par. 1).
Regarding claim 20:
Gal in view of Burgess and Cao teaches:
The apparatus of claim 17 (as shown above), wherein: the image generation model comprises a diffusion model (Gal: we outline the core details of applying our approach to a specific class of generative models — Latent Diffusion Models, Pg. 4, Section 3: Method, par. 4; Burgess: We then sample from a frozen Stable Diffusion model conditioned on this text latent to produce an image of the So object from camera view Ri, Pg. 1, Figure 1; Cao: We employ a pretrained diffusion model, Pg. 4, col. 2, par. 1).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Gal (NPL: An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion) in view of Burgess (NPL: Viewpoint Textual Inversion: Unleashing Novel View Synthesis with Pretrained 2D Diffusion Models), Cao (NPL: DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models) and Andrew (NPL: How to use negative prompts?)
Regarding claim 9:
Gal in view of Burgess and Cao teaches:
The method of claim 8 (as shown above), further comprising:
Gal in view of Burgess and Cao fails to teach:
identifying a negative prompt based on the object from the plurality of training images, wherein the synthetic image is generated based on the negative prompt.
Andrew teaches:
identifying a negative prompt based on the object from the plurality of training images, wherein the synthetic image is generated based on the negative prompt (Andrew: “You want to generate another one but with an empty street. You can use the same seed value, which specifies the image, and add the negative prompt “people”. You get an image with most people removed, Pg. 4-5, par. 1; see Note 9A).
Note 9A: Andrew teaches that a negative prompt may be used to remove objects from an already generated synthetic image. Specifically, Andrew identifies the negative prompt “people” by inspecting the input image on Pg. 4, and then generates the synthetic image on Pg. 5, and because of the negative prompt, the people in the image are removed.
Therefore, when the method of Andrew is applied to the teachings of Gal in view of Burgess and Cao, it would be obvious to one of ordinary skill in the art to remove the main element from, for example, some of the input sample images in pages 22-25 of Gal using a negative prompt.
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Andrew with Gal in view of Burgess and Cao. Identifying a negative prompt based on the object from the plurality of training images, wherein the synthetic image is generated based on the negative prompt, a plurality of synthetic images based on a same random input, as in Andrew, would benefit the Gal in view of Burgess and Cao teachings by enabling removal of elements in the image.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Gal (NPL: An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion) in view of Burgess (NPL: Viewpoint Textual Inversion: Unleashing Novel View Synthesis with Pretrained 2D Diffusion Models), Cao (NPL: DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models), and AI Prompt Directory (NPL: How to Use Seeds for Better Control in Stable Diffusion).
Regarding claim 11:
Gal in view of Burgess and Cao teaches:
The method of claim 1 (as shown above), further comprising:
obtaining a plurality of attribute values for the continuous attribute (Burgess: Novel view synthesis trained on a single scene, […] inference views that are ‘interpolated’ from the input (green), and inference views that are ‘extrapolated’ (red). Pg. 6, Figure 4); and
generating, using the image generation model, a plurality of synthetic images based on the plurality of attribute values (Burgess: Examples of novel view synthesis using ViewNeTI where the input camera parameters are In spherical coordinate system. We do single-scene NVS using an input dataset of nine multiview images of a Shape Net car with random forward-facing poses, Pg. 21, Figure 20; see Note 11A and Note 11B), respectively.
Note 11A: Burgess showcases that multiple synthetic images may be generated with various camera angles. In the rejection of claim 1, the camera angle was analogized to the continuous attribute.
Note 11B: Cao showcases on Pg. 7 in Figure 7 that for different values of the SMPL shape parameters, that multiple synthetic images may be generated for each value of the parameters: “Our method can generate 3D avatars with different shapes based on the input SMPL shape parameters, e.g., “short”, “fat”, “thin”, “tall” as above from left to right.” (Pg. 7, Fig. 7).
Gal in view of Burgess and Cao fails to teach:
generating, using the image generation model, a plurality of synthetic images based on a same random input.
AI Prompt Directory teaches:
generating, using the image generation model (AI Prompt Directory: Stable Diffusion, Pg. 1, par. 1), a plurality of synthetic images based on a same random input (AI Prompt Directory: Seeds are numbers that control the randomness. By fixing the seed value, you can reliably reproduce an image you’ve previously generated, Pg. 2, par. 1).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of AI Prompt Directory with Gal in view of Burgess and Cao. Generating, using the image generation model, a plurality of synthetic images based on a same random input, as in AI Prompt Directory, would benefit the Gal in view of Burgess and Cao teachings by enabling regeneration of images that were generated randomly.
Claims 12, 13, 14, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Burgess (NPL: Viewpoint Textual Inversion: Unleashing Novel View Synthesis with Pretrained 2D Diffusion Models) in view of Cao (NPL: DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models).
Regarding claim 12:
Burgess teaches:
A method comprising:
initializing a machine learning model (see Note 12A);
obtaining a training set including a plurality of training images depicting an object with a plurality of values of a continuous attribute (Burgess: Left: we have a small multi-view dataset, DMV containing images, xi, and known camera poses, Ri, Pg. 3, Figure 3; see Note 12B), respectively;
training, using the training set, an image generation model (Burgess: We do diffusion model training on this dataset while optimizing only Mv and Mo, Pg. 3, Figure 3; see Note 12C) to generate synthetic images with the plurality of values of the continuous attribute (Burgess: Examples of novel view synthesis using ViewNeTI where the input camera parameters are In spherical coordinate system. We do single-scene NVS using an input dataset of nine multiview images of a Shape Net car with random forward-facing poses, Pg. 21, Figure 20; see Note 12C); and
training, using the training set, a continuous control model to generate text embeddings corresponding to the semantic attribute (Burgess: We train a small neural mapper to take camera viewpoint parameters and predict text encoder latents, Abstract; Burgess: Right: textual inversion training of our neural mappers, Mv and Mo. The mappers predict the word embeddings for SRi and So respectively. Pg. 3, Figure 3) as an input for the image generation model (Burgess: The ViewNeTI mappers control the text encoding for the diffusion model, Pg. 4, Section 4.1: ViewNETI mapper architecture and inference).
Note 12A: Burgess discusses pre-training both the mappers (Pg. 6, Section 5.2.1: View-mapper pretraining) and the diffusion model (Pg. 3, col. 2, par. 1), and therefore initializes at least one machine learning model.
Note 12B: The Examiner interprets the known camera poses Ri as the plurality of values of a continuous attribute, because the specification of the present application teaches that: “The continuous attribute, such orientation of an object or an apparent camera view of the scene can be difficult to describe precisely using text” [0068] (emphasis added). As best understood by the examiner, camera poses describe an apparent camera view of the scene.
Note 12C: The Examiner understands Burgess to teach “training an image generation model using the training set to generate synthetic images with the plurality of values of the continuous attribute”, because Burgess teaches that a diffusion model, specifically StableDiffusion, will generate images (Burgess: Viewpoint Neural Textual Inversion (ViewNeTI) controls the viewpoint of objects in images generated by diffusion models, Pg. 4, Section 4: Method) after having been trained on the dataset discussed in Note 12B (see Pg. 3, Figure 3 cited above).
In Note 12B, it was also discussed that the continuous attribute, the known camera poses Ri, was utilized as part of the training dataset. The images generated by the diffusion model in Burgess are generated with the plurality of values of the continuous attribute, because on Pg. 21, Burgess showcases interpolated images generated from multiple training images, where each training image has an apparent camera view. Therefore, when Burgess “do[es] single-scene NVS using an input dataset of nine Multiview images of a Shape Net car”, Burgess is generating synthetic images while utilizing the camera poses from each training image, or in other words, the plurality of values of the continuous attribute from each training image.
Burgess fails to teach:
obtaining a training set including a plurality of training images depicting an object with a plurality of values indicating a change to a semantic attribute, respectively, wherein the semantic attribute comprises a shape or an illumination of an element;
training, using the training set, an image generation model to generate synthetic images with the plurality of values of the semantic attribute, wherein the synthetic image depicts the element with the change to the semantic attribute; and
Cao teaches:
initializing a machine learning model (Cao: We employ a pretrained diffusion model, Pg. 4, col. 2, (3) Variational Score Distillation (VSD));
obtaining a training set including a plurality of training images depicting an object with a plurality of values (Cao: Coupled with pretrained text to-image diffusion models [50, 68], DreamAvatar can be trained to generate 3D avatars using 2D self-supervision, Pg. 1, col. 2, par. 3; see Note 12A) indicating a change to a semantic attribute, respectively, wherein the semantic attribute comprises a shape (Cao: As illustrated in Fig. 3, our proposed framework takes as input a text prompt and SMPL parameters, defining the target shape and pose in the posed space, Pg. 4, Section 3.2: DreamAvatar; see also Note 12B) or an illumination of an element;
training, using the training set, an image generation model (Cao: Our network takes as input a text prompt and SMPL parameters to optimize a trainable NeRF via a pretrained denoising stable diffusion model, Pg. 5, Figure 3) to generate synthetic images with the plurality of values of the semantic attribute, wherein the synthetic image depicts the element with the change to the semantic attribute (Cao: Our method can generate 3D avatars with different shapes based on the input SMPL shape parameters, e.g., “short”, “fat”, “thin”, “tall” as above from left to right, Pg. 7, Fig. 7); and
Note 12A: Cao teaches that their method utilizes a pre-trained diffusion model. Above, the Examiner showed that Burgess teaches a diffusion model that utilizes a training dataset.
Note 12B: Cao teaches: "Our network takes as input a text prompt and SMPL parameters to optimize a trainable NeRF via a pretrained denoising stable diffusion model." (Pg. 5, Figure 3). Therefore, the Examiner submits that Cao teaches that the inputs (text prompt and SMPL parameters) may be used as training data, and that it would be obvious to one having ordinary skill in the art to include the inputs in a training data set.
Regarding claim 13:
Burgess in view of Cao teaches:
The method of claim 12 (as shown above), wherein obtaining the training set comprises: rendering the plurality of training images based on a 3D model of the object (Burgess: Novel view synthesis trained on a single scene, Pg. 6, Fig. 4; see Note 13A)
Note 13A: Burgess showcases in Fig. 4 that a 3D model of the object may be utilized to capture the training set or “ground truth”.
Regarding claim 14:
Burgess in view of Cao teaches:
The method of claim 12 (as shown above), wherein obtaining the training set comprises: generating, using a training image generation model, a training image based on a 3D model of the object (Burgess: To encourage the image to be close to the diffusion model training distribution, we generated an augmented training set by outfilling the background around the car, similar to in Appendix A, Pg. 14, Section G: Validation of spherical coordinate parameterization, par. 1; see Note 14A).
Note 14A: Burgess teaches that the training image set may be generated, and uses Stable Diffusion to outfill training set images: “In Fig. 12 we ask a diffusion model to do outfilling around a real car.” (Pg. 13, Section A: Evidence for 3D capabilities in diffusion models with image outfilling, par. 1)
Regarding claim 16:
Burgess in view of Cao teaches:
The method of claim 12 (as shown above), wherein training the image generation model comprises:
computing a reconstruction loss (Burgess: We optimize the weights of Mv and Mo using the loss in Eq. (1), Pg. 5, Section 4.2: Single-Scene Novel View Synthesis) based on the training set (Burgess: SD’s are Latent Diffusion Models (LDMs) for image generation [41] and are typically trained on web-scale datasets of text-image pairs (x,y)∼D, Pg. 3, Section 3.1: Text-to-image latent diffusion models; see Note 16A); and
updating parameters of the image generation model (see Note 16B) and parameters of the continuous control model based on the reconstruction loss (Burgess: We optimize the weights of Mv and Mo using the loss in Eq. (1), Pg. 5, Section 4.2: Single-Scene Novel View Synthesis).
Note 16A: The dataset of text-image pairs to be trained on, “(x, y) ~ D”, appears in the loss function, Equation 1 on Pg. 4 of Burgess.
Note 16B: The specification of the current application teaches: “The training component then updates parameters of the diffusion model 900 based on the comparison. For example, parameters of a U-Net may be updated using gradient descent. Time-dependent parameters of the Gaussian transitions can also be learned.”
Burgess teaches: “Both neural mappers, (Mv,Mo) are conditioned on the denoising timestep and the diffusion model UNet layer, (t,ℓ).”
In other words, Burgess teaches a UNet for the diffusion model as well as a time-dependent timestep parameter. As best understood by the examiner, Burgess also teaches that the UNet parameters may be updated: “The text is passed through a pretrained CLIP [36] text encoder, giving a d-dim conditioning vector for each token, c(y) ∈ Rd×77, which is mixed with each Unet layer via cross-attention” (Pg. 4, col.1, par. 2, emphasis added).
Therefore, the Examiner understands Burgess to teach updating parameters of the image generation model.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Burgess (NPL: Viewpoint Textual Inversion: Unleashing Novel View Synthesis with Pretrained 2D Diffusion Models), Cao (NPL: DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models), and Kowalski (US 20210335029 A1).
Regarding claim 15:
Burgess in view of Cao teaches:
The method of claim 12 (as shown above), wherein:
Burgess in view of Cao fails to explicitly teach:
the image generation model is trained individually in a first stage, and
the image generation model is trained together with the continuous control model in a second stage.
Kowalski teaches:
the image generation model is trained individually in a first stage (Kowalski: a first stage 900 involves omitting the real data encoder 904 and randomly generating 906 embeddings of real images. During the first stage the synthetic data encoder and the decoder are trained using backpropagation 908 and using synthetic images, [0079]; see Note 15A), and
the image generation model is trained together with the continuous control model in a second stage (Kowalski: In the second stage 902 the real data encoder is included 910, [0079]).
Note 15A: Omitting the real data encoder may be considered training the “generative model” (Kowalski, [0035]) individually. When the real data encoder is included with the autoencoder, this may be considered training together.
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Kowalski with Burgess in view of Cao. Training the image generation model individually in a first stage, and training the image generation model together with the continuous control model in a second stage, as in Kowalski, would benefit the Burgess in view of Cao teachings because “experiments show that this two-stage training improves controllability and image quality,” (Kowalski, [0086]).
Conclusion
The Examiner identified potential limitation(s) in the specification that would overcome the prior art rejections under 103 if amended into the claims. Note that in such a situation, further search and consideration would be required:
The Examiner notes that while the newly found Cao reference teaches shape attributes, the best prior art found regarding illumination was previously cited reference Kowalski (US 20210335029 A1) which teaches:
“To generate an output image with a different attribute, the part of z0 that corresponds to that attribute is modified. The synthetic data encoder 302 is factorised into separate parts that correspond to different attributes. To modify, for example, illumination, pass the desired illumination parameters to the synthetic data encoder 302, which generates a part v of the latent embedding that corresponds to that illumination.” [0033].
In the present action, illumination was not mapped in favor of shape due to both being listed in the disjunctive. Furthermore, Kowalski does not explicitly discuss an illumination direction as described in paragraph [0055], [0056], and [0069] of the present specification.
“In some cases, the continuous attribute may include 2-dimensional characteristics of the element, such as edges, contours, color intensity, etc.” as in [0069] of the specification of the present application.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VINCENT ALEXANDER PROVIDENCE whose telephone number is (571)270-5765. The examiner can normally be reached Monday-Thursday 8:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571)270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VINCENT ALEXANDER PROVIDENCE/Examiner, Art Unit 2617 /KING Y POON/Supervisory Patent Examiner, Art Unit 2617
