Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 5-11, 13, 15, 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Gadelha, US 2025/0061650 A1, in view of Balasubramanian, US 2024/0005096 A1.
Regarding Claim 1, Gadelha teaches:
A computer-implemented method for generating data to train a machine learning model, the method comprising (paragraph 23: “given a training set, the GAN learns to generate new data with similar properties as the training set. For example, a GAN trained on photographs can generate new images that look authentic to a human observer. GANs may be used in conjunction with supervised learning, semi-supervised learning, unsupervised learning, and reinforcement learning”):
generating, via a first machine learning model and based on the prompt, a text description of at least one of a texture or a geometry for the object (paragraph 19: “text prompts may be used to describe appearance and generate details, textures, etc.”);
generating, via a second machine learning model and based on the text description, the at least one of the texture or the geometry for the object (paragraph 46: “Using image generation model 305 (e.g., a conditional image generative model) and a text description (e.g., text prompt 325), image processing system 300 may generate an RGB image following the scene geometry and textual guidance”);
and performing one or more rendering operations based on the at least one of the texture or the geometry for the object to generate one or more rendered images (paragraph 2: “image processing systems can generate aesthetically pleasing images from text prompts and 3D scenes provided by a user (e.g., by rendering a 3D scene geometry as a depth map and using a text-guided conditional image generative model)”; And, paragraph 71: “the output image comprises a 2D rendering of the 3D model. In some aspects, the 3D model comprises a plurality of 3D shapes, and wherein each of the 3D shapes comprises a different color”).
Although Gadelha may have taught the following, Balasubramanian more directly shows:
generating a prompt based on a template and information associated with an object (paragraph 55: “The object data 510 may be any suitable information about the object that may be provided as a text string for insertion in the prompt template”; And, paragraph 63: “For determining the prompt templates, a number of known training instances may be used, such that the object data (i.e., the text string describing the object) may be known, along with the attribute label of the object, such as “dairy” or “dairy-free.” In the example of FIG. 7, the attribute is a sentiment of an object, such as “great” or “terrible.” This may be, for example, reviews of a movie. In this instance, the object data is known, as is the attribute prediction, such that an effective prompt should be generated such that the application of the prompt to the object data may effectively yield the attribute as a predicted mask token by the masked language model. More formally, the problem for generating the prompt may be characterized as identifying one or more spans of text in which the object data and the masked label may be positioned. More formally, this may be described as determining the values X and Y in: “<object data> X <attribute> Y” such that the attribute may be predicted as a mask token by the masked language model”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the teachings of Balasubramanian with that of Gadelha for generating a prompt based on a template and information associated with an object.
The ordinary artisan would have been motivated to modify Gadelha in the manner set forth above for the purposes of generating an effective prompt using a prompt template [Balasubramanian: paragraph 63].
Regarding Claim 3, Gadelha further teaches:
The computer-implemented method of claim 1, further comprising generating data associated with the one or more rendered images, wherein the data associated with the one or more rendered images includes at least one of a location, a segmentation, or a text description of the object within each image included in the one or more rendered images (paragraph 24: “some aspects of output image generation by image processing system 100 may include generating (e.g., drawing) segmentation maps, manipulating scenes, labeling segments with labels (e.g., such as sky, sea, sand, snow, etc.), among other processing tasks”. Examiner’s note: see also Tremblay, US 2022/0044075 A1, for example paragraph 23).
Regarding Claim 5, Gadelha further teaches:
The computer-implemented method of claim 1, wherein performing the one or more rendering operations is further based on at least one of a selected lighting, a selected virtual camera pose, or a selected number of other objects (paragraph 25: “in addition to such predefined objects, image processing system 100 enables finer control of shapes, camera placements, etc. (e.g., which enables users 105 to create more complex, diverse, and precise geometry information/scenes). As such, 3D modeling applications (e.g., including full 3D controls) allow users 105 to create scenes that are not restricted to specific classes of objects and enables users 105 to adjust the camera placement to their liking”; And, paragraph 56: “image processing systems may use text prompts 415 to generate image content such as wrapping 2D images around 3D models 405 and determining how light would affect it in the generated output images”).
Regarding Claim 6, Gadelha further teaches:
The computer-implemented method of claim 1, further comprising retrieving the information associated with the object from a database that stores at least one of a predefined texture or a predefined geometry for the object (paragraph 25: “image processing system 100 may include predefined classes of objects (e.g., such as basic shapes, landscapes, furniture, cars, etc., via BlockGAN, GIRAFFE, etc.) that allow/enable the user 105 to provide geometry information leveraging such predefined objects”).
Regarding Claim 7, Gadelha further teaches:
The computer-implemented method of claim 1, wherein generating the at least one of the texture or the geometry for the object comprises inputting, into the second machine learning model, the text description, a predefined geometry for the object, and a noisy texture (paragraph 2: “an image processing system configured to generate an image, e.g., based text information and three-dimensional (3D) geometry information”; And, paragraph 82: “a forward diffusion process 830 gradually adds noise to the original image features 820 to obtain noisy features 835 (also in latent space 825) at various noise levels”).
Regarding Claim 8, Gadelha further teaches:
The computer-implemented method of claim 1, wherein the prompt asks the second machine learning model to describe the at least one of the texture or the geometry for the object (paragraph 48: “The image generation model 305 uses the depth map (e.g., generated based on 3D model 310) and text prompt 325 to create an output image 330 (e.g., a RGB image) that follows the scene geometry and textual guidance. To achieve this, the image generation model 305 may understand and interpret text prompt 325 and incorporate it into the image generation process. Additionally, the image generation model 305 may generate textures and details (e.g., based on text prompt 325) that may otherwise be difficult for the user to provide using only 3D modeling”; And, paragraph 56: “image processing systems may use text prompts 415 to generate image content”).
Regarding Claim 9, Balasubramanian further teaches:
The computer-implemented method of claim 1, wherein the first machine learning model comprises a language model (paragraph 3: “a masked language model is used to predict the attribute by constructing an attribute query for the model using a prompt template and object data for the object”).
Regarding Claim 10, Gadelha further teaches:
The computer-implemented method of claim 9, wherein the second machine learning model comprises a diffusion model (paragraph 74: “diffusion models can be used to generate novel images”).
Claims 2, 12 are rejected under 35 U.S.C. 103 as being unpatentable over Gadelha, US 2025/0061650 A1, in view of Balasubramanian, US 2024/0005096 A1, and further in view of Bai, US 2023/0334834 A1.
Regarding Claim 2, with Gadelha and Balasubramanian teaching those limitations of the claim as previously pointed out, neither Gadelha nor Balasubramanian may have explicitly taught the following, however, Bai shows:
The computer-implemented method of claim 1, further comprising performing one or more operations to train a third machine learning model based on the one or more rendered images (Abstract: “synthetic images are generated by providing respective text prompts into a text-to-image generation model. Respective training labels associated with the synthetic images are also generated based on the used text prompts. A target model, which is configured to perform an image classification task, is trained based at least in part on the synthetic images and the associated training labels. Through this solution, a large scale of synthetic images can be automatically obtained and applicable for training a model for image classification”. Examiner’s note: see also Tremblay, US 2022/0044075 A1, for example paragraph 4).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the teachings of Bai with that of Gadelha and Balasubramanian for training a third machine learning model based on the one or more rendered images.
The ordinary artisan would have been motivated to modify Gadelha and Balasubramanian in the manner set forth above for the purposes of having a large scale of synthetic images that can be automatically obtained and applicable for training a model for image classification, to improve the model performance with data-scare setting or in the case of model pre-training where the training data amount matters [Bai: paragraph 2].
Claims 4, 14 are rejected under 35 U.S.C. 103 as being unpatentable over Gadelha, US 2025/0061650 A1, in view of Balasubramanian, US 2024/0005096 A1, and further in view of Graham, US 2024/0346731 A1.
Regarding Claim 4, with Gadelha and Balasubramanian teaching those limitations of the claim as previously pointed out (Gadelha teaching texture and geometry), neither Gadelha nor Balasubramanian may have explicitly taught the following, however, Graham shows:
The computer-implemented method of claim 1, wherein performing the one or more rendering operations comprises simulating, via a physics simulator, the object within a virtual environment based on the at least one of the texture or the geometry for the object (paragraph 40: “The trained machine learning model(s) 100 may receive, as input, prompt data representing one or more prompts (e.g., image prompts, voice prompts, and/or text prompts, etc.), such as the user-provided prompt data 112 representing the voice prompt 116 provided by the user 110 in the example of FIG. 1. Accordingly, the user 110 can describe a person, an object, and/or a scene, and, in response to the received user-provided prompt data 112 representing this prompt(s) from the user 110, the machine learning model(s) 100 generates output data 104 representing synthetic content 102 (e.g., a synthetic face 102(1), a synthetic body 102(2), a synthetic background 102(3), etc.), which is rendered in real-time to a display of a display device 118. In some examples, video content 108 featuring the synthetic content 102 may be viewable by the user 110 who provided the prompt and/or by one or more other users. The level of immersion experienced by the viewing user(s) may depend on the type of display device 118 used to display the video content 108 featuring the synthetic content 102. For example, a more immersive experience may be provided by displaying the video content 108 on a HMD 118(2), such as a VR headset. For example, the user 110 wearing the HMD 118(2) may speak into the microphone(s) 114 to prompt the trained machine learning model(s) 100 to generate output data 104 representing synthetic content 102 that is displayed in real-time on the HMD 118(2). For a less immersive experience, the video content 108 may be displayed on a display device 118(1). In any case, the live prompting of the model(s) 100 by the user 110 guides the model(s) 100 to generate output data 104 representing photoreal synthetic content 102. In other words, the generative output data 104 is created by an algorithm that is prompted (e.g., by the user 110) in some way as to what to do”. The algorithm representative of the physics simulator. Examiner’s note: see also Rowell, US 2020/0342652 A1, for example paragraphs 19, 29, 47).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the teachings of Graham with that of Gadelha and Balasubramanian for performing the one or more rendering operations comprises simulating, via a physics simulator, the object within a virtual environment based on the at least one of the texture or the geometry for the object.
The ordinary artisan would have been motivated to modify Gadelha and Balasubramanian in the manner set forth above for the purposes of prompting a trained artificial intelligence model to output photoreal synthetic content in real-time [Graham: Abstract].
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Gadelha, US 2025/0061650 A1, in view of Balasubramanian, US 2024/0005096 A1, and further in view of Bu, US 2024/0290027 A1.
Regarding Claim 16, with Gadelha and Balasubramanian teaching those limitations of the claim as previously pointed out, neither Gadelha nor Balasubramanian may have explicitly taught the following, however, Bu shows:
The one or more non-transitory computer-readable media of claim 15, wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform the step of performing one or more operations to randomly select the at least one of the selected lighting, the selected virtual camera pose, or the selected number of other objects (paragraph 43: “In some examples, the randomized visual parameters may include lighting (i.e., illumination), object pose, object orientation, background image and other visual aspects”; And, paragraph 45: “a randomized visual parameter may include a manipulation of an object defined by the defined object parameters. Some examples of these randomized visual parameters include selecting random object types, random object positions, random object orientation, random object dimensions and random object textures”; And, paragraph 48: “a randomized visual parameter may include the lighting of the synthetic images. Some examples of randomized lighting parameters include random lighting positions, random lighting orientation, and random lighting intensity”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the teachings of Bu with that of Gadelha and Balasubramanian for performing one or more operations to randomly select the at least one of the selected lighting, the selected virtual camera pose, or the selected number of other objects.
The ordinary artisan would have been motivated to modify Gadelha and Balasubramanian in the manner set forth above for the purposes of generating multiple synthetic images of an object based on defined object parameters and randomized visual parameters [Bu: Abstract].
Claims 11-15 and 17-19 are similar to Claims 1-5 and 8-10 respectively and are rejected under the same rationale as stated above for those claims
Claim 20 is similar to Claim 1 and is rejected under the same rationale as stated above for that claim.
Examiner's Note:
The Examiner cites particular pages, sections, columns, line numbers, and/or paragraphs in the references as applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in its entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner and the additional related prior arts made of record that are considered pertinent to applicant's disclosure to further show the general state of the art. The Examiner's interpretations in parenthesis are provided with the cited references to assist the applicants to better understand how the examiner interprets the prior art to read on the claims. Such comments are entirely consistent with the intent and spirit of compact prosecution.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO-892 for the relevant prior art where for example Choe, US 2022/0122001 A1, teaches the generation of synthetic data to fortify a dataset for use in training a network via imitation learning.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVE MISIR whose telephone number is (571)272-5243. The examiner can normally be reached M-R 8-5 pm, F some hours.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar can be reached at 5712703169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVE MISIR/Primary Examiner, Art Unit 2127
Prosecution Insights