DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 4, 11-13, 15, and 18 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Joachim (U.S. Publication 2024/0362815).
As to claim 1, Joachim discloses a method for training an image generation model (p. 35, section 0384-p. 36, section 0386; a model is trained to generate modified images), comprising:
receiving training data comprising a plurality of training images (p. 66, section 0649; training images are received to train the models), a plurality of image captions corresponding to the plurality of training images (p. 35, section 0384; the image dataset is an image-caption dataset to train a model with captions representing semantic info for the visual images), and a corresponding plurality of image features, each training image associated with one image caption and further associated with one or more image features (p. 35, sections 0384-385; p. 66, sections 0649-0651; images are each associated with a caption in the dataset as well as visual and textual features and annotations that correspond to further features);
and performing a training process to condition the image generation model using the plurality of training images, the plurality of image captions, and the plurality of image features, resulting in a trained model that generates images conditioned to the plurality of image features (p. 35, sections 0384-0385; p. 36, section 0392; p. 66, sections 0649-0651; training occurs based on the images, captions, features, and additional annotated features), wherein the image features comprise a plurality of image properties that are extracted from pixels or regions of each training image and a plurality of camera properties that are associated with each training image (p. 35, section 0384-p. 36, section 0386; p. 64, sections 0635-0638; p. 66, sections 0650-0651; camera parameters/properties are features associated with each training image used to train the model using annotations; visual features or image properties extracted from the pixels/regions in the image itself, such as horizon lines and ground-to-horizon vectors, are also features used to train the model).
As to claim 4, Joaquim discloses wherein the image features comprise a parameter that is computed from one or more other image features (p. 66, section 0651; the scale information parameter is calculated based on other features).
As to claim 11, Joaquim discloses wherein each training image in the plurality of training images comprises a corresponding image feature stored as a metadata tag (p. 53, section 0545; p. 66, section 0651; p. 93, section 0867; each camera parameter or annotation describing an image feature in a training image in a dataset is stored in metadata in each image).
As to claim 12, Joaquim discloses wherein the image generation model is one of a Generative Adversarial Network (GAN), a Variational Autoencoder (VAE), an autoregressive model, a diffusion model, and a transformer-based architecture (p. 13, section 0178; p. 63, section 0626).
As to claim 13, see the rejection to claim 1. Further, Joaquim discloses a non-transitory computer-readable medium storing a program for training an image generation model, which when executed by a computer, configures the computer (p. 95, section 0883-0885; p. 97, sections 0905-0907).
As to claim 15, see the rejection to claim 4.
As to claim 18, see the rejection to claim 1. Further, Joaquim discloses a system for training an image generation model, comprising: a processor; and a non-transitory computer readable medium storing a set of instructions, which when executed by the processor, configure the system (p. 95, section 0883-0885; p. 97, sections 0905-0907).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Joaquim in view of Viehauser (U.S. Publication 2024/0095997).
As to claim 2, Joaquim does not disclose, but Viehauser discloses wherein the plurality of image properties comprise saturation, brightness, sharpness, and contrast (p. 1, section 0009; p. 3, section 0034; p. 14, section 0101; the trained model can take into account image properties such as saturation, brightness, sharpness, and contrast from the camera when reprojecting a camera image with different settings). The motivation for this is to correct views that are not clear (p. 3, section 0036). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Joaquim to use a model with properties comprising saturation, brightness, sharpness, and contrast in order to correct views that are not clear as taught by Viehauser.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Joaquim in view of Good (U.S. Patent 11,769,311).
As to claim 3, Joaquim discloses a plurality of camera properties including focal length (p. 65, section 0642). Joaquim does not disclose, but Good discloses wherein the plurality of camera properties comprise shutter speed, focal length, field of view, aperture, ISO, lens specifications, camera manufacturer, and camera model (col. 2, lines 20-51; col. 4, lines 33-63; col. 14, line 61-col. 15, line 5; col. 17, lines 5-25; properties including shutter speed, focal length, field-of-view, aperture, ISO, lens specification such as refractive properties, and different camera sensors/models from different manufacturers are modeled using a neural network). The motivation for this is to realistically generate effects that are improved over applying effects without considering the camera parameters (col. 1, lines 21-30). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Joaquim to model shutter speed, focal length, field of view, aperture, ISO, lens specifications, camera manufacturer, and camera model in order to more realistically generate effects as taught by Good.
Claims 5-9, 16, 17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Joaquim in view of Xu (U.S. Publication 2023/0042221).
As to claim 5, Joaquim does not disclose, but Xu discloses receiving an image generation request comprising a text description of a desired image and a desired image feature (p. 2, section 0019; p. 5, section 0046; p. 5, section 0049; p. 10, section 0087; natural language text is used to describe a desired image, e.g. one with an object removed, as well as a desired image feature, e.g. one with modified brightness); providing the image generation request to the trained model (p. 3, section 0030; p. 10, sections 0087-p. 11, section 0090; the request is provided to a CAGAN which is a trained model); and receiving as an output from the trained model in response to the image generation request, an output image that comprises image content that matches at least part of the text description of the desired image and further matches the desired image feature (p. 10, sections 0087-p. 11, section 0090; an image is output from the CAGAN that matches the desired image and feature). The motivation for this is to more accurately and less rigidly modify real-world image scenes (p. 2, section 0024-p. 3, section 0026). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Joaquim to receive an image generation request comprising a text description of a desired image and a desired image feature, provide the image generation request to the trained model, and receive as an output from the trained model in response to the image generation request, an output image that comprises image content that matches at least part of the text description of the desired image and further matches the desired image feature in order to more accurately and less rigidly modify real-world image scenes as taught by Xu.
As to claim 6, Joaquim does not disclose, but Xu discloses wherein the desired image feature is a first desired image feature, the image generation request comprises a second desired image feature, and the image content further matches the second desired image feature (p. 2, section 0019; p. 16, section 0130; p. 17, section 0136; second desired features can be included in the request and used to create the content; for example, “decrease exposure and increase contrast” results in an image that has both of these feature characteristics). Motivation for the combination is similar to that given in the rejection to claim 5.
As to claim 7, Joaquim does not disclose, but Xu discloses receiving an image generation request comprising an input image acquired using a first acquisition parameter and further comprising a desired second acquisition parameter, wherein the input image is visually characterized by the first acquisition parameter and comprises an image content (fig. 2a; p. 2, section 0019; p. 4, section 0045-p. 5, section 0046; an input image is acquired with a parameter such as a particular exposure and a request includes a desired second acquisition parameter such as a modified exposure); providing the image generation request to the trained model (p. 3, section 0030; p. 10, sections 0087-p. 11, section 0090; the request is provided to a CAGAN which is a trained model); and receiving as an output from the trained model in response to the image generation request, an output image that is visually characterized by the desired second acquisition parameter and comprises the image content (p. 10, sections 0087-p. 11, section 0090; an image is output from the CAGAN that matches the desired second parameter with input image content). Motivation for the combination is similar to that given in the rejection to claim 5.
As to claim 8, Joaquim does not disclose, but Xu discloses wherein performing the training process comprises: providing, as an input to the image generation model, the training data; receiving, from the image generation model, output image data; using a loss function, computing a loss based on the output image data and the training data; and using the loss, optimizing the image generation model to generate images conditioned to the plurality of image features (p. 5, section 0051-p. 6, section 0059; training data is supplied to the CAGAN which is an image generation model; the model outputs a modified image and loss is determined between the output image and a ground truth modified image from the training data; the model learns, loss is reduced over time, and an optimized model is created to better generate images with the correct features). Motivation for the combination is similar to that given in the rejection to claim 5.
As to claim 9, Joaquim does not disclose, but Xu discloses wherein a contribution of each training image in the plurality of training images to an optimization loss of the training process is based on a corresponding image caption and a corresponding image feature (p. 5, section 0051-p. 6, section 0059; the training involves loss contribution based on differences in natural language embeddings, which would read on captions, and differences between corresponding features in output images compared with ground truth images). Motivation for the combination is similar to that given in the rejection to claim 5.
As to claim 16, see the rejection to claim 5.
As to claim 17, see the rejection to claim 7.
As to claim 20, see the rejection to claim 5.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Joaquim in view of Hamedi (U.S. Publication 2020/0210764).
As to claim 10, Joaquim does not disclose, but Hamadi discloses wherein each training image in the plurality of training images comprises a corresponding image caption stored as a metadata tag (p. 8, section 0072; a caption describing a training image is stored in metadata). The motivation for this is to be able to select images for training based on subject matter rather than characteristics of users. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Joaquim to have each training image in the plurality of training images comprise a corresponding image caption stored as a metadata tag in order to be able to select images for training based on subject matter rather than characteristics of users as taught by Hamedi.
Claims 14 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Joaquim in view of Viehauser and further in view of Good.
As to claim 14, see the rejections to claims 2 and 3. Motivation for the combinations is similar to that given in the rejections to claims 2 and 3.
As to claim 19, see the rejections to claims 2 and 3. Motivation for the combinations is similar to that given in the rejections to claims 2 and 3.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON M RICHER whose telephone number is (571)272-7790. The examiner can normally be reached 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON M RICHER/Primary Examiner, Art Unit 2617
Prosecution Insights