CTNF 18/678,221 CTNF 95852 Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 112 07-30-02 AIA The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. The term “first quality level “and “a second quality level higher than the first quality level” in claim 1 is a relative term. The term “first quality level “and “a second quality level higher than the first quality level” is not defined by the claim, Although the specification describes examples of low-quality and high-quality images, claim 1 does not identify how image quality is determined or compared, such as by resolution, blur, noise, artifact level, perceptual score, or any other objective criterion. As a result, the claim leaves unclear when an input image satisfies the claimed first quality level and when a restored image satisfies the claimed second quality level that is higher than the first. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-103 AIA The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action. 07-21-aia AIA Claim s 1-4, and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Nie et al. (US 20240104698 A1) in view of Zhang et al. (US 20220130017 A1) . Regarding claim 1, Nie et al. teaches a method comprising: obtaining an input image depicting an entity ( see para [0080]; “an image can be received 402 (or otherwise obtained)”, see also para [0056]; “such as input image 102 including a representation of at least one object”); adding noise to the input image based on the first quality level to obtain an intermediate noise image ( see para [0080]; “noise (e.g., Gaussian noise) can be added 406 to this image…..each of these forward iterations can generate a version of this input image that has more noise than a prior version….. a diffused image is generated”); and generating, using an image generation model, a restored image depicting the entity by denoising the intermediate noise image ( see para [0081]; “this diffused image can then be used as an input image for a reverse diffusion process…… an amount of noise (e.g., Gaussian noise or random noise) can be removed 410 from this image… a purified image can be generated 412”, see also para [0591]; “wherein the neural network is a diffusion network”). However, Nie et al. does not teach and having a first quality level wherein the restored image has a second quality level higher than the first quality level. In the same field of endeavor, Zhang et al. teaches and having a first quality level ( see claim 2; “acquiring a first medical image having a first image quality, wherein the first medical image is an MR medical image of a subject”), wherein the restored image has a second quality level higher than the first quality level ( see claim 2; “wherein the second medical image has a second image quality, and wherein the second image quality is higher than the first image quality”, see also claim 10; “wherein the second image quality has greater SNR, higher resolution, or less aliasing compared with the first image quality”). Accordingly, it would have been obvious to one of ordinary skill inthe art before the effective filling date to the claimed invention to modify the general use of techniques presented to remove unintended variations introduced into data of Nie rt al. in view of systems that can address various drawbacks of conventional systems of Zhang in order to improving image quality with shortened acquisition time (see claim 2 ). Regarding claim 2, the rejection of claim 1 is incorporated herein. Nie et al. in the combination further teaches wherein the denoising is performed based on the selected timestep ( see para [0060]; “a diffusion network can first add noise to adversarial input utilizing a forward SDE process 212 with a small diffusion timestep. In at least one embodiment, noise can be added over a determined number of iterations or diffusion timesteps …such a process can utilize a selected diffusion timestep, which represents an amount of noise added during forward process 212. In at least one embodiment, this amount of noise is selected to be high enough to remove adversarial perturbations but not so high as to destroy label semantics, or other such features or aspects, of purified images”), wherein generating the restored image comprises: selecting a timestep for the image generation model based on the first quality level, ( see para [0060]; “a diffusion network can first add noise to adversarial input utilizing a forward SDE process 212 with a small diffusion timestep. In at least one embodiment, noise can be added over a determined number of iterations or diffusion timesteps. In at least one embodiment, a reverse SDE process 214 can be utilized to recover, or reconstruct, a clean or purified image from a diffused adversarial image 204”). Regarding claim 3, the rejection of claim 2 is incorporated herein. Nie et al. in the combination further teach wherein generating the restored image comprises: iteratively removing noise from the intermediate noise image based on the selected timestep ( see para [0059]; “a generative process in which a version of this image is generated that has at least some noise removed relative to a prior image in this noise removal sequence. In at least one embodiment, these iterations can continue to be performed until this process arrives at a “purified” image 206 that is substantially free of noise”). Regarding claim 4, the rejection of claim 2 is incorporated herein. Nie et al. in the combination further teach wherein: the selected timestep is based on the first quality level of the input image ( see para [0060]; “such a process can utilize a selected diffusion timestep, which represents an amount of noise added during forward process 212. In at least one embodiment, this amount of noise is selected to be high enough to remove adversarial perturbations but not so high as to destroy label semantics, or other such features or aspects, of purified images”). Regarding claim 9, the rejection of claim 1 is incorporated herein. Zhang et al. in the combination further teaches wherein: the restored image has a higher image quality than the input image ( see claim 2; “applying a deep network model to the first medical image to generate a second medical image, wherein the second medical image has a second image quality, and wherein the second image quality is higher than the first image quality”) . 07-21-aia AIA Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Nie et al. in view of Zhang et al. as applied in claim 1 above, and further in view of Gandelsman et al. (US 20240161462 A1.) Regarding claim 5, the rejection of claim 1 is incorporated herein. The combination Nie et al. and Zhang et al. as a whole does not teach wherein: the image generation model has a constrained latent space based on training using at least one training image depicting the entity. In the same field of endeavor, Gandelsman et al. teach wherein: the image generation model has a constrained latent space based on training using at least one training image depicting the entity ( see para [0004]; “a pre-trained diffusion model is tuned based on a single target image so that the output of the model consistently maintains similarities to the target image”, see also para [0006]; “fine-tuning a pre-trained diffusion model based on a single image to obtain a tuned diffusion model”, and [0078]; “forward process for a latent diffusion model, the model maps an observed variable x.sub.0 (either in a pixel space or a latent space) intermediate variables x.sub.1, . . . , x.sub.T using a Markov chain”). Accordingly, it would have been obvious to one of ordinary skill inthe art before the effective filling date to the claimed invention to modify the general use of techniques presented to remove unintended variations introduced into data of Nie rt al. in view of systems that can address various drawbacks of conventional systems of Zhang and image generation using machine learning of Gandelsman et al. in order to generate a modified image based on the image and the encoded text prompt (see para [0004]) . 07-21-aia AIA Claim s 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Nie et al. in view of Zhang et al. as applied in claim 1 above, and further in view of Varanka et al. NPL “PFStorer: Personalized Face Restoration and Super-Resolution” . Regarding claim 6, the rejection of claim 1 is incorporated herein. The combination Nie et al. and Zhang et al. as a whole does not teach wherein: the restored image is generated without providing an image as guidance to an intermediate stage of the image generation model. In the same field of endeavor, Varanka et al. teaches wherein: the restored image is generated without providing an image as guidance to an intermediate stage of the image generation model ( see page 2375, Fig. 3; “During training the reference image is randomly sampled from a set of reference images for each training iteration. During inference, no reference images are required as the identity is learned in the personalization blocks as a neural representation ”). Accordingly, it would have been obvious to one of ordinary skill inthe art before the effective filling date to the claimed invention to modify the general use of techniques presented to remove unintended variations introduced into data of Nie rt al. in view of systems that can address various drawbacks of conventional systems of Zhang and personalized face restoration and super-resolution of Varanka et al. in order to improve the training pipeline of face restoration models to enable an alignment-free approach (see page 2375, Fig. 3). Regarding claim 7, the rejection of claim 1 is incorporated herein. Varanka et al. in the combination further teaches further comprising: generating a synthetic image depicting the entity, wherein the image generation model is trained based on the synthetic image ( see page 2375, 3.2; “During the personalization fine-tuning, the model takes as input a synthesized LQ image ILQ”, see also page 2377, left col. 3 rd para; “Synthetic Noise Generation In order to generate LQ images for training,” ). Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date to the claimed invention to modify the general use of techniques presented to remove unintended variations introduced into data of Nie rt al. in view of systems that can address various drawbacks of conventional systems of Zhang and personalized face restoration and super-resolution of Varanka et al. in order to generate high-quality image while mostly retaining the identity (see page 2375, 3.2). Regarding claim 8, the rejection of claim 1 is incorporated herein. Varanka et al. in the combination further teaches wherein: the restored image preserves an identity of the entity from the input image ( see page 2378, 4.1; “CodeFormer is able to output a high-quality image while mostly retaining the identity. Our method is able to retain even small details such as the wrinkles and skin texture ”). Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date to the claimed invention to modify the general use of techniques presented to remove unintended variations introduced into data of Nie rt al. in view of systems that can address various drawbacks of conventional systems of Zhang and personalized face restoration and super-resolution of Varanka et al. in order to generate both the qualitative results and quantitative metrics (see page 2378, 4.1) . 07-21-aia AIA Claim s 10-11, and 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US 20240161327 A1) in view of Nie et al . Regarding claim 10, Chen et al. teaches a method for training a machine learning model, comprising: obtaining a training set including a training image depicting an entity ( see para [0125]; “a training image can be identified, where the training image includes at least one object. A training dataset can include a plurality of high-resolution training images of variable sizes. The training image(s) can include a ground truth, where the ground truth indicates an image with a predefined resolution and scale” Note: depicting an entity implies object ); generating a noisy image and guidance information based on the training image ( see para [0006]; “obtaining a noise map and a global image code encoded from a training image and representing semantic content of the training image”, Note: global image code is encoded from the training image to represent its semantic content, acting as the conditioning/guidance); and training an image generation model ( see claim 9; “training the diffusion model to generate the plurality of image patches”) , wherein the image generation model is trained using the noisy image, the training image, and the guidance information (see para [0006]; “obtaining a noise map and a global image code encoded from a training image… generating a plurality of predicted image patches based on the noise map and the global image code… and training the diffusion model to generate image patches by updating the parameters based on the loss function ”, Note: the neural network looks at the noisy input and attempts to predict the noise or clean the patches at each step, using the guidance (global code)). Chen et al. does not explicitly disclose to generate a restored image depicting the entity based on an input image depicting the entity. In the same field of endeavor, Nie et al. teaches to generate a restored image depicting the entity based on an input image depicting the entity ( see para [0082]; “a first image of an object is received…. this image can be processed to add 504 noise this first image…. this added noise can then be removed”, see also para [0081]; “a purified image can be generated” Note; the process adds noise to corrupt the original information, and then uses a trained model to remove that noise, resulting in a cleaner (“restored") image). Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date to the claimed invention to modify the general use of techniques presented to remove unintended variations introduced into data of Nie rt al. in view of generating high-resolution images using a diffusion model of Cheng et al. in order to produce an output image including the semantic content (see para [0082]). Regarding claim 11, the rejection of claim 10 is incorporated herein. Chen et al. in the combination further teaches wherein obtaining the training set comprises: generating the training image based on the input image ( see para [0004]; “The diffusion model can be trained with input images at various resolution”, see also para [0041]; “the diffusion model 260 can be a stable diffusion model, which can be used as the base generative model”). Regarding claim 14, the rejection of claim 10 is incorporated herein. Chen et al. in the combination further teaches wherein training the image generation model comprises: generating a noise prediction based on the noisy image; computing a diffusion loss based on the noise prediction and the training image; and updating parameters of the image generation model based on the diffusion loss ( see para [0039]; “the training component 240 can provide a training set of images having different resolutions, and calculate a loss for differences between a ground truth image and a predicted image. The loss can be used to update the diffusion model parameters and/or encoder parameters, so as to generate more accurate output image(s), where the output image(s) can have a different resolution than the input image(s)”, Note; the input and output images have different resolutions. This implies it is a specialized training mechanism (such as super-resolution) that handles training images of varying resolutions and includes updating "encoder parameters" in addition to standard diffusion parameters, which is common in Latent Diffusion Models). Regarding claim 15, the rejection of claim 10 is incorporated herein. Chen et al. in the combination further teaches wherein: the noise prediction is generated based on the guidance information ( see para [0080]; “an encoder 760 that can be configured and trained to generate an embedding of the initial image 605 that captures the image identity and summarizes the image content, as a global image code”, Note: the mechanism (encoder 760) used to create that "guidance information"). Regarding claim 16, the rejection of claim 10 is incorporated herein. Chen et al. in the combination further teaches wherein training the image generation model comprises: performing a diffusion process at a first timestep using the guidance information; and performing a diffusion process at a second timestep without the guidance information ( see para [0043]; “The model takes as input x (i.e., noisy or partially denoised image depending on the timestep), the timestep t, and conditioning information the model was trained to use. In some cases, the conditioning information can be a text prompt (e.g., TP, “ ”, and AP are text prompts). Classifier-free guidance is a mechanism to vary and control the influence of the conditioning on the sampled distribution at inference. In some cases, the conditioning can be replaced by the null token (i.e., the empty string, “ ”, in case of text conditioning) during training. A single scalar can control effect of the conditioning during inference” Note: using the guidance implies running the model with conditioning (e.g., text prompt), without the guidance implies running the model with a null token (empty string) and Classifier-free guidance connects these two separate runs (with/without guidance) to produce a combined, guided result) . 07-21-aia AIA Claim s 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. in view of Nie et al. as applied in claim 10 above, and further in view of Varanka et al . Regarding claim 12, the rejection of claim 10 is incorporated herein. the combination of Nie et al. and Zhang et al. as a whole does not teach wherein obtaining the training set comprises: obtaining a real image of the entity other than the input image Varanka et al. teaches wherein obtaining the training set comprises: obtaining a real image of the entity other than the input image ( see page 2373, right col. 2 nd para; “Given a few HQ reference images (e.g. 3-5 selfies from a photo gallery) a restoration model is fine-tuned to a personalized restoration model. The reference images can have significantly different illumination, pose, expression and do not have to be aligned with the LQ image”). Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date to the claimed invention to modify the general use of techniques presented to remove unintended variations introduced into data of Nie rt al. in view of generating high-resolution images using a diffusion model of Cheng et al and personalized face restoration and super-resolution of Varanka et al. in order to improve the training pipeline of face restoration models to enable an alignment-free approach (see page 2375, Fig. 3). Regarding claim 13, the rejection of claim 10 is incorporated herein. Varanka et al. in the combination further teaches further comprising: initializing the image generation model based on a pre-trained image generation model ( see page 2376, 3.3; “We train our model with the facial dataset FFHQ using the steps described below, which is initialized from the pre-trained Sta bleSR”). Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date to the claimed invention to modify the general use of techniques presented to remove unintended variations introduced into data of Nie rt al. in view of generating high-resolution images using a diffusion model of Cheng et al and personalized face restoration and super-resolution of Varanka et al. in order to generate high-quality image while mostly retaining the identity (see page 2375, 3.2) . 07-21-aia AIA Claim s 17-20 rejected under 35 U.S.C. 103 as being unpatentable over Gandelsman et al. view of Nie et al . Regarding claim 17, Gandelsman et al. teaches an apparatus comprising: at least one processor; at least one memory storing instructions executable by the at least one processor ( see para [0007]; “One or more aspects of the apparatus and method include one or more processors; one or more memories including instructions executable by the one or more processors to obtain an image and a prompt for editing the image”); and an image generation model, comprising parameters stored in the at least one memory ( see para [0062]; “An apparatus and method for image generation are described.. fine-tune a pre-trained diffusion model based on the image to obtain a tuned diffusion model; and generate a modified image based on the image and the prompt using the tuned diffusion model”), wherein the input image is combined with a noise input to obtain a noisy image ( see para [0110]; “adding noise at the different noise levels to the single image to obtain a plurality of noisy images”, see laso para [0022]; “Next, a reverse diffusion process gradually removes the noise from the noisy images at the various noise levels to obtain an output image”), and wherein the image generation model is trained using a training image depicting the entity ( see para [00045]; “a pre-trained diffusion model is tuned based on a single target image so that the output of the model consistently maintains similarities to the target image”, see also para [0030]; “the input image is an image of a specific person”, and para [0037]; “training component 220 trains and fine-tunes the diffusion model 235. In some examples, training component 220 trains the diffusion model 235 based on a diverse training set and fine-tunes the diffusion model 235 based on a single target image”). However, Gandelsman et al. does not teach and trained to generate a restored image based on an input image depicting an entity, wherein the restored image is generated based on the noisy image. In the same field of endeavor, Nei et al. teaches and trained to generate a restored image based on an input image depicting an entity ( see para [0059]; “generative process in which a version of this image is generated that has at least some noise removed relative to a prior image in this noise removal sequence. In at least one embodiment, these iterations can continue to be performed until this process arrives at a “purified” image 206 that is substantially free of noise, that is substantially free of adversarial perturbations, and has sufficient structure to enable an accurate classification of an object in this purified image”, see also para [0063]; “purified image 306 can be passed through a trained classifier 308 that can generate a correct label 310 for an object in purified image 306” ), wherein the restored image is generated based on the noisy image ( see para [0081]; “removing noise can be accomplished by generating a new version of this image at each reverse iteration that has less noise than a prior version of this image. In at least one embodiment, this process can be continued until a final iteration, in which a purified image can be generated”). Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date to the claimed invention to modify the general use of image generation using machine learning of Gandelsman et al. in view of techniques presented to remove unintended variations introduced into data of Nie et al. in order to generate a modified image based on the image and the encoded text prompt (see para [0059]). Regarding claim 18, the rejection of claim 17 is incorporated herein. Gandelsman et al. in the combination further teaches wherein: the image generation model comprises a diffusion model ( see Abstract; “A diffusion model is tuned based on the image to generate different versions of the image”). Regarding claim 19, the rejection of claim 17 is incorporated herein. Gandelsman et al. in the combination further teaches wherein: the image generation model comprises a U-Net architecture ( see para [0012]; “a diffusion model using a U-Net”). Regarding claim 20, the rejection of claim 17 is incorporated herein. Gandelsman et al. in the combination further teaches further comprising: an output space of the image generation model is constrained to images depicting the entity based on the training ( see para [0024]; “The present disclosure uses diffusion models to generate images that retain the identity of a target image (i.e., recognizable characteristics that are not captured in a semantic label)”) Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to WINTA GEBRESLASSIE whose telephone number is (571)272-3475. The examiner can normally be reached Monday-Friday9:00-5:00. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at 571-270-5180. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /WINTA GEBRESLASSIE/Examiner, Art Unit 2677 Application/Control Number: 18/678,221 Page 2 Art Unit: 2677 Application/Control Number: 18/678,221 Page 3 Art Unit: 2677 Application/Control Number: 18/678,221 Page 4 Art Unit: 2677 Application/Control Number: 18/678,221 Page 5 Art Unit: 2677 Application/Control Number: 18/678,221 Page 6 Art Unit: 2677 Application/Control Number: 18/678,221 Page 7 Art Unit: 2677 Application/Control Number: 18/678,221 Page 8 Art Unit: 2677 Application/Control Number: 18/678,221 Page 9 Art Unit: 2677 Application/Control Number: 18/678,221 Page 10 Art Unit: 2677 Application/Control Number: 18/678,221 Page 11 Art Unit: 2677 Application/Control Number: 18/678,221 Page 12 Art Unit: 2677 Application/Control Number: 18/678,221 Page 13 Art Unit: 2677 Application/Control Number: 18/678,221 Page 14 Art Unit: 2677 Application/Control Number: 18/678,221 Page 15 Art Unit: 2677 Application/Control Number: 18/678,221 Page 16 Art Unit: 2677
Prosecution Insights