Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 12,175,641. Although the claims at issue are not identical, they are not patentably distinct from each other because the present claims are either anticipated by or obvious variants of the patent claims. The following table shows the corresponding limitations between representative claims and representative patent claims.
Present Application Claims
Patent claims
1. A method comprising: generating, utilizing a defect detection neural network, a defect segmentation mask distinguishing between local-defect pixels and non-local-defect pixels of a digital image; determining, utilizing an inpainting model to process the local-defect pixels of the digital image, replacement pixels that reduce visual appearance of a local defect by replacing a portion the local-defect pixels within the digital image; and generating, utilizing the inpainting model, a modified digital image by inpainting the local defect using the replacement pixels replacing the portion of the local-defect pixels.
8. A non-transitory computer readable medium storing instructions which, when executed by a processing device, cause the processing device to perform operations comprising: detecting, using a face enhancement neural network, a face depicted in a digital image; projecting, using the face enhancement neural network, the face depicted in the digital image into a latent space; determining, from the latent space, a latent code corresponding to the face depicted in the digital image; and generating, utilizing the face enhancement neural network, an enhanced digital image by reconstructing pixels from the latent code corresponding to the face depicted in the digital image.
15. A system comprising: a memory component; and one or more processing devices coupled to the memory component, the one or more processing devices to perform operations comprising: generating, utilizing a defect detection neural network, a defect segmentation mask distinguishing between local-defect pixels and non-local-defect pixels of a digital image; generating, utilizing an inpainting model, a modified digital image by inpainting one or more of the local-defect pixels in the digital image; projecting, using a face enhancement neural network, the modified digital image into a latent space; and generating, using the face enhancement neural network, an enhanced digital image from the modified digital image by reconstructing face pixels from a latent code in the latent space.
Dependent claims 2-7, 9-14, 16-20 are similarly rejected.
Claim 3.
1. A non-transitory computer readable medium comprising instructions that, when executed by at least one processor, cause a computing device to: generate, utilizing a defect detection neural network, a segmentation mask indicating one or more local defects localized to specific portions of a digital image; determine pixels from the digital image for filling the one or more local defects indicated by the segmentation mask utilizing an inpainting model; generate a modified digital image by inpainting the one or more local defects utilizing the determined pixels from the inpainting model; and extract, utilizing a global correction neural network, a feature vector from the modified digital image depicting one or more global imperfections comprising one or more of image blur, faded colors, faded saturation, image graininess, or sepia effects.
3. The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the at least one processor, cause the computing device to determine the pixels for filling the one or more local defects by determining pixels within the digital image that reduce appearance of the one or more local defects when used to replace pixels of the one or more local defects utilizing the inpainting model.
Claim 6.
1. A non-transitory computer readable medium comprising instructions that, when executed by at least one processor, cause a computing device to: generate, utilizing a defect detection neural network, a segmentation mask indicating one or more local defects localized to specific portions of a digital image; determine pixels from the digital image for filling the one or more local defects indicated by the segmentation mask utilizing an inpainting model; generate a modified digital image by inpainting the one or more local defects utilizing the determined pixels from the inpainting model; and extract, utilizing a global correction neural network, a feature vector from the modified digital image depicting one or more global imperfections comprising one or more of image blur, faded colors, faded saturation, image graininess, or sepia effects.
6. The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the at least one processor, cause the computing device to: map the modified digital image into a latent space utilizing a face enhancement neural network; determine a latent code corresponding to the modified digital image within the latent space; and generate, from the latent code corresponding to the modified digital image, a third modified digital image utilizing the face enhancement neural network to correct pixels depicting one or more faces.
Claim 6.
1. A non-transitory computer readable medium comprising instructions that, when executed by at least one processor, cause a computing device to: generate, utilizing a defect detection neural network, a segmentation mask indicating one or more local defects localized to specific portions of a digital image; determine pixels from the digital image for filling the one or more local defects indicated by the segmentation mask utilizing an inpainting model; generate a modified digital image by inpainting the one or more local defects utilizing the determined pixels from the inpainting model; and extract, utilizing a global correction neural network, a feature vector from the modified digital image depicting one or more global imperfections comprising one or more of image blur, faded colors, faded saturation, image graininess, or sepia effects.
6. The non-transitory computer readable medium of claim 1, further comprising instructions that, when executed by the at least one processor, cause the computing device to: map the modified digital image into a latent space utilizing a face enhancement neural network; determine a latent code corresponding to the modified digital image within the latent space; and generate, from the latent code corresponding to the modified digital image, a third modified digital image utilizing the face enhancement neural network to correct pixels depicting one or more faces.
Claims 1-20.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 8, 10-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. [“Transforming the Latent Space of StyleGAN for Real Face Editing”], hereinafter Li in view of YAN (US 2022/0261968).
As to claim 8, Li discloses a non-transitory computer readable medium storing instructions which, when executed by a processing device, cause the processing device to perform operations comprising:
projecting, using the face enhancement neural network, the face depicted in the digital image into a latent space (pages Section 4.3, 4.3.1-4.3.2.3, pages 7-9, Fig. 4, e.g., “After projecting real images into the latent space, we perform manipulation in the obtained latent codes for semantic editing”);
determining, from the latent space, a latent code corresponding to the face depicted in the digital image (pages Section 4.3, 4.3.1-4.3.2.3, pages 7-9, Fig. 4, e.g.,“After projecting real images into the latent space, we perform manipulation in the obtained latent codes for semantic editing”; and
generating, utilizing the face enhancement neural network, an enhanced digital image by reconstructing pixels from the latent code corresponding to the face depicted in the digital image (pages Section 4.3, 4.3.1-4.3.2.3, pages 7-9, Fig. 4, e.g., “After projecting real images into the latent space, we perform manipulation in the obtained latent codes for semantic editing”, “Manipulating real faces with respect to the attribute age in different latent spaces. Given a real image to edit, we first invert it back to the latent space using StyleGAN projector and then manipulate the latent code with InterFaceGAN. Our results (highlighted by the red box) achieve considerably stronger robustness for long-distance manipulation”).
Li is silent regarding detecting, using a face enhancement neural network, a face depicted in a digital image.
Yan teaches detecting, using a face enhancement neural network, a face depicted in a digital image (para. 0050, 0066, 0111).
It would have been obvious to one of ordinary skill in the art to incorporate YAN’s teachings into Li since doing so would merely combine prior art elements according to known methods to yield predictable results, and improve face enhancement performance.
As to claim 10, the combination of Li and YAN discloses
10. The non-transitory computer readable medium of claim 8, wherein projecting the face depicted in the digital image comprises mapping pixels of the digital image to the latent space using an encoder of the face enhancement neural network (YAN, page 3, e.g., StyleGAN2Encoder; page 7, e.g., “we use the StyleGANv2’s projector to obtain latent codes for real images”).
As to claim 11, the combination of Li and YAN discloses the non-transitory computer readable medium of claim 8, wherein determining the latent code corresponding to the face depicted in the digital image comprises determining a latent code nearest to a projection of the face depicted in the digital image within the latent space image (YAN, Section 4.3, 4.3.1-4.3.2.3, pages 7-9, Fig. 4, e.g., “After projecting real images into the latent space, we perform manipulation in the obtained latent codes for semantic editing”, “Manipulating real faces with respect to the attribute age in different latent spaces. Given a real image to edit, we first invert it back to the latent space using StyleGAN projector and then manipulate the latent code with InterFaceGAN. Our results (highlighted by the red box) achieve considerably stronger robustness for long-distance manipulation”).
As to claim 12, the combination of Li and YAN discloses the non-transitory computer readable medium of claim 8, wherein generating the enhanced digital image comprises reconstructing the pixels from the latent code by using a decoder of the face enhancement neural network image (YAN, Section 4.3, 4.3.1-4.3.2.3, pages 7-9, Fig. 4, e.g., “After projecting real images into the latent space, we perform manipulation in the obtained latent codes for semantic editing”, “Manipulating real faces with respect to the attribute age in different latent spaces. Given a real image to edit, we first invert it back to the latent space using StyleGAN projector and then manipulate the latent code with InterFaceGAN. Our results (highlighted by the red box) achieve considerably stronger robustness for long-distance manipulation”.
As to claim 13, the combination of Li and YAN discloses the non-transitory computer readable medium of claim 8, wherein the operations further comprise generating the latent space by generating a plurality of latent noise vectors that convert into digital images depicting faces upon processing by the face enhancement neural network (YAN, page 3, ““Using an additive ramped-down noise, the original StyleGANv2 [12] proposes to embed images in the W space which enables better editing at the cost of worse reconstruction. To the contrary, Image2StyleGAN and Image2StyleGAN++ [1, 2] embed images into the extended W+ space which effectively optimizes a separate style for each scale. This approach sacrifices editing quality for reconstruction quality”; Section 4.3, 4.3.1-4.3.2.3, pages 7-9, Fig. 4, e.g., “After projecting real images into the latent space, we perform manipulation in the obtained latent codes for semantic editing”, “Manipulating real faces with respect to the attribute age in different latent spaces. Given a real image to edit, we first invert it back to the latent space using StyleGAN projector and then manipulate the latent code with InterFaceGAN. Our results (highlighted by the red box) achieve considerably stronger robustness for long-distance manipulation”).
As to claim 14, the combination of Li and YAN discloses the non-transitory computer readable medium of claim 13, wherein generating the enhanced digital image comprises: determining a latent noise vector in the latent space closest to a latent vector extracted from the face depicted in the digital image; and reconstructing pixels of the enhanced digital image from the latent noise vector using the face enhancement neural network (YAN, page 3, ““Using an additive ramped-down noise, the original StyleGANv2 [12] proposes to embed images in the W space which enables better editing at the cost of worse reconstruction. To the contrary, Image2StyleGAN and Image2StyleGAN++ [1, 2] embed images into the extended W+ space which effectively optimizes a separate style for each scale. This approach sacrifices editing quality for reconstruction quality”; Section 4.3, 4.3.1-4.3.2.3, pages 7-9, Fig. 4, e.g., “After projecting real images into the latent space, we perform manipulation in the obtained latent codes for semantic editing”, “Manipulating real faces with respect to the attribute age in different latent spaces. Given a real image to edit, we first invert it back to the latent space using StyleGAN projector and then manipulate the latent code with InterFaceGAN. Our results (highlighted by the red box) achieve considerably stronger robustness for long-distance manipulation”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHUOC TRAN whose telephone number is (571)272-7399. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at 571-272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHUOC TRAN/Primary Examiner, Art Unit 2668
Prosecution Insights