Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-6, 8-9, 11-13, 15-16, 18, 20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-2, 4, 6-9, 11, 14-16, 20 of U.S. Patent No. 12266039.
US App. #19064948
1
2
3
4
5
6
8
9
11
12
13
15
16
18
20
U.S. Patent # 12266039
1
1
6
2
4
7
8
8
9
11
14
15
15
16
20
US App. #19064948 claim 1
U.S. Patent # 12266039 claim 1
1. A method comprising:
accessing, by a processor, a material generation prior that includes a statistical function;
encoding, by the processor and based on the material generation prior, an input material
appearance from an input material map to produce a projected latent vector;
generating, by the processor, a modified value of the projected latent vector by modifying a current value of the projected latent vector to reduce a statistical difference between a target image and a renderable image, the renderable image being associated with the current value of the projected latent vector, and the target image including a target material appearance;
generating, by the processor, an output material map based on the modified value of the projected latent vector; and
rendering, by the processor, a scene based on the output material map, wherein the output material map provides the target material appearance to the scene.
1. A method comprising:
accessing a scene, a target image including a target material appearance, and an input material map defining a coarse structure of the scene;
accessing a material generation prior produced using a generative adversarial network (GAN), wherein the material generation prior includes a statistical function;
encoding, based on the material generation prior and using the GAN, an input material appearance from the input material map to produce a projected latent vector;
optimizing a current value of the projected latent vector based on the material generation prior to minimize a statistical difference between the target image and a renderable image associated with the current value of the projected latent vector; and
rendering, responsive to the optimizing, the scene based on a final value of the projected latent vector and an output material map providing the target material appearance applied to the coarse structure of the scene.
US App. #19064948 claim 2
U.S. Patent # 12266039 claim 1
2. The method of claim 1,
wherein the material generation prior is produced by a generative adversarial network (GAN).
1. A method comprising:
accessing a scene, a target image including a target material appearance, and an input material map defining a coarse structure of the scene;
accessing a material generation prior produced using a generative adversarial network (GAN), wherein the material generation prior includes a statistical function;
encoding, based on the material generation prior and using the GAN, an input material appearance from the input material map to produce a projected latent vector;
optimizing a current value of the projected latent vector based on the material generation prior to minimize a statistical difference between the target image and a renderable image associated with the current value of the projected latent vector; and
rendering, responsive to the optimizing, the scene based on a final value of the projected latent vector and an output material map providing the target material appearance applied to the coarse structure of the scene.
US App. #19064948 claim 3
U.S. Patent # 12266039 claim 6
3. The method of claim 2, further comprising training the GAN with a dataset of synthetic
material maps, wherein the training of the GAN is separate from the generating of the modified value of the projected latent vector.
6. The method of claim 1, further comprising training the GAN with a dataset of synthetic material maps, wherein the training of the GAN is separate from the optimizing of the current value of the projected latent vector.
US App. #19064948 claim 4
U.S. Patent # 12266039 claim 2
4. The method of claim 1, wherein generating the modified value of the projected latent vector
involves minimizing a loss function including a style loss and a feature description loss.
2. The method of claim 1, wherein optimizing the current value of the projected latent vector further comprises minimizing, using an embedded latent code, a loss function including a style loss and a feature description loss.
US App. #19064948 claim 5
U.S. Patent # 12266039 claim 4
5. The method of claim 4, wherein the style loss is determined using a Wasserstein loss.
4. The method of claim 2, wherein the style loss is defined using sliced Wasserstein loss.
US App. #19064948 claim 6
U.S. Patent # 12266039 claim 7
6. The method of claim 1, wherein the target material appearance comprises color or shading.
7. The method of claim 1, wherein the target material appearance comprises at least one of color or shading.
US App. #19064948 claim 8
U.S. Patent # 12266039 claim 8
8. A system comprising:
a processing device; and
a memory component coupled to the processing device, the memory component storing instructions that are executable by the processing device to perform operations including:
encoding, based on a material generation prior that includes a statistical function, an input material appearance from an input material map to produce a projected latent vector;
generating a modified value of the projected latent vector by modifying a current value of the projected latent vector to reduce a statistical difference between a target image and a
renderable image, the renderable image being associated with the current value of the projected
latent vector, and the target image including a target material appearance; and
storing or rendering a scene based on the modified value of the projected latent vector to provide the target material appearance to the scene.
8. A system comprising:
a memory component; and
a processing device coupled to the memory component, the processing device to perform operations comprising:
encoding, based on a material generation prior from a generative adversarial network (GAN), an input material appearance from an input material map to produce a projected latent vector, the input material map defining a coarse structure of a scene, and the material generation prior including a statistical function;
optimizing a current value for the projected latent
vector based on the material generation prior to
minimize a statistical difference between a target
image including a target material appearance and a renderable image associated with the current value of the projected latent vector; and
storing or rendering, responsive to the optimizing, the scene based on a final value of the projected latent vector and an output material map augmented by the target material appearance applied to the coarse structure of the scene.
US App. #19064948 claim 9
U.S. Patent # 12266039 claim 8
9. The system of claim 8,
wherein the material generation prior is produced by a generative adversarial network (GAN).
8. A system comprising:
a memory component; and
a processing device coupled to the memory component, the processing device to perform operations comprising:
encoding, based on a material generation prior from a generative adversarial network (GAN), an input material appearance from an input material map to produce a projected latent vector, the input material map defining a coarse structure of a scene, and the material generation prior including a statistical function;
optimizing a current value for the projected latent
vector based on the material generation prior to
minimize a statistical difference between a target
image including a target material appearance and a renderable image associated with the current value of the projected latent vector; and
storing or rendering, responsive to the optimizing, the scene based on a final value of the projected latent vector and an output material map augmented by the target material appearance applied to the coarse structure of the scene.
US App. #19064948 claim 11
U.S. Patent # 12266039 claim 9
11. The system of claim 8, wherein generating the modified value of the projected latent vector involves minimizing a loss function including a style loss and a feature description loss.
9. The system of claim 8, wherein optimizing the current value of the projected latent vector further comprises minimizing, using an embedded latent code, a loss function including a style loss and a feature description loss.
US App. #19064948 claim 12
U.S. Patent # 12266039 claim 11
12. The system of claim 11, wherein the style loss is determined using a Wasserstein loss.
11. The system of claim 9, wherein the style loss is defined using sliced Wasserstein loss.
US App. #19064948 claim 13
U.S. Patent # 12266039 claim 14
13. The system of claim 8, wherein the target material appearance comprises color or shading.
14. The system of claim 8, wherein the target material appearance comprises at least one of color or shading.
US App. #19064948 claim 15
U.S. Patent # 12266039 claim 15
15. A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising:
encoding, based on a material generation prior that includes a statistical function, an input material appearance from an input material map to produce a projected latent vector;
a step for generating a modified value of the projected latent vector; and
rendering a scene based on the modified value of the projected latent vector to provide a target material appearance to the scene.
15. A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the
processing device to perform operations
comprising:
accessing a scene, a target image including a target material appearance, and an input material map defining a coarse structure of the scene;
encoding, based on a material generation prior produced by a generative adversarial network (GAN), an input material appearance from the input material map to produce a projected latent vector, wherein the material generation prior includes a statistical function;
a step for producing an optimized value for the projected latent vector based on the material generation prior; and
rendering the scene based on the optimized value for the projected latent vector and an output material map augmented by the target material appearance applied to the coarse structure of the scene.
US App. #19064948 claim 16
U.S. Patent # 12266039 claim 15
16. The non-transitory computer-readable medium of claim 15,
wherein the material generation prior is produced by a generative adversarial network (GAN).
15. A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the
processing device to perform operations
comprising:
accessing a scene, a target image including a target material appearance, and an input material map defining a coarse structure of the scene;
encoding, based on a material generation prior produced by a generative adversarial network (GAN), an input material appearance from the input material map to produce a projected latent vector, wherein the material generation prior includes a statistical function;
a step for producing an optimized value for the projected latent vector based on the material generation prior; and
rendering the scene based on the optimized value for the projected latent vector and an output material map augmented by the target material appearance applied to the coarse structure of the scene.
US App. #19064948 claim 18
U.S. Patent # 12266039 claim 16
18. The non-transitory computer-readable medium of claim 15, wherein the step for generating the modified value of the projected latent vector involves minimizing a loss function including a
style loss and a feature description loss.
16. The non-transitory computer-readable medium of claim 15, wherein the step for producing the optimized value for the projected latent vector further comprises minimizing, using an embedded latent code, a loss function including a style loss and a feature description loss.
US App. #19064948 claim 20
U.S. Patent # 12266039 claim 20
20. The non-transitory computer-readable medium of claim 15,
wherein the operations further comprise executing an iterative process, wherein each iteration of the iterative process involves:
determining a current value for the projected latent vector;
generating an intermediate material map based on the current value for the projected latent vector;
determining a statistical difference between a target image and a renderable image corresponding to the intermediate material map; and
adjusting the current value for the projected latent vector based on the statistical difference.
20. The non-transitory computer-readable medium of claim 16,
wherein the step for producing the optimized value for the projected latent vector comprises an iterative process, and wherein each iteration of the iterative process involves:
determining a current value for the projected latent vector;
generating an intermediate material map based on the current value for the projected latent vector;
determining a statistical difference between the target image and a renderable image corresponding to the intermediate material map; and
adjusting the current value for the projected latent vector based on the statistical difference.
Although the claims at issue are not identical, they are not patentably distinct
from each other. For example, Claim 1 of Patent 12266039 discloses "optimizing a current value of the projected latent vector based on the material generation prior to minimize a statistical difference" while Application 19064948 Claim 1 discloses "generating, by the processor, a modified value of the projected latent vector by modifying a current value of the projected latent vector to reduce a statistical difference". Optimizing projected latent vector includes modifying value of projected latent vector. Therefore, Patent 12266039 Claim 1 discloses all limitations of Application 19064948 Claim 1.
Allowable Subject Matter
Claims 1-20 are allowable if the double patenting rejection is overcome.
Regarding Claim 1, it recites “A method comprising:
accessing a scene, a target image including a target material appearance, and an input material map defining a coarse structure of the scene;
accessing a material generation prior produced using a generative adversarial network (GAN), wherein the material generation prior includes a statistical function;
encoding, based on the material generation prior and using the GAN, an input material appearance from the input material map to produce a projected latent vector;
optimizing a current value of the projected latent vector based on the material generation prior to minimize a statistical difference between the target image and a renderable image associated with the current value of the projected latent vector; and
rendering, responsive to the optimizing, the scene based on a final value of the projected latent vector and an output material map providing the target material appearance applied to the coarse structure of the scene.” in the context of Claim 1.
Xu et al (CN112419334), abstract, the invention discloses a micro-surface material reconstruction method and system based on deep learning. The micro-surface material reconstruction method comprises the steps: acquiring a shot image of a real world material; inputting the shot image of the material into a pre-training generator network, and outputting a diffuse reflection map, a reflection map, a roughness map and a normal map which have higher resolution than the shot image; and drawing a rendered image by using the chartlet, wherein the appearance of the material presented by the rendered image is similar to that of the shotimage. A rendering image is drawn from a diffuse reflection map, a reflection map, a roughness map and a normal map by using a drawing module in a neural network on the basis of an adversarial generative network framework, and the rendering image and a real material shot picture are discriminated by using a discriminator module. Therefore, the defect that a traditional machine learning method needs to depend on a large number of material mapping labels is avoided, and the difficulty of material collection is reduced.
Lipton et al ("Precise recovery of latent vectors from generative adversarial networks, 2017), abstract, the paper describes a simple, gradient-based technique called stochastic clipping. In experiments, for images generated by the GAN, we precisely recover their latent vector pre-images 100% of the time. Additional experiments demonstrate that this method is robust to noise. Finally, we show that even for unseen images, our method appears to recover unique encodings.
Chen (US20190251713), abstract, the invention describes a system and method for reconstructing an image of a subject acquired using a tomographic imaging system includes at least one computer processor configured to form an image reconstruction pipeline. The reconstruction pipeline at least includes an automated correction module configured to receive imaging data acquired from a subject using ionizing radiation generated by the tomographic imaging system and generate corrected data using a first learning network. The reconstruction pipeline also includes an intelligent reconstruction module configured to receive at least one of the imaging data and the corrected data and reconstruct an image of the subject using a second learning network.
The prior arts of record either alone or in combination fails to teach or suggest the above quoted limitation of Claim 1. Therefore, Claim 1 is allowable over prior art.
Claims 2-7 depend from Claim 1 with respective additional limitations. Therefore, Claims 2-7 are allowable over prior art.
Claim 8 recites similar limitations as discussed above with regard to claim 1. Therefore, claim 8 is allowable over prior art.
Claims 9-14 depend from Claim 8 with respective additional limitations. Therefore, Claims 9-14 are allowable over prior art.
Claim 15 recites similar limitations as discussed above with regard to claim 1. Therefore, claim 15 is allowable over prior art.
Claims 16-20 depend from Claim 15 with respective additional limitations. Therefore, Claims 16-20 are allowable over prior art.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Venkataraman (US20220215266), abstract, the invention teaches a method of generating synthetic images of virtual scenes includes: placing, by a synthetic data generator implemented by a processor and memory, three-dimensional (3-D) models of objects in a 3-D virtual scene; adding, by the synthetic data generator, lighting to the 3-D virtual scene, the lighting including one or more illumination sources; applying, by the synthetic data generator, imaging modality-specific materials to the 3-D models of objects in the 3-D virtual scene in accordance with a selected multimodal imaging modality, each of the imaging modality-specific materials including an empirical model; setting a scene background in accordance with the selected multimodal imaging modality; and rendering, by the synthetic data generator, a two-dimensional image of the 3-D virtual scene based on the selected multimodal imaging modality to generate a synthetic image in accordance with the selected multimodal imaging modality.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIN SHENG whose telephone number is (571)272-5734. The examiner can normally be reached M-F 9:30AM-3:30PM 6:00PM-8:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 5712723022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Xin Sheng/Primary Examiner, Art Unit 2619
Prosecution Insights