DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-9, 11-19 rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1-9, 11-19 of U.S. Patent No. 12,260,483. Although the claims at issue are not identical, they are not patentably distinct from each other because the notion of the claims does refer to the same invention and claim 1 of the current application corresponds with claim 1 of U.S. Patent No. 12,260,483. Claim 1 of U.S. Patent No. 12,260,483 anticipates claim 1 of the current application because it includes all of the limitations of claim 1.
Below is a limitation mapping between claim 1 of the current application and claim 1 of U.S. Patent No. 12,260,483.
Current Application
12,260,483
1. An apparatus comprising: a host interface; one or more processors coupled with the host interface, the one or more processors including a graphics processor, wherein the one or more processors are configured to execute instructions stored in a memory, the instructions to cause the one or more processors to:
1. An apparatus comprising: a host interface; and one or more processors coupled with the host interface, the one or more processors including a graphics processor, wherein the one or more processors are configured to execute instructions stored in a memory, the instructions to cause the one or more processors to:
process, via at least one machine learning model, at least one input image;
generate, via at least one machine learning model, a set of latent codes to estimate a surface of a material for an object in a scene, wherein the set of latent codes includes an illumination latent code to represent lighting information for the surface of the material, and reflectance latent codes to represent surface reflectance for the material;
generate, via the at least one machine learning model, a set of latent codes for an object within the at least one input image, wherein the set of latent codes includes an illumination latent code to represent lighting information for the object, and reflectance latent codes to represent surface reflectance for the object;
and render a three-dimensional representation of the object via a real-time renderer by shading a mesh via sampled latent codes from the set of latent codes to represent an effect of the material.
and render a three-dimensional representation of the object via a real-time renderer by shading a mesh via sampled latent codes from the set of latent codes.
Below is part 1 of claim mapping between the current application and U.S. Patent No. 12,260,483
Current Application
1
2
3
4
5
6
7
8
9
11
12
13
U.S. Patent No. 12,260,483
1
2
3
4
5
6
7
8
9
11
12
13
Below is part 2 of claim mapping between the current application and U.S. Patent No. 12,260,483
Current Application
14
15
16
17
18
19
U.S. Patent No. 12,260,483
14
15
16
17
18
19
Claims 1-4, are rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1-6 of U.S. Patent No. 11,972,519 in view of Surti et al. (US 2020/0301597)(Hereinafter referred to as Surti). U.S. Patent No. 11,972,519 teaches all of the limitations except “a host interface”
Surti teaches a host interface connecting GPGPU to a host processor (The GPGPU 1430 includes a host interface 1432 to enable a connection with a host processor. In one embodiment the host interface 1432 is a PCI Express interface. However, the host interface can also be a vendor specific communications interface or communications fabric. The GPGPU 1430 receives commands from the host processor and uses a global scheduler 1434 to distribute execution threads associated with those commands to a set of compute clusters 1436A-1436H. The compute clusters 1436A-1436H share a cache memory 1438. The cache memory 1438 can serve as a higher-level cache for cache memories within the compute clusters 1436A-1436H. See paragraph [0136]).
U.S. Patent No. 11,972,519 and Suri teach of processing graphics data and Surti teaches that a host interface allows the system to enable a connection with the host processor, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of U.S. Patent No. 11,972,519 with the host interface capabilities of Surti such that processors could communicate effectively with each other.
Claims 11-13 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 11, 12 of U.S. Patent No. 11,972,519. Although the claims at issue are not identical, they are not patentably distinct from each other because the notion of the claims does refer to the same invention and claim 11 of the current application corresponds with claims 11 and 12 of U.S. Patent No. 11,972,519.
Claims 11-13 of the current application are merely a different statutory category to claims 11 and 12 of U.S. Patent No. 11,972,519. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the statutory category of claims 11 and 12 of U.S. Patent No. 11,403,875 from a method to a non-transitory computer readable medium as it would achieve predictable results.
Claim mapping between claim 1 of the current application and claims 1 and 2 of U.S. Patent No. 11,972,519.
Current application
U.S. Patent No. 11,972,519
1. An apparatus comprising: a host interface; one or more processors coupled with the host interface, the one or more processors including a graphics processor, wherein the one or more processors are configured to execute instructions stored in a memory, the instructions to cause the one or more processors to:
1. A data processing system comprising: a memory to store instructions; and one or more processors including a graphics processor, the graphics processor including circuitry to execute the instructions, wherein the instructions cause the circuitry to:
process, via one or more machine learning models, a set of input images;
generate, via at least one machine learning model, a set of latent codes to estimate a surface of a material for an object in a scene, wherein the set of latent codes includes an illumination latent code to represent lighting information for the surface of the material, and reflectance latent codes to represent surface reflectance for the material;
generate, via the one or more machine learning models, an illumination latent code and a set of reflectance latent codes for an object within the set of input images;
and render a three-dimensional representation of the object via a real-time renderer by shading a mesh via sampled latent codes from the set of latent codes to represent an effect of the material.
generate a shader based on a machine learning model of the one or more machine learning models, the shader configured to sample the illumination latent code and the set of reflectance latent codes for the object; and render a three-dimensional (3D) object by shading a mesh via the shader.
2. The data processing system as in claim 1, wherein the circuitry is configured to render the 3D object via a real-time renderer.
Current Application
1
2
3
4
11
12
13
U.S. Patent No. 11,972,519
2, 1
2, 1
6, 5, 4, 3, 2, 1
2, 1
12, 11
12, 11
12, 11
Allowable Subject Matter
Claims 1-9-11-19 would be allowable if rewritten or amended to overcome the rejection(s) under double patenting, set forth in this Office action by filing a terminal disclaimer or amending the claims as appropriate.
Claims 10 and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: The closest prior art of record is Lombardi et al. (“Deep Appearance models for Face Rendering”, 2018)(Hereinafter referred to as Lombardi).
Regarding claim 1, Lombardi teaches An apparatus (We introduce a deep appearance model for rendering the human face. Inspired by Active Appearance Models, we develop a data-driven rendering pipeline that learns a joint representation of facial geometry and appearance from a multiview capture setup. Vertex positions and view-specific textures are modeled using a deep variational autoencoder that captures complex nonlinear effects while producing a smooth and compact latent representation. View-specific texture enables the modeling of view-dependent effects such as specularity. See abstract) (Architecture of the Encoder and Decoder. The textures Tμ t and Tvt are 3-channel 1024 × 1024 images. Each convolutional layer has stride 2 and the number of channels doubles after the first convolution every two layers. We combine the texture and mesh subencodings via concatenation. The decoder runs these steps in reverse: we split the network into two branches and use transposed convolutions of stride 2 to double the image resolution at every step. This decoder network executes in less than 5 milliseconds on an NVIDIA GeForce GTX 1080 GPU. See figure 3)(General purpose computer with GPU and CPU) comprising: a host interface; one or more processors coupled with the host interface, the one or more processors including a graphics processor, wherein the one or more processors are configured to execute instructions stored in a memory, the instructions to cause the one or more processors (Architecture of the Encoder and Decoder. The textures Tμ t and Tvt are 3-channel 1024 × 1024 images. Each convolutional layer has stride 2 and the number of channels doubles after the first convolution every two layers. We combine the texture and mesh subencodings via concatenation. The decoder runs these steps in reverse: we split the network into two branches and use transposed convolutions of stride 2 to double the image resolution at every step. This decoder network executes in less than 5 milliseconds on an NVIDIA GeForce GTX 1080 GPU. See figure 3 and caption)(General purpose computer with GPU and CPU) to:
generate, via at least one machine learning model, a set of latent codes to estimate a surface of a material for an object in a scene (Fig. 1. Our model jointly encodes and decodes geometry and view-dependent appearance into a latent code z, from data captured from a multi-camera rig, enabling highly realistic data-driven facial rendering. We use this rich data to drive our avatars from cameras mounted on a head-mounted display (HMD). We do this by creating synthetic HMD images through image-based rendering, and using another variational autoencoder to learn a common representation y of real and synthetic HMD images. We then regress from y to the latent rendering code z and decode into mesh and texture to render. Our method enables high-fidelity social interaction in virtual reality. See figure 1 and caption); and
and render a three-dimensional representation of the object via a real-time renderer by shading a mesh via sampled latent codes from the set of latent codes to represent an effect of the material. (The decoder network must execute in less than 11.1 milliseconds to achieve 90Hz rendering for real-time VR systems. We are able to achieve this using transposed strided convolutions even with a final texture size of 1024 × 1024. This is a major departure from most previous work for generating facial imagery that has been limited to significantly smaller output sizes. See page 68:5, right col.)( Our primary goal is to create extremely high-fidelity facial models that can be built automatically from a multi-camera capture setup and rendered and driven in real time in VR (90Hz). In achieving this goal, we avoid using hand-crafted models or priors, and instead rely on the rich data we acquired from our multiview capture apparatus. See section 4, first paragraph), but is silent to wherein the set of latent codes includes an illumination latent code to represent lighting information for the surface of the material, and reflectance latent codes to represent surface reflectance for the material.
The prior art of record alone or in combination is silent to the limitations “wherein the set of latent codes includes an illumination latent code to represent lighting information for the surface of the material, and reflectance latent codes to represent surface reflectance for the material.” Of claim 1 when read in light of the rest of the limitations in claim 1 and thus claim 1 contains allowable subject matter.
The prior art of record alone or in combination is silent to the limitations “wherein the set of latent codes includes an illumination latent code to represent lighting information for the surface of the material, and reflectance latent codes to represent surface reflectance for the material,” Of claim 11 when read in light of the rest of the limitations in claim 11 and thus claim 11 contains allowable subject matter.
Claims 2-10 and 12-20 contain allowable subject matter because they depend on a claim that contains allowable subject matter.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Bradley et al. (US 2023/0252714), generally relates to appearance reconstruction based on a set of parameters using a neural network (See Abstract).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS R WILSON whose telephone number is (571)272-0936. The examiner can normally be reached M-F 7:30-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (572)-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICHOLAS R WILSON/Primary Examiner, Art Unit 2611
Prosecution Insights