DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
1. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
2. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
3. Claims 1-5, and 8-15 are rejected under 35 U.S.C. 103 as being unpatentable over Fabien Danieau (“Automatic Generation and Stylization of 3D Facial Rigs”, 2019), and further in view of Chang et al. (US 2023/0326136 A1).
4. As per claim 1, Danieau discloses: A method of simulating a three-dimensional (3D) avatar, comprising:
obtaining image data comprising a first face mesh and an operator converting a first reference face mesh into a second reference face mesh; (Danieau, section 3.5:” The goal of this step is to register the generic face mesh to the reconstructed one. This will allow to move the vertices of the generic mesh to make its geometry like the reconstructed mesh (see Figure 6). Using the approach of Sumner et al. [35], we morph the generic mesh to the reconstructed mesh by solving per-vertex affine transformation. The landmarks, computed previously, constraint the optimization process which corresponds to an iterative closest point algorithm (ICP) with regularization. The triangle correspondence is computed and for each vertex of the generic face mesh, we have the corresponding point on the photogrammetry mesh. This point, which is not a vertex, is expressed in barycentric coordinates.”)
generating a second conversion face mesh using the first face mesh, the first reference face mesh, and the operator; (Danieau, section 3.5:” Figure 6: The generic mesh (center) is registered on to the raw photogrammetry mesh (left). Result of the retargeting is shown on the right. Texture is also transferred to the retargeted mesh using correspondence based on barycentric coordinates and continuous texture.”)
generating a second face mesh using the second conversion face mesh and the second reference face mesh; and applying the second face mesh to a 3D avatar. (Danieau section 3.7, :” Finally, the morphed facial mesh has to be merged back to the body (more precisely to the head, see Figure 2). While there still is a vertex to vertex correspondence between the two meshes (the topology has been preserved), the scale and the geometry of the face has changed. Hence a method to merge the two meshes is required. First, a rigid transformation is computed to align the reconstructed mesh to the generic face one [36]. The computation is based on the landmarks of the two meshes. The merge between the reconstructed face and the hood is based on the method proposed by Deng et al. [10]. The smoothing is however performed differently since we want to keep the border of the hood. Artifacts are often generated at the edge of the forehead because of the hairs (see Figure 9). They are smoothed by aligning the tangents of the mesh boundary to the ones of the forehead. This step may create a hole between the hood and the forehead. The hood is vertically adjusted with an FDD box to remove the distance between the forehead and the hood [31].”, and fig. 9)
5. Danieau discloses: The operator, the second conversion face mesh representing displacements between the second reference face mesh and the second face mesh;
6. Chang discloses: The operator, the second conversion face mesh representing displacements between the second reference face mesh and the second face mesh; (Chang, [0017], “Based on the first mesh data and the displacement map, the one or more processors may generate second mesh data characterizing a second mesh of the image.”)
7. Chang is analogous art with respect to Danieau because they are from the same field of endeavor, namely image processing. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to include the process of that the operator, the second conversion face mesh representing displacements between the second reference face mesh and the second face mesh.” as taught by Chang into the teaching of Danieau. The suggestion for doing so would generate output data characterizing an aligned mesh. Therefore, it would have been obvious to combine Danieau with Chang.
8. As per claim 2, Danieau in view of Chang discloses: The method of claim 1, wherein the second face mesh and the second reference face mesh are for simulating the 3D avatar. (Danieau, Abstract: ” In this paper, we present a fully automatic pipeline for generating and stylizing high geometric and textural quality facial rigs.”, and Section 5, “We presented a fully automatic pipeline for generating high-quality facial rigs. From a set of input photos and a generic full-body character, this pipeline outputs a fully rigged character ready to be integrated into any real-time engine or other 3D application in less than seven minutes.”)
9. As per claim 3, Danieau in view of Chang discloses: The method of claim 1, wherein the operator comprises: a function that enhances a first topology of the first face mesh to a second topology of the second face mesh, based on a mapping function that is based on barycentric coordinates of 3D positions of the first reference face mesh and 3D positions of the second reference face mesh. (Danieau, Section 3.5:” The landmarks, computed previously, constraint the optimization process which corresponds to an iterative closest point algorithm (ICP) with regularization. The triangle correspondence is computed and for each vertex of the generic face mesh, we have the corresponding point on the photogrammetry mesh. This point, which is not a vertex, is expressed in barycentric coordinates. Using this correspondence, we transfer the blendshapes from the original generic to the morphed generic with preservation of the connectivity between triangles [34]. Since people are more sensitive to changes around the eyes and the mouth [6], we also include the high-level facial feature lines which enable to better transfer the intensity of the blendshapes [43]. Blendshapes transfer can be performed in exactly three minutes for 102 blendshapes. This set can be reduced for a VR usage. Results are presented on Figure 7.”)
10. As per claim 4, Danieau in view of Chang discloses: The method of claim 1, wherein the generating of the second conversion face mesh comprises: applying position differences between the first face mesh and the first reference face mesh to the operator. (Danieau, Section 3.3:” To simplify computation, the viewport is captured using an orthographic camera. The predicted 2D landmarks are back projected to the facial mesh in the 3D viewer by ray-triangle (or raypoint) intersection algorithm. To get better jaw line landmarks, we also run the DAN algorithm on both side views, left and right. As the prediction of their positions is more precise and accurate on the side views, these are the values we trust. Positions of the other landmarks (eyes, eyebrow, nose, mouth and chin) are taken from the prediction of the front-view picture.”
12. As per claim 5, Danieau in view of Chang discloses: The method of claim 1, further comprising:
generating, based on a first texture map of the first face mesh, a second texture map of the second face mesh; and applying the second texture map to the 3D avatar. (Danieau, Section 4.2, “ With texS the style texture (i.e. the non-human texture), texC the content texture (i.e. the human texture), texSav the style average texture, and texSC the output. The non-human texture is the starting point of the output texture. Enforcing the relative style becomes a global loss and there is no longer any reason to use a content loss. Individual features are thus transferred, such as the skin tone, facial hair and wrinkles, as depicted on Figure 17.”)
13. As per claim 8, Danieau in view of Chang discloses: The method of claim 1, further comprising: applying skin tone information in the image data to the 3D avatar. (Danieau, Section 4.2, “ With texS the style texture (i.e. the non-human texture), texC the content texture (i.e. the human texture), texSav the style average texture, and texSC the output. The non-human texture is the starting point of the output texture. Enforcing the relative style becomes a global loss and there is no longer any reason to use a content loss. Individual features are thus transferred, such as the skin tone, facial hair and wrinkles, as depicted on Figure 17.”)
14. As per claim 9, Danieau discloses: The method of claim 1, wherein the applying to the 3D avatar comprises: providing a 3D avatar customizing function to the 3D avatar applied with the second face mesh. (Danieau, Section 4.2,” We use here the texture of a CG character having an artificially flawless skin. Hence facial features such as hair, scars or wrinkles are transferred. The style loss function of the neural network is modified as follow, to minimize the relative style difference argmin((style(texS)−style(texSav))wstyle −(style(textSC)−style(texC)))2 (2) With texS the style texture (i.e. the non-human texture), texC the content texture (i.e. the human texture), texSav the style average texture, and texSC the output. The non-human texture is the starting point of the output texture. Enforcing the relative style becomes a global loss and there is no longer any reason to use a content loss. Individual features are thus transferred, such as the skin tone, facial hair and wrinkles, as depicted on Figure 17”).
15. As per claim 10, Danieau in view of Chang discloses: The method of claim 1, wherein the image data comprises: a 3D face mesh and texture map extracted from at least one of a two-dimensional (2D) image, a 3D image, or a customized 3D image. (Danieau, Section 4.3.2 Protocol)
16. Claims 11-15, which are similar in scope to claim 1, thus rejected under the same rationale.
17. Claims 6, and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Fabien Danieau (“Automatic Generation and Stylization of 3D Facial Rigs”, 2019), and in view of Chang et al. (US 2023/0326136 A1), and further in view of Cernigliaro et al. (US 20190114822 A1).
18. As per claim 6, Danieau in view of Chang discloses: The method of claim 5, (See rejection of claim 5 above.)
19. Danieau doesn’t expressly disclose: wherein the generating of the second texture map comprises:
obtaining a first UV map of the first texture map;
calculating second UV coordinates based on barycentric coordinates of first UV coordinates of the first UV map;
generating a second UV map corresponding to the second face mesh based on the second UV coordinates; and
generating the second texture map based on the second UV map.
20. Cernigliaro discloses: the generating of the second texture map comprises:
obtaining a first UV map of the first texture map; (Cernigliaro, [0004], ”UV mapping is the process of generating (e.g., from the list of surface elements) a list of correspondences along with an image (i.e., the texture map) such that the color of a surface element is equal to the color of its corresponding pixel in the texture map..”)
calculating second UV coordinates based on barycentric coordinates of first UV coordinates of the first UV map; (Cernigliaro, [0011], “Since the surface is usually represented by a mesh, the correspondence is usually determined by first determining, for each 3D vertex (x.sub.n, y.sub.n, z.sub.n) of each face of the mesh, a corresponding 2D point (u.sub.n, v.sub.n) on the texture map. Then, the 2D point (u, v) corresponding to a point (x, y, z) on a face is obtained by calculating the barycentric coordinates of (x, y, z) with respect to the vertices of the face. This correspondence constitutes a UV map.”)
generating a second UV map corresponding to the second face mesh based on the second UV coordinates; (Cernigliaro, [0012],” Once the texture map and the UV map are determined (e.g., by solving the “inverse” problem), they can be used for texture mapping (e.g., to solve the “forward” problem, such as when texturing the surface of a 3D model of the original 3D object). The solution to the (“inverse”) UV mapping problem is often not well-defined. That is, given the colors S on the surface , there can be many choices of 1 and f such that C(x, y, z)=I(f(x, y, z)).”, and [0015], “Then, chart parametrization may include assigning a 2D point (u.sub.n, v.sub.n) to each 3D vertex (x.sub.n, y.sub.n, z.sub.n) of each face of each sub-mesh. Then, the atlas becomes a texture map, which can be used to re-color the surface.”) and
generating the second texture map based on the second UV map. (Cernigliaro, [0015], “Charting and chart parametrization are often done jointly, and such a joint process is sometimes called atlas parametrization. Atlas parametrization is thus a form of UV mapping. As used herein, “atlas” refers both to a texture map of charts, and also to a texture map of charts in combination with the UV map that underlies it; that is, atlas refers to a representation of the result or output or solution of atlas parametrization or the UV mapping problem.”)
21. Cernigliaro is analogous art with respect to Danieau in view of Chang because they are from the same field of endeavor, namely image processing. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to include the process of installing the processes of:” obtaining a first UV map of the first texture map; calculating second UV coordinates based on barycentric coordinates of first UV coordinates of the first UV map; generating a second UV map corresponding to the second face mesh based on the second UV coordinates; and generating the second texture map based on the second UV map.” as taught by Cernigliaro into the teaching of Danieau in view of Chang. The suggestion for doing so would improves the final result. Therefore, it would have been obvious to combine Danieau in view of Chang with Cernigliaro.
22. As per claim 7, Danieau in view of Chang, and in view of Cernigliaro discloses: The method of claim 6, further comprising: applying masking information of the image data to the 3D avatar based on the second UV map. (Danieau, Section 3.8.1: ”To evaluate the quality of the reconstruction, we ran our pipeline under various conditions. The pipeline was evaluated until the registration and blendshape transfer step (Section 3.5). The aim of this benchmark is to determine the minimal configuration (i.e. number of cameras and image resolution) that provides the best visual facial mask that can be merged to the generic body.”)
Response to Arguments
23. Applicant’s arguments with respect to claims 1-15 filed 01/17/2026 have been considered but are moot because Applicant submitted new amended claims. Accordingly, new grounds of rejection are set forth above. The new grounds of rejection conclusion have been necessitated by Applicant's amendments to the claims.
Conclusion
24. Applicants amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABDERRAHIM MEROUAN whose telephone number is (571)270-5254. The examiner can normally be reached on Monday to Friday 7:30 AM to 5:00 PM.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABDERRAHIM MEROUAN/Supervisory Patent Examiner, Art Unit 2683
Prosecution Insights