DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This Office Action is in response to Applicant’s amendment filed 06/25/2026 which has
been entered and made of record. Claims 1, 10, and 19 have been amended. No
claim has been cancelled or newly added. Claims 1-20 are pending in the application. Applicant’s amendments to the Specification, have overcome each and every objection
previously set forth in the Non-Final Office Action mailed March 3rd 2025.
Response to Arguments
Applicant's arguments filed with respect to claims 1-20, filed on 06/25/2026 with respect
to the rejection under 35 USC 103, have been fully considered but they are not persuasive.
In response to applicant’s argument that, “the transfer process of Mordvintsev is not a transfer of stylistic information included in a map to a three-dimensional model, but rather is a transfer of three-dimensional rendering information to an attribute map that is then used downstream to render a three-dimensional object” examiner respectfully disagrees. As recited in claim 1, these limitations are taught by the combination of MACDONALD and Mordvintsev. In particular, and in addition to the citations in claim 1 below, Mordvintsev teaches transfer of stylistic information due to “texture map can provide texture values for each location of the three-dimensional model. For example, the texture values can include color values such as RGB values, RGBA values, CMYK values, HSV values, HSL values, etc.” in paragraph 29 of Mordvintsev which shows the attribute map would be texture map (due to citation in claim 1 below mentioning “attribute rendering map (e.g., texture map,” as recited in paragraph 26 of Mordvintsev) and here contains color. Both of these (texture and color) are style transfer according to the applicant’s disclosure paragraph 34 which states “Style transfer applies stylistic elements, such as colors, textures, or lighting characteristics”. Therefore, Mordvintsev does teach transfer of stylistic information included in a map to a 3D (three-dimensional) model when Mordvintsev is to "iteratively modify an attribute rendering map (e.g., texture map, bump map, etc.) based on information collected from a different rendering of the model at each of a plurality of iterations" (Mordvintsev, paragraph 26) due to iterative modification and different rendering of model at each iteration. Thus, transfer of 3D rendering information of Mordvintsev that the applicant argues, would be stylistic transfer of the styles which include: color and texture.
In response to applicant’s argument that, “Mordvintsev merely states that the attribute rendering map can be used to render the three-dimensional object, but does not discuss the use of a machine learning model transfer information from these maps to any sort of three-dimensional model” examiner respectfully disagrees. As recited in claim 1, these limitations are taught by the combination of MACDONALD and Mordvintsev. In particular, and in addition to the citations in claim 1 below, Mordvintsev teaches machine learning would be used for the transferring of the aforementioned information from the maps to the 3D model (Mordvintsev paragraph 81 teaches “Each application can include its own style transfer manager. Each application contains its own machine learning library and machine-learned model(s)” and paragraph 26 teaches “methods of the present disclosure can use machine-learned models such as, for example, convolutional neural networks to generate image style and content information which may then be used to perform style transfer.”); examiner would like to note that this is a explicit teaching of using machine learning for style transfer from the aforementioned maps to 3D models. Therefore, the examiner is not persuaded by the applicant’s arguments.
Claim Rejections - 35 USC § 112
Previous 35 U.S.C. 112(b) rejection for claims 1-20 have been withdrawn.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 7-14 and 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over MACDONALD et al. (U.S. Patent Application Publication No. 2022/0108514), hereinafter referenced as MACDONALD in view of Mordvintsev et al. (U.S. Patent Application Publication No. 2019/0228587), hereinafter referenced as Mordvintsev.
Regarding claim 1, MACDONALD teaches A computer-implemented method for performing face micro detail recovery, the computer-implemented method comprising: (abstract teaches "methods... animation of realistic facial performances of avatars are provided. Such an example system includes a memory to store a facial gesture model of a subject head derived from a photogrammetric scan of the subject head" and paragraph 37 teaches "simulate bumps or wrinkles on the surface of the avatar"); this shows facial micro detail such as wrinkles recovery as a computer implemented method; generating one or more skin texture displacement maps based on images of one or more skin surfaces; (paragraph 28 teaches "dynamic texture map 130 is derived at least in part from the video 104 of the subject's face," and paragraph 37 teaches "dynamic texture map 130 may further include...displacement map"); video of subject's face shows images of skin surfaces and the dynamic texture map (including displacement) is based on such thus is considered a skin texture displacement map;
However, MACDONALD fails to explicitly teach transferring, via one or more machine learning models, stylistic elements included in the one or more skin texture displacement maps onto one or more regions included in a modified three-dimensional (3D) facial reconstruction; and generating a final 3D facial reconstruction that includes structural elements included in the modified 3D facial reconstruction and the stylistic elements included in the one or more skin texture displacement maps.
However, Mordvintsev explicitly teaches transferring, via one or more machine learning models, stylistic elements included in the one or more skin texture displacement maps onto one or more regions included in a modified three-dimensional (3D) facial reconstruction (Mordvintsev, paragraph 1 teaches "image style transfer for three-dimensional models through, for example, the use of machine-learned models such as, for example, convolutional neural networks" and paragraph 26 teaches "iteratively modify an attribute rendering map (e.g., texture map, bump map, etc.) based on information collected from a different rendering of the model at each of a plurality of iterations"); style transfer includes stylistic elements (from maps mentioned which would be skin texture displacement map from MACDONALD above when viewed in combination), iteratively modifying and different rendering of model at multiple iterations shows these stylistic elements (from style transfer and attribute rendering map) are transferred onto region(s) of a modified 3D facial reconstruction; and generating a final 3D facial reconstruction that includes structural elements included in the modified 3D facial reconstruction and the stylistic elements included in the one or more skin texture displacement maps (Mordvintsev, paragraph 26 teaches "different rendering of the model at each of a plurality of iterations, with the end result being that the attribute rendering map mimics the style of one or more reference images in a content-preserving way"); end/final iteration (with rendering of model) means a final 3D facial reconstruction, this includes structural elements of 3D facial reconstruction (since paragraph 37 of MACDONALD mentions "dynamic texture map 130 may further include a bump map to simulate bumps or wrinkles on the surface of the avatar" [since wrinkles are structural elements per paragraph 118 of applicant's disclosure]), and it also includes stylistic elements (from skin texture displacement map as explained above) since it is a part of style transfer. Mordvintsev is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of style transfer for 3D reconstruction using machine learning. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify MACDONALD's invention with the style transfer techniques of Mordvintsev to ensure many potential applications can be improved or enhanced through the use of style transfer to a three-dimensional model as described (Mordvintsev, paragraph 60). This would be done by the transfer of details carried out from the style transfer.
Regarding claim 2, the combination of MACDONALD and Mordvintsev teaches wherein the modified 3D facial reconstruction includes one or more simulated skin textures (MACDONALD, paragraph 37 teaches "dynamic texture map 130 may further include a bump map to simulate bumps or wrinkles on the surface of the avatar"); bumps or wrinkles being simulated on surface of avatar shows simulated skin texture(s) and this is included in the 3D facial reconstruction because as aforementioned and when viewed in combination, the dynamic texture map would be used to arrive at the 3D facial reconstruction.
Regarding claim 3, the combination of MACDONALD and Mordvintsev teaches wherein the structural elements include one or more pores and one or more wrinkles (MACDONALD, paragraph 28 teaches "dynamic texture map 130 includes subtle details (e.g. winkles, skin pores)"; these wrinkles and pores are of texture map which is used for the model rendering above thus these are part of those structural elements thereof.
Regarding claim 4, the combination of MACDONALD and Mordvintsev teaches wherein the stylistic elements include one or more skin texture variations defined by the one or more skin texture displacement maps (Mordvintsev, paragraph 50 teaches "transfer of style via optimization of the style loss can be balanced against the preservation of content via optimization of the content loss. To continue the example provided above, the texture map for the model of the zebra can be updated to receive a style transfer, however, it can retain content-descriptive features such as, for example, stripes within the style-transferred texture"); this shows the style transfer (with content-descriptive features / stylistic elements) include skin texture variations such as partial texture from texture map (which is skin texture displacement map when viewed in combination as aforementioned). The same motivations used in claim 1 apply here in claim 4.
Regarding claim 5, the combination of MACDONALD and Mordvintsev teaches wherein generating the final 3D facial reconstruction is based at least on the modified 3D facial reconstruction (Mordvintsev, paragraph 26 teaches "different rendering of the model at each of a plurality of iterations, with the end result being that the attribute rendering map mimics the style of one or more reference images in a content-preserving way"); this shows end result/final 3D reconstruction (in a last iteration) coming after the modified 3D facial reconstruction (previous iteration model) thus based on such; and a user-controllable blending factor (MACDONALD, paragraph 30 teaches "At block 202...subject head performing a series of facial gesture is photogrammetrically scanned. Individual photogrammetric scans are solved and combined into the facial gesture model 108 (i.e., a single blendshape model" and paragraph 35 teaches "one or more of the blocks of the method 200 may be initiated or assisted by a user"); one of ordinary skill in the art would understand that blendshape model require blending weights/factor, user initiating method 200 including block 202 means that the blending factor used to derive the blendshape model is user-controllable and this means the final 3D reconstruction is based on such since the blending here happens in early steps. The same motivations used in claim 1 apply here in claim 5.
Regarding claim 7, the combination of MACDONALD and Mordvintsev teaches wherein the modified 3D facial reconstruction includes a 3D mesh of triangles or other polygons (MACDONALD, fig. 8A shows a polygon mesh and Mordvintsev, paragraph 27 teaches "three-dimensional model (e.g., mesh model)"); this means 3D facial reconstruction includes 3D polygonal mesh. The same motivations used in claim 1 apply here in claim 7.
Regarding claim 8, the combination of MACDONALD and Mordvintsev teaches wherein each of the one or more machine learning models includes a convolutional neural network or a generative adversarial network (Mordvintsev, paragraph 1 teaches "image style transfer for three-dimensional models through, for example, the use of machine-learned models such as, for example, convolutional neural networks"); this shows the machine learning model includes convolutional neural network. The same motivations used in claim 1 apply here in claim 8.
Regarding claim 9, the combination of MACDONALD and Mordvintsev teaches wherein the modified 3D facial reconstruction includes depictions of one or more of a mouth, a nose, eyes, or eyebrows (MACDONALD, paragraph 50 teaches "orifice region of the dynamic texture map 130 is applied to the orifice mesh. That is, the mouth map 742 is applied to the mouth mesh 702, or the eye map(s) 842 is/are applied to the eye mesh(es) 802."); this shows eyes and mouth and this is included in the 3D facial reconstruction because as aforementioned and when viewed in combination, the dynamic texture map would be used to arrive at the 3D facial reconstruction.
Regarding claim 10, the non-transitory computer-readable media claim 10 recites similar limitations as method claim 1, and thus is rejected under similar rationale. In addition, paragraph 29 of MACDONALD teaches “programming instructions stored on a non-transitory machine-readable storage medium executable by a processor of a computing device”.
Regarding claim 11, the non-transitory computer-readable media claim 11 recites similar limitations as method claim 2, and thus is rejected under similar rationale.
Regarding claim 12, the non-transitory computer-readable media claim 12 recites similar limitations as method claim 3, and thus is rejected under similar rationale.
Regarding claim 13, the non-transitory computer-readable media claim 13 recites similar limitations as method claim 4, and thus is rejected under similar rationale.
Regarding claim 14, the non-transitory computer-readable media claim 14 recites similar limitations as method claim 5, and thus is rejected under similar rationale.
Regarding claim 16, the non-transitory computer-readable media claim 16 recites similar limitations as method claim 7, and thus is rejected under similar rationale.
Regarding claim 17, the non-transitory computer-readable media claim 17 recites similar limitations as method claim 8, and thus is rejected under similar rationale.
Regarding claim 18, the non-transitory computer-readable media claim 18 recites similar limitations as method claim 9, and thus is rejected under similar rationale
Regarding claim 19, the system claim 19 recites similar limitations as method claim 1, and thus is rejected under similar rationale. In addition, fig. 1 of MACDONALD shows system 110 with memory 120 and abstract teaches “instructions to be executed by a processor”.
Regarding claim 20, the system claim 20 recites similar limitations as method claim 4, and thus is rejected under similar rationale
Claim(s) 6 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of MACDONALD and Mordvintsev as applied to claims 1 and 10 above, and further in view of Litwinowicz (U.S. Patent No. 5,995,110), hereinafter referenced as Litwinowicz.
Regarding claim 6, the combination of MACDONALD and Mordvintsev fails to teach further comprising receiving user input defining a correspondence between one of the one or more skin texture displacement maps and at least one of the one or more regions.
However, Litwinowicz teaches further comprising receiving user input defining a correspondence between one of the one or more skin texture displacement maps and at least one of the one or more regions. (Litwinowicz, claim 25 teaches "a user input mechanism which enables a user to designate points of correspondence on each of the displayed texture image and the displayed surface representation"); this texture image point is part of skin texture displacement map when viewed in combination because it's of texture and as aforementioned the dynamic texture map from above includes the displacement map, also the correspondence here is to surface representation of 3D object (region(s)) as defined by user input. Litwinowicz is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of using user input for correspondence between skin texture displacement and regions. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of MACDONALD and Mordvintsev with the user input defining correspondence techniques of Litwinowicz so it is desirable to place texture on three-dimensional objects, to enhance surface detail (Litwinowicz, col. 1, lines 11-13). This would be due to the user being able to define correspondence leading to a better and more tailored surface detail for the user and also increasing engagement with the program due to further user interaction.
Regarding claim 15, the non-transitory computer-readable media claim 15 recites similar limitations as method claim 6, and thus is rejected under similar rationale.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NAUMAN U AHMAD whose telephone number is (703)756-5306. The examiner can normally be reached Monday - Friday 9:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
/N.U.A./Examiner, Art Unit 2611
Prosecution Insights