DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 07/01/2026 has been entered.
Response to Arguments
Applicant's arguments filed 07/01/2026 have been fully considered.
Regarding to claim 1, the applicant argues that cited arts fail to teaches or suggest "the first object has a concave-convex surface;" "determining material parameter information of the coating according to the light source information and a normal map of the first image;" "the normal map is configured to improve a detail level of a lighting direction and a reflection effect presented on the concave-convex surface;" and "determining a second image by rendering the material on a second object distinct from the first object”. The arguments have been fully considered. The argument according to "the normal map is configured to improve a detail level of a lighting direction and a reflection effect presented on the concave-convex surface;" is persuasive. Therefore, the 35 U.S.C 103 rejection has been withdrawn. However, upon further consideration, new grounds of rejection are made in newly applied art. The argument according to other claim limitations is not persuasive. The examiner cannot concur with the applicant for following reasons:
Laine discloses “the first object has a concave-convex surface”. For example, in Fig. 1A and paragraph [0021], Laine teaches a set of 2D images of an object 110 are human faces and contain a concave-convex surface as illustrated in Fig. 1A;
PNG
media_image1.png
212
232
media_image1.png
Greyscale
; Laine further teaches the 3D model 130 is a base model that is deformed to produce a specific 3D model corresponding to each of the 2D images in the set; Laine furthermore teaches the 3D model 130 of a human head contains concave part and convex part as illustrated in Fig. 1A;
PNG
media_image2.png
296
254
media_image2.png
Greyscale
. In Fig. 1B, and paragraph [0026], Laine teaches producing a rendered image 115 that matches the reference image 112;
PNG
media_image3.png
388
494
media_image3.png
Greyscale
; Laine further teaches a concave-convex surface as illustrated in Fig. 1B.
Li discloses “determining material parameter information of the coating according to the light source information and a normal of the first image”. For example, in paragraph [0038], Li teaches the swapping technique enforces consistency of the texture images across the frames; Li further teaches the loss function Le enforces consistency between the texture images. In Fig. 2B and paragraph [0053], Li teaches the propagated part map for each frame is mapped to the UV texture space with the predicted texture flow 234 to produce part UV map 236. In paragraph [0141], Li teaches one or more vertex attributes include color, texture coordinates, surface normal, etc.; Li further teaches the vertex shading stage 620 manipulates individual vertex attributes such as position, color, texture coordinates, and the like; modify color attributes for a vertex; Li furthermore teaches a normalized-device-coordinate space.
Li further teaches “wherein the material parameter information comprises at least one of color, texture, smoothness, transparency, reflectivity, refractive index, and luminosity of the material”. For example, in paragraph [0021], Li teaches recovery of the 3D shape, texture, and camera pose of objects from 2D images; Li further teaches texture is a coating of a material. In paragraph [0028], Li teaches different faces defined by the vertices in the mesh are colored either black or white. n Fig. 1 and paragraph [0029], Li teaches the UV texture image; Li further teaches the material parameter information includes the texture flow maps pixels and a texture image 106; Li further more teaches applying the texture image 106, i.e. a 2D coating, to the 3D shape representation 108, i.e. an object, to produce a 3D object 104, e.g., textured mesh and a second image;
PNG
media_image4.png
550
442
media_image4.png
Greyscale
.
In addition, Laine discloses “wherein the material parameter information comprises at least one of color, texture, smoothness, transparency, reflectivity, refractive index, and luminosity of the material”. For example, in Fig. 1A and paragraph [0021], Laine teaches global surface texture defines lighting and materials properties 120;
PNG
media_image5.png
560
530
media_image5.png
Greyscale
. In paragraph [0038], Laine teaches a proper differentiable renderer that provides gradients for all the parameters, e.g., lighting and material parameters, as well as the contents of texture maps used in the process. In paragraph [0077], Laine teaches the surface texture represents lighting and material properties of the 3D model.
Li furthermore discloses “determining a second image by rendering the material on a second object distinct from the first object, based on the material parameter information and the editing parameter information”. For example, in Fig. 1A and paragraph [0029], Li further more teaches applying the texture image 106, i.e. a 2D coating, to the 3D shape representation 108, i.e. an object, to produce a 3D object 104, e.g., textured mesh and a second object;
PNG
media_image6.png
434
656
media_image6.png
Greyscale
; Li further teaches a predicted 3D object 104, i.e. second object, is distinct from the 3D shape representation 108 and the texture image 106 as illustrated in Fig. 1A;
PNG
media_image7.png
164
142
media_image7.png
Greyscale
;
PNG
media_image8.png
98
262
media_image8.png
Greyscale
. In Fig. 1B and paragraph [0035], Li teaches the texture image is transferred onto the 3D shape representation to produce the 3D object, i.e. a second image, corresponding to the object in the first image; Li further teaches when the 3D object is rendered according to the camera pose for each frame of video, the rendered object appears as the object in the frame. In paragraph [0038], Li teaches a texture invariance constraint maybe used to encourage consistent texture reconstruction from all frames; Li further teaches the swapping technique enforces consistency of the texture images across the frames; Li furthermore teaches the loss function Le enforces consistency between the texture images. In Fig. 2B and paragraph [0054], Li teaches the wrapped 3D shape representation 237 is rendered by the differentiable renderer 222 according to the predicted camera to produce rendered image 238. In Fig. 2D and paragraph [0072], Li teaches the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence.
Claims 2-13 and 15-21 are not allowable due to a newly applied art and the similar reasons as discussed above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-13 and 15-21 are rejected under 35 U.S.C. 103 as being unpatentable over Li (US 20220036635 A1) in view of Laine (US 20220051481 A1), and further in view of Porcino (US 20150348316 A1).
Regarding to claim 1 (Currently amended), Li discloses a method of drawing an image (Fig. 1A; [0025]: the 3D object construction system 100 includes a neural network model comprising at least an encoder 105, shape decoder 115, and motion decoder 120; the encoder 105 extracts features 110 from each frame, e.g., image, in the video; [0029]: the texture decoder 125 receives the features 110 and predicts a texture image 106 for each frame of the video; [0032]: construct a 3D representation of an object using the 3D object reconstruction system 100; [0033]: the 3D object construction system 100 receives a video including images of the object captured from a camera pose), comprising:
obtaining a first image (Fig. 1A; [0025]: receive each frame, e.g., image, in the video;
PNG
media_image9.png
446
670
media_image9.png
Greyscale
; Fig. 1B; [0033]: the 3D object construction system 100 receives a video including images of the object captured from a camera pose), wherein the first image comprises a coating of a material, the coating is a substance coated on a first object, and the coating serves as a carrier of the material ([0021]: recovery of the 3D shape, texture, and camera pose of objects from 2D images; texture is a coating of a material; Fig. 1A; [0025]: an input image 102 includes an object with a coating of a materials, such as texture, white or black color as illustrated in Fig. 1A;
PNG
media_image10.png
246
382
media_image10.png
Greyscale
; a texture is a 2D coating of a material; [0028]: the different faces defined by the vertices in the mesh are colored either black or white; Fig. 1; [0029]: the UV texture image; apply the texture image 106, i.e. a 2D coating, to the 3D shape representation 108, i.e. an object, to produce a 3D object 104, e.g., textured mesh and second image.
PNG
media_image4.png
550
442
media_image4.png
Greyscale
; [0141]: modify color attributes for a vertex; Fig. 2B; [0050]: the painting appears as differently colored vertical stripes within a silhouette of the object visible in a propagated part map 232);
determining light source information and editing parameter information corresponding to the first image respectively ([0037]: a differentiable renderer renders a texture mesh to an RGB image; [0038]: lighting conditions; [0046]: the differentiable renderer 222 receives the 3D mesh and texture image and renders the 3D mesh; [0141]: the vertex shading stage 620 processes vertex data by performing a set of operations; operations commonly include lighting operations, e.g., modifying color attributes for a vertex; [0147]: perform lighting operations; perform sampling texture maps using interpolated texture coordinates for the fragment; the fragment shading stage 670 generates pixel data, e.g., color values, for the fragment such as by performing lighting operations; [0155]: the rendering of the game session includes ray or path-traced lighting and/or shadow effects, computed using one or more parallel processing units);
determining material parameter information of the coating according to the light source information and a normal of the first image ([0038]: the swapping technique enforces consistency of the texture images across the frames; the loss function Le enforces consistency between the texture images; Fig. 2B; [0053]: the propagated part map for each frame is mapped to the UV texture space with the predicted texture flow 234 to produce part UV map 236; [0141]: one or more vertex attributes include color, texture coordinates, surface normal, etc.; the vertex shading stage 620 manipulates individual vertex attributes such as position, color, texture coordinates, and the like; modify color attributes for a vertex; a normalized-device-coordinate space), wherein the material parameter information comprises at least one of color, texture, smoothness, transparency, reflectivity, refractive index, and luminosity of the material (one of; [0021]: recovery of the 3D shape, texture, and camera pose of objects from 2D images; texture is a coating of a material; [0028]: different faces defined by the vertices in the mesh are colored either black or white; Fig. 1A; [0029]: the UV texture image; the material parameter information includes the texture flow maps pixels and a texture image 106; apply the texture image 106, i.e. a 2D coating, to the 3D shape representation 108, i.e. an object, to produce a 3D object 104, e.g., textured mesh and a second image;
PNG
media_image4.png
550
442
media_image4.png
Greyscale
); and
determining a second image by rendering the material on a second object distinct from the first object, based on the material parameter information and the editing parameter information (Fig. 1A; [0029]: apply the texture image 106, i.e. a 2D coating, to the 3D shape representation 108, i.e. an object, to produce a 3D object 104, e.g., textured mesh and a second object;
PNG
media_image4.png
550
442
media_image4.png
Greyscale
; a predicted 3D object 104, i.e. a second object, is distinct from the 3D shape representation 108 and the texture image 106 as illustrated in Fig. 1A;
PNG
media_image7.png
164
142
media_image7.png
Greyscale
;
PNG
media_image8.png
98
262
media_image8.png
Greyscale
; Fig. 1B; [0035]: the texture image is transferred onto the 3D shape representation to produce the 3D object corresponding to the object in the first image; when the 3D object is rendered according to the camera pose for each frame of video, the rendered object appears as the object in the frame; [0038]: a texture invariance constraint maybe used to encourage consistent texture reconstruction from all frames; the swapping technique enforces consistency of the texture images across the frames; the loss function Le enforces consistency between the texture images; Fig. 2B; [0054]: the wrapped 3D shape representation 237 is rendered by the differentiable renderer 222 according to the predicted camera to produce rendered image 238; Fig. 2D; [0072]: the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence).
Li fails to explicitly disclose:
the first object has a concave-convex surface;
normal map; and
the normal map is configured to improve a detail level of a lighting direction and a reflection effect presented on the concave-convex surface.
In same field of endeavor, Laine teaches:
the first object has a concave-convex surface (Fig. 1A; [0021]: a set of 2D images of an object 110 are human faces and contain a concave-convex surface as illustrated in Fig. 1A;
PNG
media_image1.png
212
232
media_image1.png
Greyscale
; the 3D model 130 is a base model that is deformed to produce a specific 3D model corresponding to each of the 2D images in the set; the 3D model 130 of a head contains concave part and convex part as illustrated in Fig. 1A;
PNG
media_image2.png
296
254
media_image2.png
Greyscale
; Fig. 1B; [0026]: produce a rendered image 115 that matches the reference image 112;
PNG
media_image3.png
388
494
media_image3.png
Greyscale
; a concave-convex surface as illustrated in Fig. 1B);
normal map ([0038-0039]: normal vectors are normal map; the final color is obtained by sampling the result at the pixel center; [0055]: the depth in normalized device coordinates);
wherein the material parameter information comprises at least one of color, texture, smoothness, transparency, reflectivity, refractive index, and luminosity of the material (one of; Fig. 1A; [0021]: global surface texture defines lighting and materials properties 120;
PNG
media_image5.png
560
530
media_image5.png
Greyscale
; [0038]: a proper differentiable renderer provides gradients for all the parameters—e.g., lighting and material parameters, as well as the contents of texture maps used in the process; [0077]: the surface texture represents lighting and material properties of the 3D model.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Li to include the first object has a concave-convex surface; normal map; wherein the material parameter information comprises at least one of color, texture, smoothness, transparency, reflectivity, refractive index, and luminosity of the material as taught by Laine. The motivation for doing so would have been reconstruct the 3D model using highly optimized hardware graphics pipelines; to improve the 3D model; to obtain the final color using normal vectors; to improve accuracy of the 3D model as taught by Laine in paragraphs [0020], [0029], [0038-0039], and [0045].
Li in view of Laine fails to explicitly disclose:
the normal map is configured to improve a detail level of a lighting direction and a reflection effect presented on the concave-convex surface.
In same field of endeavor, Porcino teaches:
the normal map is configured to improve a detail level of a lighting direction and a reflection effect presented on the concave-convex surface ([0009]: a normal vector for each pixel is taken from the surface normal map, read in by a Graphics Processing Unit (GPU), and used to create the appropriate light reflections and shadows on the surface texture; Fig. 1; [0026]: normal mapping is defined as a technique for simulating the appearance of lighting of bumps and dents on a surface texture; normal maps add additional detail to surfaces without using more polygons; [0032]: adjusting the bumpiness” or “smoothness” of the normal map in real-time allows the program or programmer a finer degree of control over the “look and feel” of the rendered 3D effects; Fig. 4; [0035]: the normal map information may be used by the shader to dynamically create specular highlighting and shading effects on the texture map; render equivalent lighting effects on the mixed 2D and 3D scene using the various dynamically generated normal maps; Fig. 5; [0036]: the output, represented by the dynamic 3D rendering with lighting effects 506 reflects the fact that the inferred heights and bumpiness of the texture encapsulated in dynamically generated normal map 504 have resulted in different lighting effects being applied to different parts of the texture 502;
PNG
media_image11.png
146
182
media_image11.png
Greyscale
;
PNG
media_image12.png
122
188
media_image12.png
Greyscale
; [0037]: render the appropriate 3D lighting effects on the input 2D image based on the relevant normal map and height map information from the data structure).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Li in view of Laine to include the normal map is configured to improve a detail level of a lighting direction and a reflection effect presented on the concave-convex surface as taught by Porcino. The motivation for doing so would have been to add additional detail to surfaces by normal maps without using more polygons; to render dynamic 3D lighting effects in a mixed 2D/3D scene; to render equivalent lighting effects on the mixed 2D and 3D scene using the various dynamically generated normal maps; to dynamically generate normal map 504; generate an image with textures and equivalent lighting effects as taught by Porcino in Fig. 4, Fig. 5, and paragraph [0026] and [0035-0036].
Regarding to claim 2 (Previously presented), Li in view of Laine and Porcino discloses the method according to claim 1, wherein obtaining the first image (same as rejected in claim 1) comprises:
obtaining a third image to be used by photographing the first object coated with the coating (Li; Fig. 1A; [0025]: an input image 102 includes an object;
PNG
media_image13.png
240
380
media_image13.png
Greyscale
; Fig. 1B; [0033]: images of the object are captured from a camera pose; the video is captured by a camera); and
obtaining the first image by processing, according to an image processing approach, the first image (Laine; Fig. 1A; [0021]: a set of 2D images of an object 110 are captured from a variety of camera positions; a complete 3D model 130 is constructed for each of one or more of the 2D images in the set);
wherein the first object is presented in the first image at a ratio (Laine; Fig. 1A; [0022]: match the set of 2D images of the object 110; the set of 2D images of the object 110 comprise a video;
PNG
media_image14.png
294
686
media_image14.png
Greyscale
; Fig. 1B; [0026]: produce a rendered image 115 that matches the reference image 112).
Same motivation of claim 1 is applied here.
Regarding to claim 3 (Previously presented), Li in view of Laine and Porcino discloses the method according to claim 2, wherein the first object being presented in the first image at the ratio (same as rejected in claim 2) comprises:
the first image being filled with the first object, and an object edge displayed in the first image being tangent to an edge line of the first image (Laine; Fig. 1B; [0026]: produce a rendered image 115 that matches the reference image 112;
PNG
media_image15.png
412
482
media_image15.png
Greyscale
; [0028]: rendered geometry 136 forms a silhouette edge 138 of an object; Fig. 1C; [0030]: the shaded pixels 141 and 142 appear the same for many different positions and orientations of the edge 138;
PNG
media_image16.png
118
210
media_image16.png
Greyscale
).
Same motivation of claim 1 is applied here.
Regarding to claim 4 (Previously presented), Li in view of Laine and Porcino discloses the method according to claim 1, wherein determining the light source information and the editing parameter information corresponding to the first image respectively (same as rejected in claim 1) comprises:
determining the light source information corresponding to the first image by processing, based on an illumination estimation model obtained by pre-training, the first image (Li; [0038]: the neural network model 150 predicts and determines the texture images 154 and 158 for the frames 132 and 136, respectively; the neural network model 150 is fine-tuned on a particular video with the invariance constraints enforced by Equations; [0040]: the texture images are predicted for each frame by the neural network model 150; Fig. 1D; [0042]: refinement of the neural network model 150 is completed; [0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object); and
obtaining the target editing parameter information corresponding to the first image by processing, based on an editor selection model obtained by pre-training, the first image (Li; [0038]: the neural network model 150 predicts and determines the texture images 154 and 158 for the frames 132 and 136, respectively; the neural network model 150 is fine-tuned on a particular video with the invariance constraints enforced by Equations; [0046]: the differentiable renderer 222 receives the 3D mesh and texture image and renders the 3D mesh according to the predicted camera pose provided by the camera pose unit 225; the textured 3D object is then projected by the differentiable renderer 222; [0071-0072]: the 3D object construction system 100 is trained using self-supervision for videos; the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence; [0141]: the vertex shading stage 620 processes vertex data by performing a set of operations, e.g., a vertex shader).
Regarding to claim 5 (Previously presented), Li in view of Laine and Porcino discloses the method according to claim 4, wherein determining the light source information corresponding to the first image by processing, based on the illumination estimation model obtained by pre-training, the first image (same as rejected in claim 4), comprises:
obtaining pixel coordinate information of a highlight point in the first image output by the illumination estimation model by inputting the first image into the illumination estimation model (Li; [0141]: vertices are specified as a 4-coordinate vector; modify the coordinate space for a vertex; vertices are specified using coordinates in an object-coordinate space); and
determining, based on the pixel coordinate information, the light source information of a light source upon obtaining the first image by photographing (Li; [0141]: lighting operations; modify color attributes for a vertex; [0147]: generate pixel data, e.g., color values, for the fragment such as by performing lighting operations),
Li in view of Laine and Porcino further discloses:
obtaining pixel coordinate information (Laine; [0037]: the x, y pixel coordinates in clip space; [0038]: the final color of the pixel at screen coordinates);
wherein the light source information comprises an illumination angle at which the light source illuminates the first object (Laine; [0038]: lighting models; [0039]: the shade function typically models light-surface interactions; [0047]: light sources by θL ; [0174]: shadow effects).
Same motivation of claim 1 is applied here.
Regarding to claim 6 (Previously presented), Li in view of Laine and Porcino discloses the method according to claim 4, wherein obtaining the target editing parameter information corresponding to the first image by processing, based on an editor selection model obtained by pre-training, the first image (Li; [0038]: the neural network model 150 predicts and determines the texture images 154 and 158 for the frames 132 and 136, respectively; the neural network model 150 is fine-tuned on a particular video with the invariance constraints enforced by Equations; [0046]: the differentiable renderer 222 receives the 3D mesh and texture image and renders the 3D mesh according to the predicted camera pose provided by the camera pose unit 225; the textured 3D object is then projected by the differentiable renderer 222; [0071-0072]: the 3D object construction system 100 is trained using self-supervision for videos; the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence; [0141]: the vertex shading stage 620 processes vertex data by performing a set of operations, e.g., a vertex shader) comprises:
obtaining an attribute value output by the editor selection model corresponding to each editing parameter to be selected by inputting the first image into the editor selection model (Li; Fig. 2B; [0054]: the wrapped 3D shape representation 237 is rendered by the differentiable renderer 222 according to the predicted camera to produce rendered image 238; Fig. 2D; Fig. 2D; [0070]: the 3D object construction system 100 is trained to learn a set of shape bases from single-view images; [0072]: the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence); and
determining the editing parameter information from a plurality of editing parameters to be selected based on each attribute value (Li; Fig. 2B; [0054]: the wrapped 3D shape representation 237 is rendered by the differentiable renderer 222 according to the predicted camera to produce rendered image 238; [0071]: compute the foreground mask loss; [0072]: the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence).
Regarding to claim 7 (Previously presented), Li in view of Laine and Porcino discloses the method according to claim 1, wherein determining the material parameter information of the coating according to the light source information and the normal map of the first image (same as rejected in claim 1) comprises:
determining the normal map of the first image (Laine; [0038-0039]: normal vectors are normal map; the final color is obtained by sampling the result at the pixel center; [0055]: the depth in normalized device coordinates); and
obtaining the material parameter information of the coating output by a parameter generation model obtained by pre-training by processing, based on the parameter generation model, the normal map and the light source information (Laine; [0038-0039]: the final color is obtained by sampling the result at the pixel center based on normal vectors;
PNG
media_image17.png
54
338
media_image17.png
Greyscale
; [0055]: the depth in normalized device coordinates).
Same motivation of claim 1 is applied here.
Regarding to claim 8 (Previously presented), Li in view of Laine and Porcino discloses the method according to claim 7, wherein the material parameter information further comprises reflection function parameters (Laine; [0039]: the shade function typically models light-surface interactions; [0041]: an initial surface texture map corresponding to the initial 3D model may be a uniform color; [0044]: produce a surface texture map; [0047]: the surface factors are parameterized by θM, and light sources by θL.).
Same motivation of claim 1 Is applied here.
Regarding to claim 9 (Original), Li in view of Laine and Porcino discloses the method according to claim 8, wherein the reflectance function parameters comprise at least one of bidirectional reflectance distribution function, metallicity and roughness (one of may include one; Laine; [0038]: lighting models; [0039]: the shade function typically models light-surface interactions; [0021]: a set of texture maps 125 is global surface texture defining lighting and materials properties 120 that may be applied to the 3D model 130; [0022]: produce rendered images that closely match the set of 2D images of the object 110).
Same motivation of claim 1 is applied here.
Regarding to claim 10 (Currently amended), Li in view of Laine and Porcino discloses the method according to claim 1, wherein determining the second image by rendering the material on the second object based on the material parameter information and the target editing parameter information (same as rejected in claim 1) comprises:
drawing the second image based on an editor using the material parameter information as a parameter (Li; Fig. 1A; [0029]: apply the texture image 106 to the 3D shape representation 108 produces a 3D object 104;
PNG
media_image6.png
434
656
media_image6.png
Greyscale
; Fig. 1B; [0035]: the texture image is transferred onto the 3D shape representation to produce the 3D object corresponding to the object in the first image; when the 3D object is rendered according to the camera pose for each frame of video, the rendered object appears as the object in the frame);
Li in view of Laine and Porcino further discloses wherein the editor matches the editing parameter information (Li; [0061]: when the 2D key points are projected onto the 3D representation, e.g., mesh surface, the same semantic key point for different object instances is matched to the same face on the mesh surface).
Li in view of Laine further more discloses wherein the editor matches the editing parameter information (Laine; Fig. 1B; [0026]: produce a rendered image 115 that matches the reference image 112; Fig. 1C; [0030]: the shaded pixels 141 and 142 appear the same for many different positions and orientations of the edge 138;
PNG
media_image16.png
118
210
media_image16.png
Greyscale
; [0044]: the result is a 3D model of the object and corresponding surface texture map that, when rendered, match the target images).
Same motivation of claim 1 is applied here.
Regarding to claim 11 (Previously presented), Li in view of Laine and Porcino discloses the method according to claim 1, wherein before determining the light source information, the editing parameter information and the material parameter information (Li; Fig. 1A; [0025]: an input image 102 includes an object and a predicted 3D object 104; Fig. 1B; [0033]: the 3D object construction system 100 receives a video including images of the object captured from a camera pose), the method further comprises:
determining the light source information based on an illumination estimation model (Li; [0038]: the neural network model 150 predicts and determines the texture images 154 and 158 for the frames 132 and 136, respectively; the neural network model 150 is fine-tuned on a particular video with the invariance constraints enforced by Equations), determining the editing parameter information based on an editor selection model (Li; [0038]: the neural network model 150 predicts and determines the texture images 154 and 158 for the frames 132 and 136, respectively; the neural network model 150 is fine-tuned on a particular video with the invariance constraints enforced by Equations; [0046]: the differentiable renderer 222 receives the 3D mesh and texture image and renders the 3D mesh according to the predicted camera pose provided by the camera pose unit 225; the textured 3D object is then projected by the differentiable renderer 222; [0071-0072]: the 3D object construction system 100 is trained using self-supervision for videos; the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence; [0141]: the vertex shading stage 620 processes vertex data by performing a set of operations, e.g., a vertex shader), and determining the material parameter information based on a parameter generation model by training the illumination estimation model, the editor selection model, and the parameter generation model (Li; [0038]: the neural network model 150 predicts and determines the texture images 154 and 158 for the frames 132 and 136, respectively; the neural network model 150 is fine-tuned on a particular video with the invariance constraints enforced by Equations; [0046]: the differentiable renderer 222 receives the 3D mesh and texture image and renders the 3D mesh according to the predicted camera pose provided by the camera pose unit 225; the textured 3D object is then projected by the differentiable renderer 222; [0071-0072]: the 3D object construction system 100 is trained using self-supervision for videos; the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence; [0141]: the vertex shading stage 620 processes vertex data by performing a set of operations, e.g., a vertex shader).
Regarding to claim 12 (Original), Li in view of Laine and Porcino discloses the method according to claim 11, wherein training the illumination estimation model, the editor selection model, and the parameter generation model (same as rejected in claim 11) comprises:
obtaining a plurality of images to be trained (Li; Fig. 1D; [0040]: the neural network model 150 receives a sequence of frames for a video including an object; Fig. 1D; [0041]; Fig. 2D; [0070]); wherein the images to be trained are coated with a coating to be trained (Li; Fig. 1D; [0041]: a loss function is computed based on texture invariance and shape identity invariance; Fig. 2D; [0070]);
for each image to be trained, obtaining actual light source information of the image to be trained output by the illumination estimation model to be trained by inputting a current image to be trained into the illumination estimation model to be trained (Li; Fig. 1A; [0025]: receive each frame, e.g., image, in the video; Fig. 1D; [0040]: the neural network model 150 receives a sequence of frames for a video including an object; Fig. 1D; [0041]: a loss function is computed based on texture invariance and shape identity invariance; Fig. 2D; [0070]); and
determining an editing parameter to be used from a plurality of editing parameters to be selected by inputting the current image to be trained into the editor selection model to be trained (Li; Fig. 1D; [0040]: texture images are predicted for each frame by the neural network model; [0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object; Fig. 2D; [0070-0072]: a flowchart of a method 255 for training the 3D object construction system);
obtaining actual material parameter information of the coating to be trained corresponding to the current image to be trained output by the parameter generation model to be trained by using the actual light source information and the normal map of the current image to be trained as an input of the parameter generation model to be trained, and drawing an image to be compared based on the actual material parameter information (Li; Fig. 1D; [0041]: a loss function is computed based on texture invariance and shape identity invariance; [0042-0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object; Fig. 2D; [0070-0072]: a flowchart of a method 255 for training the 3D object construction system);
correcting parameters in the illumination estimation model to be trained, the editor selection model to be trained and the parameter generation model to be trained based on theoretical light source information, a theoretical editing parameter, the image to be compared, the actual light source information, the editing parameter to be used corresponding to the current image to be trained and the current image to be trained (Li; Fig. 1D; [0040-0041]: a loss function is computed based on texture invariance and shape identity invariance; [0042]: refinement of the neural network model 150 is completed; [0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object; Fig. 2D; [0070-0072]: a flowchart of a method 255 for training the 3D object construction system); and
obtaining the illumination estimation model, the editor selection model, and the parameter generation model by taking convergences of loss functions in the illumination estimation model to be trained, the editor selection model to be trained, and the parameter generation model to be trained as training targets (Li; Fig. 1D; [0040-0042]: refinement of the neural network model 150 is completed; [0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object; Fig. 2D; [0070-0072]: a flowchart of a method 255 for training the 3D object construction system).
Regarding to claim 13 (Previously presented), Li in view of Laine and Porcino discloses the method according to claim 12, wherein correcting parameters in the illumination estimation model to be trained, the editor selection model to be trained and the parameter generation model to be trained based on theoretical light source information, a theoretical editing parameter, the image to be compared, the actual light source information, the editing parameter to be used corresponding to the current image to be trained and the current image to be trained (same as rejected in claim 12) comprises:
correcting model parameters in the illumination estimation model to be trained according to an actual distance difference by determining the actual distance difference according to the theoretical light source information and the actual light source information of the current image to be trained; or, determining a fourth image according to the actual light source information and the actual material parameter information of the current image to be trained, and correcting the model parameters in the illumination estimation model to be trained according to the fourth image and the current image to be trained (or is optional; Li; Fig. 1D; [0040-0041]: a loss function is computed based on texture invariance and shape identity invariance; [0042]: refinement of the neural network model 150 is completed; [0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object; Fig. 2D; [0070-0072]: a flowchart of a method 255 for training the 3D object construction system);
correcting model parameters in the editor selection model to be trained according to the theoretical editing parameters and the editing parameters to be used corresponding to the current image to be trained (Li; Fig. 1D; [0040-0041]: a loss function is computed based on texture invariance and shape identity invariance; [0042]: refinement of the neural network model 150 is completed; [0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object; Fig. 2D; [0070-0072]: a flowchart of a method 255 for training the 3D object construction system); and
correcting model parameters in the parameter generation model to be trained according to the image to be compared and the current image to be trained (Li; Fig. 1D; [0040-0041]: a loss function is computed based on texture invariance and shape identity invariance; [0042]: refinement of the neural network model 150 is completed; [0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object; Fig. 2D; [0070-0072]: a flowchart of a method 255 for training the 3D object construction system).
Regarding to claim 15 (Currently amended), Li discloses an electronic device (Fig. 1A; [0025]: the 3D object construction system 100 includes a neural network model comprising at least an encoder 105, shape decoder 115, and motion decoder 120; the encoder 105 extracts features 110 from each frame, e.g., image, in the video; [0029]: the texture decoder 125 receives the features 110 and predicts a texture image 106 for each frame of the video; [0032]: construct a 3D representation of an object using the 3D object reconstruction system 100; [0033]: the 3D object construction system 100 receives a video including images of the object captured from a camera pose; Fig. 4; [0104]: GPCs; [0161]: a Personal Computer; [0162]: the executable instructions are stored in a computer readable medium for use by a processor-based instruction execution machine, system, apparatus, or device.), comprising:
at least one processor (Fig. 4; [0104]: GPCs; [0108]: GPUs and CPU; [0162]: a processor); and
a store configured to store at least one program ([0162]: the executable instructions are stored in a computer readable medium for use by a processor-based instruction execution machine, system, apparatus, or device; a random-access memory);
the at least one program, when executed by the at least one processor, causes the at least one processor to ([0162]: the executable instructions are stored in a computer readable medium for use by a processor-based instruction execution machine, system, apparatus, or device. ):
the rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject claim 15.
Regarding to claim 16 (Currently amended), Li discloses a non-transitory storage medium containing computer-executable instructions, the computer-executable instructions, when executed by a computer processor, cause the computer processor to (Fig. 1A; [0025]: the 3D object construction system 100 includes a neural network model comprising at least an encoder 105, shape decoder 115, and motion decoder 120; the encoder 105 extracts features 110 from each frame, e.g., image, in the video; [0029]: the texture decoder 125 receives the features 110 and predicts a texture image 106 for each frame of the video; [0032]: construct a 3D representation of an object using the 3D object reconstruction system 100; [0033]: the 3D object construction system 100 receives a video including images of the object captured from a camera pose; Fig. 4; [0104]: GPCs; [0161]: a Personal Computer; [0162]: the executable instructions are stored in a computer readable medium for use by a processor-based instruction execution machine, system, apparatus, or device; a random-access memory):
The rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject claim 16.
Regarding to claim 17 (Previously presented), Li in view of Laine and Porcino discloses the electronic device according to claim 15, wherein the at least one program causes the at least one processor to obtain the image to be processed by (Li; Fig. 4; [0104]: GPCs; [0108]: GPUs and CPU; [0162]: a processor; the executable instructions are stored in a computer readable medium for use by a processor-based instruction execution machine, system, apparatus, or device; a random-access memory):
The rest claim limitations are similar to claim limitations recited in claim 2. Therefore, same rational used to reject claim 2 is also used to reject claim 17.
Regarding to claim 18 (Previously presented), Li in view of Laine and Porcino discloses the electronic device according to claim 17,
The rest claim limitations are similar to claim limitations recited in claim 3. Therefore, same rational used to reject claim 3 is also used to reject claim 18.
Regarding to claim 19 (Previously presented), Li in view of Laine and Porcino discloses the electronic device according to claim 15, wherein the at least one program causes the at least one processor to (same as rejected in claim 15)
The rest claim limitations are similar to claim limitations recited in claim 4. Therefore, same rational used to reject claim 4 is also used to reject claim 19.
Regarding to claim 20 (Previously presented), Li in view of Laine and Porcino discloses the electronic device according to claim 19, wherein the at least one program causes the at least one processor (same as rejected in claim 15)
The rest claim limitations are similar to claim limitations recited in claim 5. Therefore, same rational used to reject claim 5 is also used to reject claim 20.
Regarding to claim 21 (Previously presented), Li in view of Laine and Porcino discloses the electronic device according to claim 19, wherein the at least one program causes the at least one processor to (same as rejected in claim 15)
The rest claim limitations are similar to claim limitations recited in claim 6. Therefore, same rational used to reject claim 6 is also used to reject claim 21.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hai Tao Sun whose telephone number is (571)272-5630. The examiner can normally be reached 9:00AM-6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 5712727642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAI TAO SUN/Primary Examiner, Art Unit 2616
Prosecution Insights