DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This office action is responsive to the amendment received 03/16/2026.
In the response to the Non-Final Office Action 12/16/2025, the applicant states that claims 1-13 and 15-21 are pending in the application. Claims 1-8, 10, 11, 13, and 15-21 are amended.
Claims 1-8, 10, 11, 13, and 15-21 have been amended. In summary, claims 1-13 and 15-21 are pending in current application.
Response to Arguments
Applicant's arguments filed 03/16/2026 have been fully considered but they are not persuasive.
Regarding to the objection of drawings, the amendment has cured the basis of the objection of drawings. Therefore, the objection of drawings is hereby withdrawn.
Regarding to claim 1, the applicant argues that Li in view of Laine does not teach or suggest "wherein the first image comprises a coating of a material, the coating is a substance coated on a first object as a carrier of the material;" "determining material parameter information of the coating, wherein the material parameter information comprises at least one of color, texture, smoothness, transparency, reflectivity, refractive index, and luminosity of the material;" and "determining a second image by rendering the material on a second object based on the material parameter information and the editing parameter information." The arguments have been fully considered, but they are not persuasive. The examiner cannot concur with the applicant for following reasons:
Li discloses “the first image comprises a coating of a material, the coating is a substance coated on a first object as a carrier of the material”. For example, in paragraph [0021], Li teaches recovery of the 3D shape, texture, and camera pose of objects from 2D images; Li further teaches texture is a coating of a material. In Fig. 1A and paragraph [0025], Li teaches an input image 102 includes an object with a coating of a materials, such as texture, white or black color as illustrated in Fig. 1A;
PNG
media_image1.png
246
382
media_image1.png
Greyscale
; Li further teaches a texture is a 2D coating of material. In paragraph [0028], Li teaches the different faces defined by the vertices in the mesh are colored either black or white. In Fig. 1 and paragraph [0029], Li teaches the UV texture image; Li further teaches applying the texture image 106, i.e. a 2D coating, to the 3D shape representation 108, i.e. an object, to produce a 3D object 104, e.g., textured mesh and a second image;
PNG
media_image2.png
550
442
media_image2.png
Greyscale
. In paragraph [0141], Li teaches modifying color attributes for a vertex. In Fig. 2B and paragraph [0050], Li teaches the painting appears as differently colored vertical stripes within a silhouette of the object visible in a propagated part map 232; Li further teaches the painting is a coating of material.
Li further discloses “determining material parameter information of the coating according to the light source information and a normal of the first image”. For example, in paragraph [0038], Li teaches the swapping technique enforces consistency of the texture images across the frames; Li further teaches the loss function Le enforces consistency between the texture images. In Fig. 2B and paragraph [0053], Li teaches the propagated part map for each frame is mapped to the UV texture space with the predicted texture flow 234 to produce part UV map 236. In paragraph [0141], Li teaches one or more vertex attributes include color, texture coordinates, surface normal, etc.; Li further teaches the vertex shading stage 620 manipulates individual vertex attributes such as position, color, texture coordinates, and the like; modify color attributes for a vertex; Li further more teaches a normalized-device-coordinate space.
Li further more teaches “wherein the material parameter information comprises at least one of color, texture, smoothness, transparency, reflectivity, refractive index, and luminosity of the material”. For example, in paragraph [0021], Li teaches recovery of the 3D shape, texture, and camera pose of objects from 2D images; Li further teaches texture is a coating of a material. In paragraph [0028], Li teaches different faces defined by the vertices in the mesh are colored either black or white. n Fig. 1 and paragraph [0029], Li teaches the UV texture image; Li further teaches the material parameter information includes the texture flow maps pixels and a texture image 106; Li further more teaches applying the texture image 106, i.e. a 2D coating, to the 3D shape representation 108, i.e. an object, to produce a 3D object 104, e.g., textured mesh and a second image;
PNG
media_image2.png
550
442
media_image2.png
Greyscale
.
In addition, Laine discloses “wherein the material parameter information comprises at least one of color, texture, smoothness, transparency, reflectivity, refractive index, and luminosity of the material”. For example, in Fig. 1A and paragraph [0021], Laine teaches global surface texture defines lighting and materials properties 120;
PNG
media_image3.png
560
530
media_image3.png
Greyscale
. In paragraph [0038], Laine teaches a proper differentiable renderer provides gradients for all the parameters—e.g., lighting and material parameters, as well as the contents of texture maps used in the process. In paragraph [0077], Laine teaches the surface texture represents lighting and material properties of the 3D model.
Li further more “determining a second image by rendering the material on a second object based on the material parameter information and the editing parameter information”. For example, in Fig. 1A and paragraph [0029], Li further more teaches applying the texture image 106, i.e. a 2D coating, to the 3D shape representation 108, i.e. an object, to produce a 3D object 104, e.g., textured mesh and a second image;
PNG
media_image4.png
434
656
media_image4.png
Greyscale
. In Fig. 1B and paragraph [0035], Li teaches the texture image is transferred onto the 3D shape representation to produce the 3D object, i.e. a second image, corresponding to the object in the first image; Li further teaches when the 3D object is rendered according to the camera pose for each frame of video, the rendered object appears as the object in the frame. In paragraph [0038], Li teaches a texture invariance constraint maybe used to encourage consistent texture reconstruction from all frames; Li further teaches the swapping technique enforces consistency of the texture images across the frames; Li further more teaches the loss function Le enforces consistency between the texture images. In Fig. 2B and paragraph [0054], Li teaches the wrapped 3D shape representation 237 is rendered by the differentiable renderer 222 according to the predicted camera to produce rendered image 238. In Fig. 2D and paragraph [0072], Li teaches the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence.
Claims 2-13 and 15-21 are not allowable due to the similar reasons as discussed above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-13 and 15-21 are rejected under 35 U.S.C. 103 as being unpatentable over Li (US 20220036635 A1) and in view of Laine (US 20220051481 A1).
Regarding to claim 1 (Currently amended), Li discloses a method of drawing an image (Fig. 1A; [0025]: the 3D object construction system 100 includes a neural network model comprising at least an encoder 105, shape decoder 115, and motion decoder 120; the encoder 105 extracts features 110 from each frame, e.g., image, in the video; [0029]: the texture decoder 125 receives the features 110 and predicts a texture image 106 for each frame of the video; [0032]: construct a 3D representation of an object using the 3D object reconstruction system 100; [0033]: the 3D object construction system 100 receives a video including images of the object captured from a camera pose), comprising:
obtaining a first image (Fig. 1A; [0025]: receive each frame, e.g., image, in the video;
PNG
media_image5.png
446
670
media_image5.png
Greyscale
; Fig. 1B; [0033]: the 3D object construction system 100 receives a video including images of the object captured from a camera pose), wherein the first image comprises a coating of a material, the coating is a substance coated on a first object as a carrier of the material ([0021]: recovery of the 3D shape, texture, and camera pose of objects from 2D images; texture is a coating of a material; Fig. 1A; [0025]: an input image 102 includes an object with a coating of a materials, such as texture, white or black color as illustrated in Fig. 1A;
PNG
media_image1.png
246
382
media_image1.png
Greyscale
; a texture is a 2D coating of a material; [0028]: the different faces defined by the vertices in the mesh are colored either black or white; Fig. 1; [0029]: the UV texture image; apply the texture image 106, i.e. a 2D coating, to the 3D shape representation 108, i.e. an object, to produce a 3D object 104, e.g., textured mesh and second image.
PNG
media_image2.png
550
442
media_image2.png
Greyscale
;[0141]: modify color attributes for a vertex; Fig. 2B; [0050]: the painting appears as differently colored vertical stripes within a silhouette of the object visible in a propagated part map 232);
determining light source information and editing parameter information corresponding to the first image respectively ([0037]: a differentiable renderer renders a texture mesh to an RGB image; [0038]: lighting conditions; [0046]: the differentiable renderer 222 receives the 3D mesh and texture image and renders the 3D mesh; [0141]: the vertex shading stage 620 processes vertex data by performing a set of operations; operations commonly include lighting operations, e.g., modifying color attributes for a vertex; [0147]: perform lighting operations; perform sampling texture maps using interpolated texture coordinates for the fragment; the fragment shading stage 670 generates pixel data, e.g., color values, for the fragment such as by performing lighting operations; [0155]: the rendering of the game session includes ray or path-traced lighting and/or shadow effects, computed using one or more parallel processing units);
determining material parameter information of the coating according to the light source information and a normal of the first image ([0038]: the swapping technique enforces consistency of the texture images across the frames; the loss function Le enforces consistency between the texture images; Fig. 2B; [0053]: the propagated part map for each frame is mapped to the UV texture space with the predicted texture flow 234 to produce part UV map 236; [0141]: one or more vertex attributes include color, texture coordinates, surface normal, etc.; the vertex shading stage 620 manipulates individual vertex attributes such as position, color, texture coordinates, and the like; modify color attributes for a vertex; a normalized-device-coordinate space), wherein the material parameter information comprises at least one of color, texture, smoothness, transparency, reflectivity, refractive index, and luminosity of the material (one of; [0021]: recovery of the 3D shape, texture, and camera pose of objects from 2D images; texture is a coating of a material; [0028]: different faces defined by the vertices in the mesh are colored either black or white; Fig. 1A; [0029]: the UV texture image; the material parameter information includes the texture flow maps pixels and a texture image 106; apply the texture image 106, i.e. a 2D coating, to the 3D shape representation 108, i.e. an object, to produce a 3D object 104, e.g., textured mesh and a second image;
PNG
media_image2.png
550
442
media_image2.png
Greyscale
); and
determining a second image by rendering the material on a second object based on the material parameter information and the editing parameter information (Fig. 1A; [0029]: apply the texture image 106, i.e. a 2D coating, to the 3D shape representation 108, i.e. an object, to produce a 3D object 104, e.g., textured mesh and a second image;
PNG
media_image2.png
550
442
media_image2.png
Greyscale
; Fig. 1B; [0035]: the texture image is transferred onto the 3D shape representation to produce the 3D object corresponding to the object in the first image; when the 3D object is rendered according to the camera pose for each frame of video, the rendered object appears as the object in the frame; [0038]: a texture invariance constraint maybe used to encourage consistent texture reconstruction from all frames; the swapping technique enforces consistency of the texture images across the frames; the loss function Le enforces consistency between the texture images; Fig. 2B; [0054]: the wrapped 3D shape representation 237 is rendered by the differentiable renderer 222 according to the predicted camera to produce rendered image 238; Fig. 2D; [0072]: the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence).
Li fails to explicitly disclose normal map.
In same field of endeavor, Laine teaches:
normal map ([0038-0039]: normal vectors are normal map; the final color is obtained by sampling the result at the pixel center; [0055]: the depth in normalized device coordinates);
wherein the material parameter information comprises at least one of color, texture, smoothness, transparency, reflectivity, refractive index, and luminosity of the material (one of; Fig. 1A; [0021]: global surface texture defines lighting and materials properties 120;
PNG
media_image3.png
560
530
media_image3.png
Greyscale
; [0038]: a proper differentiable renderer provides gradients for all the parameters—e.g., lighting and material parameters, as well as the contents of texture maps used in the process; [0077]: the surface texture represents lighting and material properties of the 3D model.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Li to include normal map; wherein the material parameter information comprises at least one of color, texture, smoothness, transparency, reflectivity, refractive index, and luminosity of the material as taught by Laine. The motivation for doing so would have been reconstruct the 3D model using highly optimized hardware graphics pipelines; to improve the 3D model; to obtain the final color using normal vectors; to improve accuracy of the 3D model as taught by Laine in paragraphs [0020], [0029], [0038-0039], and [0045].
Regarding to claim 2 (Currently amended), Li in view of Laine discloses the method according to claim 1, wherein obtaining the first image (same as rejected in claim 1) comprises:
obtaining a third image to be used by photographing the first object coated with the coating (Li; Fig. 1A; [0025]: an input image 102 includes an object;
PNG
media_image6.png
240
380
media_image6.png
Greyscale
; Fig. 1B; [0033]: images of the object are captured from a camera pose; the video is captured by a camera); and
obtaining the first image by processing, according to an image processing approach, the first image (Laine; Fig. 1A; [0021]: a set of 2D images of an object 110 are captured from a variety of camera positions; a complete 3D model 130 is constructed for each of one or more of the 2D images in the set);
wherein the first object is presented in the first image at a ratio (Laine; Fig. 1A; [0022]: match the set of 2D images of the object 110; the set of 2D images of the object 110 comprise a video;
PNG
media_image7.png
294
686
media_image7.png
Greyscale
; Fig. 1B; [0026]: produce a rendered image 115 that matches the reference image 112).
Same motivation of claim 1 is applied here.
Regarding to claim 3 (Currently amended), Li in view of Laine discloses the method according to claim 2, wherein the first object being presented in the first image at the ratio (same as rejected in claim 2) comprises:
the first image being filled with the first object, and an object edge displayed in the first image being tangent to an edge line of the first image (Laine; Fig. 1B; [0026]: produce a rendered image 115 that matches the reference image 112;
PNG
media_image8.png
412
482
media_image8.png
Greyscale
; [0028]: rendered geometry 136 forms a silhouette edge 138 of an object; Fig. 1C; [0030]: the shaded pixels 141 and 142 appear the same for many different positions and orientations of the edge 138;
PNG
media_image9.png
118
210
media_image9.png
Greyscale
).
Same motivation of claim 1 is applied here.
Regarding to claim 4 (Currently amended), Li in view of Laine discloses the method according to claim 1, wherein determining the light source information and the editing parameter information corresponding to the first image respectively (same as rejected in claim 1) comprises:
determining the light source information corresponding to the first image by processing, based on an illumination estimation model obtained by pre-training, the first image (Li; [0038]: the neural network model 150 predicts and determines the texture images 154 and 158 for the frames 132 and 136, respectively; the neural network model 150 is fine-tuned on a particular video with the invariance constraints enforced by Equations; [0040]: the texture images are predicted for each frame by the neural network model 150; Fig. 1D; [0042]: refinement of the neural network model 150 is completed; [0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object); and
obtaining the target editing parameter information corresponding to the first image by processing, based on an editor selection model obtained by pre-training, the first image (Li; [0038]: the neural network model 150 predicts and determines the texture images 154 and 158 for the frames 132 and 136, respectively; the neural network model 150 is fine-tuned on a particular video with the invariance constraints enforced by Equations; [0046]: the differentiable renderer 222 receives the 3D mesh and texture image and renders the 3D mesh according to the predicted camera pose provided by the camera pose unit 225; the textured 3D object is then projected by the differentiable renderer 222; [0071-0072]: the 3D object construction system 100 is trained using self-supervision for videos; the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence; [0141]: the vertex shading stage 620 processes vertex data by performing a set of operations, e.g., a vertex shader).
Regarding to claim 5 (Currently amended), Li in view of Laine discloses the method according to claim 4, wherein determining the light source information corresponding to the first image by processing, based on the illumination estimation model obtained by pre-training, the first image (same as rejected in claim 4), comprises:
obtaining pixel coordinate information of a highlight point in the first image output by the illumination estimation model by inputting the first image into the illumination estimation model (Li; [0141]: vertices are specified as a 4-coordinate vector; modify the coordinate space for a vertex; vertices are specified using coordinates in an object-coordinate space); and
determining, based on the pixel coordinate information, the light source information of a light source upon obtaining the first image by photographing (Li; [0141]: lighting operations; modify color attributes for a vertex; [0147]: generate pixel data, e.g., color values, for the fragment such as by performing lighting operations),
Li in view of Laine further discloses:
obtaining pixel coordinate information (Laine; [0037]: the x, y pixel coordinates in clip space; [0038]: the final color of the pixel at screen coordinates);
wherein the light source information comprises an illumination angle at which the light source illuminates the first object (Laine; [0038]: lighting models; [0039]: the shade function typically models light-surface interactions; [0047]: light sources by θL ; [0174]: shadow effects).
Same motivation of claim 1 is applied here.
Regarding to claim 6 (Currently amended), Li in view of Laine discloses the method according to claim 4, wherein obtaining the target editing parameter information corresponding to the first image by processing, based on an editor selection model obtained by pre-training, the first image (Li; [0038]: the neural network model 150 predicts and determines the texture images 154 and 158 for the frames 132 and 136, respectively; the neural network model 150 is fine-tuned on a particular video with the invariance constraints enforced by Equations; [0046]: the differentiable renderer 222 receives the 3D mesh and texture image and renders the 3D mesh according to the predicted camera pose provided by the camera pose unit 225; the textured 3D object is then projected by the differentiable renderer 222; [0071-0072]: the 3D object construction system 100 is trained using self-supervision for videos; the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence; [0141]: the vertex shading stage 620 processes vertex data by performing a set of operations, e.g., a vertex shader) comprises:
obtaining an attribute value output by the editor selection model corresponding to each editing parameter to be selected by inputting the first image into the editor selection model (Li; Fig. 2B; [0054]: the wrapped 3D shape representation 237 is rendered by the differentiable renderer 222 according to the predicted camera to produce rendered image 238; Fig. 2D; Fig. 2D; [0070]: the 3D object construction system 100 is trained to learn a set of shape bases from single-view images; [0072]: the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence); and
determining the editing parameter information from a plurality of editing parameters to be selected based on each attribute value (Li; Fig. 2B; [0054]: the wrapped 3D shape representation 237 is rendered by the differentiable renderer 222 according to the predicted camera to produce rendered image 238; [0071]: compute the foreground mask loss; [0072]: the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence).
Regarding to claim 7 (Currently amended), Li in view of Laine discloses the method according to claim 1, wherein determining the material parameter information of the coating according to the light source information and the normal map of the first image (same as rejected in claim 1) comprises:
determining the normal map of the first image (Laine; [0038-0039]: normal vectors are normal map; the final color is obtained by sampling the result at the pixel center; [0055]: the depth in normalized device coordinates); and
obtaining the material parameter information of the coating output by a parameter generation model obtained by pre-training by processing, based on the parameter generation model, the normal map and the light source information (Laine; [0038-0039]: the final color is obtained by sampling the result at the pixel center based on normal vectors;
PNG
media_image10.png
54
338
media_image10.png
Greyscale
; [0055]: the depth in normalized device coordinates).
Same motivation of claim 1 is applied here.
Regarding to claim 8 (Currently amended), Li in view of Laine discloses the method according to claim 7, wherein the material parameter information further comprises reflection function parameters (Laine; [0039]: the shade function typically models light-surface interactions; [0041]: an initial surface texture map corresponding to the initial 3D model may be a uniform color; [0044]: produce a surface texture map; [0047]: the surface factors are parameterized by θM, and light sources by θL.).
Same motivation of claim 1 Is applied here.
Regarding to claim 9 (Original), Li in view of Laine discloses the method according to claim 8, wherein the reflectance function parameters comprise at least one of bidirectional reflectance distribution function, metallicity and roughness (one of may include one; Laine; [0038]: lighting models; [0039]: the shade function typically models light-surface interactions; [0021]: a set of texture maps 125 is global surface texture defining lighting and materials properties 120 that may be applied to the 3D model 130; [0022]: produce rendered images that closely match the set of 2D images of the object 110).
Same motivation of claim 1 is applied here.
Regarding to claim 10 (Currently amended), Li in view of Laine discloses the method according to claim 1, wherein determining the second image by rendering the material on the second object based on the material parameter information and the target editing parameter information (same as rejected in claim 1) comprises:
drawing the second image based on an editor using the material parameter information as a parameter (Li; Fig. 1A; [0029]: apply the texture image 106 to the 3D shape representation 108 produces a 3D object 104;
PNG
media_image4.png
434
656
media_image4.png
Greyscale
; Fig. 1B; [0035]: the texture image is transferred onto the 3D shape representation to produce the 3D object corresponding to the object in the first image; when the 3D object is rendered according to the camera pose for each frame of video, the rendered object appears as the object in the frame);
Li in view of Laine further discloses wherein the editor matches the editing parameter information (Li; [0061]: when the 2D key points are projected onto the 3D representation, e.g., mesh surface, the same semantic key point for different object instances is matched to the same face on the mesh surface).
Li in view of Laine further more discloses wherein the editor matches the editing parameter information (Laine; Fig. 1B; [0026]: produce a rendered image 115 that matches the reference image 112; Fig. 1C; [0030]: the shaded pixels 141 and 142 appear the same for many different positions and orientations of the edge 138;
PNG
media_image9.png
118
210
media_image9.png
Greyscale
; [0044]: the result is a 3D model of the object and corresponding surface texture map that, when rendered, match the target images).
Same motivation of claim 1 is applied here.
Regarding to claim 11 (Currently amended), Li in view of Laine discloses the method according to claim 1, wherein before determining the light source information, the editing parameter information and the material parameter information (Li; Fig. 1A; [0025]: an input image 102 includes an object and a predicted 3D object 104; Fig. 1B; [0033]: the 3D object construction system 100 receives a video including images of the object captured from a camera pose), the method further comprises:
determining the light source information based on an illumination estimation model (Li; [0038]: the neural network model 150 predicts and determines the texture images 154 and 158 for the frames 132 and 136, respectively; the neural network model 150 is fine-tuned on a particular video with the invariance constraints enforced by Equations), determining the editing parameter information based on an editor selection model (Li; [0038]: the neural network model 150 predicts and determines the texture images 154 and 158 for the frames 132 and 136, respectively; the neural network model 150 is fine-tuned on a particular video with the invariance constraints enforced by Equations; [0046]: the differentiable renderer 222 receives the 3D mesh and texture image and renders the 3D mesh according to the predicted camera pose provided by the camera pose unit 225; the textured 3D object is then projected by the differentiable renderer 222; [0071-0072]: the 3D object construction system 100 is trained using self-supervision for videos; the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence; [0141]: the vertex shading stage 620 processes vertex data by performing a set of operations, e.g., a vertex shader), and determining the material parameter information based on a parameter generation model by training the illumination estimation model, the editor selection model, and the parameter generation model (Li; [0038]: the neural network model 150 predicts and determines the texture images 154 and 158 for the frames 132 and 136, respectively; the neural network model 150 is fine-tuned on a particular video with the invariance constraints enforced by Equations; [0046]: the differentiable renderer 222 receives the 3D mesh and texture image and renders the 3D mesh according to the predicted camera pose provided by the camera pose unit 225; the textured 3D object is then projected by the differentiable renderer 222; [0071-0072]: the 3D object construction system 100 is trained using self-supervision for videos; the 3D object construction system 100 is fine-tuned for constructing a particular 3D object using the one or more of the invariance constraints for texture, identity shape, and part correspondence; [0141]: the vertex shading stage 620 processes vertex data by performing a set of operations, e.g., a vertex shader).
Regarding to claim 12 (Original), Li in view of Laine discloses the method according to claim 11, wherein training the illumination estimation model, the editor selection model, and the parameter generation model (same as rejected in claim 11) comprises:
obtaining a plurality of images to be trained (Li; Fig. 1D; [0040]: the neural network model 150 receives a sequence of frames for a video including an object; Fig. 1D; [0041]; Fig. 2D; [0070]); wherein the images to be trained are coated with a coating to be trained (Li; Fig. 1D; [0041]: a loss function is computed based on texture invariance and shape identity invariance; Fig. 2D; [0070]);
for each image to be trained, obtaining actual light source information of the image to be trained output by the illumination estimation model to be trained by inputting a current image to be trained into the illumination estimation model to be trained (Li; Fig. 1A; [0025]: receive each frame, e.g., image, in the video; Fig. 1D; [0040]: the neural network model 150 receives a sequence of frames for a video including an object; Fig. 1D; [0041]: a loss function is computed based on texture invariance and shape identity invariance; Fig. 2D; [0070]); and
determining an editing parameter to be used from a plurality of editing parameters to be selected by inputting the current image to be trained into the editor selection model to be trained (Li; Fig. 1D; [0040]: texture images are predicted for each frame by the neural network model; [0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object; Fig. 2D; [0070-0072]: a flowchart of a method 255 for training the 3D object construction system);
obtaining actual material parameter information of the coating to be trained corresponding to the current image to be trained output by the parameter generation model to be trained by using the actual light source information and the normal map of the current image to be trained as an input of the parameter generation model to be trained, and drawing an image to be compared based on the actual material parameter information (Li; Fig. 1D; [0041]: a loss function is computed based on texture invariance and shape identity invariance; [0042-0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object; Fig. 2D; [0070-0072]: a flowchart of a method 255 for training the 3D object construction system);
correcting parameters in the illumination estimation model to be trained, the editor selection model to be trained and the parameter generation model to be trained based on theoretical light source information, a theoretical editing parameter, the image to be compared, the actual light source information, the editing parameter to be used corresponding to the current image to be trained and the current image to be trained (Li; Fig. 1D; [0040-0041]: a loss function is computed based on texture invariance and shape identity invariance; [0042]: refinement of the neural network model 150 is completed; [0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object; Fig. 2D; [0070-0072]: a flowchart of a method 255 for training the 3D object construction system); and
obtaining the illumination estimation model, the editor selection model, and the parameter generation model by taking convergences of loss functions in the illumination estimation model to be trained, the editor selection model to be trained, and the parameter generation model to be trained as training targets (Li; Fig. 1D; [0040-0042]: refinement of the neural network model 150 is completed; [0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object; Fig. 2D; [0070-0072]: a flowchart of a method 255 for training the 3D object construction system).
Regarding to claim 13 (Currently amended), Li in view of Laine discloses the method according to claim 12, wherein correcting parameters in the illumination estimation model to be trained, the editor selection model to be trained and the parameter generation model to be trained based on theoretical light source information, a theoretical editing parameter, the image to be compared, the actual light source information, the editing parameter to be used corresponding to the current image to be trained and the current image to be trained (same as rejected in claim 12) comprises:
correcting model parameters in the illumination estimation model to be trained according to an actual distance difference by determining the actual distance difference according to the theoretical light source information and the actual light source information of the current image to be trained; or, determining a fourth image according to the actual light source information and the actual material parameter information of the current image to be trained, and correcting the model parameters in the illumination estimation model to be trained according to the fourth image and the current image to be trained (or is optional; Li; Fig. 1D; [0040-0041]: a loss function is computed based on texture invariance and shape identity invariance; [0042]: refinement of the neural network model 150 is completed; [0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object; Fig. 2D; [0070-0072]: a flowchart of a method 255 for training the 3D object construction system);
correcting model parameters in the editor selection model to be trained according to the theoretical editing parameters and the editing parameters to be used corresponding to the current image to be trained (Li; Fig. 1D; [0040-0041]: a loss function is computed based on texture invariance and shape identity invariance; [0042]: refinement of the neural network model 150 is completed; [0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object; Fig. 2D; [0070-0072]: a flowchart of a method 255 for training the 3D object construction system); and
correcting model parameters in the parameter generation model to be trained according to the image to be compared and the current image to be trained (Li; Fig. 1D; [0040-0041]: a loss function is computed based on texture invariance and shape identity invariance; [0042]: refinement of the neural network model 150 is completed; [0043]: the parameters of the neural network model 150 are updated to encourage consistency between the first projected 3D object and the second projected 3D object; Fig. 2D; [0070-0072]: a flowchart of a method 255 for training the 3D object construction system).
Regarding to claim 15 (Currently amended), Li discloses an electronic device (Fig. 1A; [0025]: the 3D object construction system 100 includes a neural network model comprising at least an encoder 105, shape decoder 115, and motion decoder 120; the encoder 105 extracts features 110 from each frame, e.g., image, in the video; [0029]: the texture decoder 125 receives the features 110 and predicts a texture image 106 for each frame of the video; [0032]: construct a 3D representation of an object using the 3D object reconstruction system 100; [0033]: the 3D object construction system 100 receives a video including images of the object captured from a camera pose; Fig. 4; [0104]: GPCs; [0161]: a Personal Computer; [0162]: the executable instructions are stored in a computer readable medium for use by a processor-based instruction execution machine, system, apparatus, or device.), comprising:
at least one processor (Fig. 4; [0104]: GPCs; [0108]: GPUs and CPU; [0162]: a processor); and
a store configured to store at least one program ([0162]: the executable instructions are stored in a computer readable medium for use by a processor-based instruction execution machine, system, apparatus, or device; a random-access memory);
the at least one program, when executed by the at least one processor, causes the at least one processor to ([0162]: the executable instructions are stored in a computer readable medium for use by a processor-based instruction execution machine, system, apparatus, or device. ):
the rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject claim 15.
Regarding to claim 16 (Currently amended), Li discloses a non-transitory storage medium containing computer-executable instructions, the computer-executable instructions, when executed by a computer processor, cause the computer processor to (Fig. 1A; [0025]: the 3D object construction system 100 includes a neural network model comprising at least an encoder 105, shape decoder 115, and motion decoder 120; the encoder 105 extracts features 110 from each frame, e.g., image, in the video; [0029]: the texture decoder 125 receives the features 110 and predicts a texture image 106 for each frame of the video; [0032]: construct a 3D representation of an object using the 3D object reconstruction system 100; [0033]: the 3D object construction system 100 receives a video including images of the object captured from a camera pose; Fig. 4; [0104]: GPCs; [0161]: a Personal Computer; [0162]: the executable instructions are stored in a computer readable medium for use by a processor-based instruction execution machine, system, apparatus, or device; a random-access memory):
The rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject claim 16.
Regarding to claim 17 (Currently amended), Li in view of Laine discloses the electronic device according to claim 15, wherein the at least one program causes the at least one processor to obtain the image to be processed by (Li; Fig. 4; [0104]: GPCs; [0108]: GPUs and CPU; [0162]: a processor; the executable instructions are stored in a computer readable medium for use by a processor-based instruction execution machine, system, apparatus, or device; a random-access memory):
The rest claim limitations are similar to claim limitations recited in claim 2. Therefore, same rational used to reject claim 2 is also used to reject claim 17.
Regarding to claim 18 (Currently amended), Li in view of Laine discloses the electronic device according to claim 17,
The rest claim limitations are similar to claim limitations recited in claim 3. Therefore, same rational used to reject claim 3 is also used to reject claim 18.
Regarding to claim 19 (Currently amended), Li in view of Laine discloses the electronic device according to claim 15, wherein the at least one program causes the at least one processor to (same as rejected in claim 15)
The rest claim limitations are similar to claim limitations recited in claim 4. Therefore, same rational used to reject claim 4 is also used to reject claim 19.
Regarding to claim 20 (Currently amended), Li in view of Laine discloses the electronic device according to claim 19, wherein the at least one program causes the at least one processor (same as rejected in claim 15)
The rest claim limitations are similar to claim limitations recited in claim 5. Therefore, same rational used to reject claim 5 is also used to reject claim 20.
Regarding to claim 21 (Currently amended), Li in view of Laine discloses the electronic device according to claim 19, wherein the at least one program causes the at least one processor to (same as rejected in claim 15)
The rest claim limitations are similar to claim limitations recited in claim 6. Therefore, same rational used to reject claim 6 is also used to reject claim 21.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hai Tao Sun whose telephone number is (571)272-5630. The examiner can normally be reached 9:00AM-6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 5712727642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAI TAO SUN/Primary Examiner, Art Unit 2616