Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claim 2 stands cancelled. Claims 33-36 are newly added. Claims 1 and 3-36 are pending.
Response to Arguments
Applicant’s arguments, see Remarks, filed May 22, 2026, with respect to the rejection(s) of claim(s) 1, 9, 17 and 25 under 35 USC 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of CN110246209B to Liu et al. in view of First Order Motion Model for Image Animation to Siarohin et al., and PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization to Saito et al.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 4, 8-10, 13, 17-26, 29 and 31-36 is/are rejected under 35 U.S.C. 103 as being unpatentable over CN110246209B to Liu et al., hereinafter, “Liu” in view of PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization to Saito et al., hereinafter, “Saito”.
Claim 1. (Currently Amended) Liu teaches One or more [[A]] processors comprising: [page 1] one processor 201, memory 202,
circuitry to use one or more neural networks [page 1] The three-dimensional human body network is a pre-trained convolutional neural network model and is trained through paired pictures and three-dimensional posture marking data.
to generate a three-dimensional model of a first object oriented according to a first pose based, at least in part, on: step 301: the server respectively carries out three-dimensional reconstruction on a first image containing a first human body and a second image containing a second human body to obtain three-dimensional parameters of the first human body and the second human body.
a first image of the first object oriented according to a second pose; step 302: and rendering the image based on the three-dimensional parameters to obtain a third image containing a third human body, wherein the form of the third human body is the same as that of the first human body, and the posture of the third human body is the same as that of the second human body.
and a second image of a second object oriented according to the first pose, Step step 302: and rendering the image based on the three-dimensional parameters to obtain a third image containing a third human body, wherein the form of the third human body is the same as that of the first human body, and the posture of the third human body is the same as that of the second human body.
Liu fails to explicitly teach the three-dimensional model comprises a three-dimensional occupancy RGB field. Saito, in the field of reconstruction in image data, teaches wherein the three-dimensional model comprises a three-dimensional occupancy RGB field. Figure 1, our algorithm can handle a wide range of complex clothing, such as skirts, scarfs, and even high-heels while capturing high frequency details such as wrinkles that match the input image at the pixel level. By simply adopting the implicit function to regress RGB values at each queried point along the ray, PIFu can be naturally extended to infer per-vertex colors.
[3.1. Single-view Surface Reconstruction] For surface reconstruction, we represent the groundtruth surface as a 0.5level-set of a continuous 3D occupancy field
[3.2. Texture Inference] 3.2 While texture inference is often performed on either a 2D parameterization of the surface [31, 21] or in view-space [39]
Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Liu with the teachings of Saito [Abstract] to demonstrate high-resolution and robust reconstructions on real world images from the DeepFashion dataset.
Claim 4. (Currently Amended) Liu teaches wherein: the first object is a human being; step 301: the server respectively carries out three-dimensional reconstruction on a first image containing a first human body and a second image containing a second human body to obtain three-dimensional parameters of the first human body and the second human body.
and the one or more processors generate[[s]] a parametric model of the human being based at least on part on features determined from the first image. step 302: and rendering the image based on the three-dimensional parameters to obtain a third image containing a third human body, wherein the form of the third human body is the same as that of the first human body, and the posture of the third human body is the same as that of the second human body.
Claim 8. (Currently Amended) Liu teaches wherein the one or more processors: construct[[s]] a parametric 3-D model of the first object in the first pose; step 302: and rendering the image based on the three-dimensional parameters to obtain a third image containing a third human body, wherein the form of the third human body is the same as that of the first human body, and the posture of the third human body is the same as that of the second human body.
and generate[[s]] the three-dimensional model based at least in part on the parametric 3-D model. step 302: and rendering the image based on the three-dimensional parameters to obtain a third image containing a third human body…
Claim 9. (Previously Presented) Reviewed and analyzed in the same way as claim 1. See the above analysis and rationale.
Claim 10. (Original) Liu teaches wherein the computer system: determines a set of pose parameters from the second image; determines a set of shape parameters from the first image; and generates a parametric model of the first object based at least in part on the set of pose parameters and the set of shape parameters. step 302: and rendering the image based on the three-dimensional parameters to obtain a third image containing a third human body, wherein the form of the third human body is the same as that of the first human body, and the posture of the third human body is the same as that of the second human body.
Claim 13. (Original) Liu teaches wherein the three- dimensional model is a 3-D mesh. [page 13] …outputs three-dimensional mesh models of the first human body and the second human body
Claim 17. (Previously Presented) Reviewed and analyzed in the same way as claim 1. See the above analysis and rationale.
Claim 18. (Original) Saito teaches further comprising: receiving information that specifies a point of view; [4.2. Qualitative Results]… we demonstrated that our PIFu can handle a variety of clothing, including skirts, jackets, and dresses. Our method can generate high-resolution local details while inferring reasonable 3D surfaces in unseen areas. Complete textures can also be successfully inferred from a single input image, enabling us to view our 3D models from 360 degrees
and generating, from the three-dimensional model, a 2-D image of the first object from the point of view. [4.2. Qualitative Results] Complete textures can also be successfully inferred from a single input image, enabling us to view our 3D models from 360 degrees
Claim 19. (Original) Saito teaches further comprising generating, from the three-dimensional model, a plurality of 2-D images of the first object from a corresponding plurality of points of view. Figure 1. PIFu can also be naturally extended to multi-view input images (bottom row)
[Abstract] Using PIFu, we propose an end-to-end deep learning method for digitizing highly detailed clothed humans that can infer both 3D surface and texture from a single image, and optionally, multiple input images, Figure 1
Claim 20. (Original) Saito teaches wherein the one or more neural networks are trained by at least training the one or more neural networks to produce a parametric model of the first object from an image of the first object. [Abstract] Using PIFu, we propose an end-to-end deep learning method for digitizing highly detailed clothed humans that can infer both 3D surface and texture from a single image, and optionally, multiple input images
[1. Introduction] For certain domain-specific objects, such as faces, human bodies, or known man made objects, it is already possible to infer relatively accurate 3D surfaces from images with the help of parametric models, data-driven techniques, or deep neural networks
[2. Related Work]… parametric models of human bodies and shapes [4, 35] are widely used for digitizing humans from input images
Claim 21. (Original) Saito teaches wherein the one or more neural networks are trained by at least training the one or more neural networks to produce a parametric model of the first object from an image of the first object and an image of the first object according to a different pose. [Abstract] Using PIFu, we propose an end-to-end deep learning method for digitizing highly detailed clothed humans that can infer both 3D surface and texture from a single image, and optionally, multiple input images, Figure 1
Claim 22. (Original) Saito teaches wherein the one or more neural networks are trained by at least training the one or more neural networks using two images from a segment of video of the first object. [Abstract] Using PIFu, we propose an end-to-end deep learning method for digitizing highly detailed clothed humans that can infer both 3D surface and texture from a single image, and optionally, multiple input images, Figure 1
Saito [Introduction]… high-resolution examples of monocular and textured 3D reconstructions of dynamic clothed human bodies reconstructed from a video sequence (video segment)
Claim 23. (Original) Liu teaches wherein the three-dimensional model is generated from a human parametric model. [page 13] a first image including a first human body and a second image including the first human body into the three-dimensional human body model, and outputs three-dimensional mesh models of the first human body and the second human body (parametric model of a human)
step 302: and rendering the image based on the three-dimensional parameters to obtain a third image containing a third human body, wherein the form of the third human body is the same as that of the first human body, and the posture of the third human body is the same as that of the second human body.
Claim 24. (Original) Liu teaches wherein the three-dimensional model is generated by applying, to a parametric model, two dimensional features determined from the first image. [page 10] performing texture mapping on the three-dimensional grid model of the first human body based on the first image to obtain a texture mapping result; extracting texture information corresponding to the first human body from the texture mapping result; and rendering the image based on the morphological parameter of the first human body, the texture information of the first human body and the posture parameter of the second human body to obtain a third image containing a third human body
Claim 25. (Previously Presented) Reviewed and analyzed in the same way as claim 1. See the above analysis and rationale.
Claim 26. (Previously Presented) Liu teaches wherein the one or more processors: constructs a parametric 3-D model of the first object in the first pose; step 302: and rendering the image based on the three-dimensional parameters to obtain a third image containing a third human body, wherein the form of the third human body is the same as that of the first human body, and the posture of the third human body is the same as that of the second human body.
and generates the three-dimensional model based at least in part on the parametric 3-D model and the second image. step 302: and rendering the image based on the three-dimensional parameters to obtain a third image containing a third human body,
Claim 29. (Previously Presented) Saito teaches wherein the three-dimensional model is a three-dimensional point field. [3.2. Texture Inference] PIFu enables us to directly predict the RGB colors on the surface geometry by defining s in Eq. 1 as an RGB vector field instead of a scalar field. This supports texturing of shapes with arbitrary topology and self-occlusion. However, extending PIFu to color prediction is a non-trivial task as RGB colors are defined only on the surface while the 3D occupancy field is defined over the entire 3D space. Here, we highlight the modification of PIFu in terms of training procedure and network architecture. Given sampled 3D points on the surface X ∈ Ω, the objective function for texture inference is the average of L1 error of the sampled colors…
Claim 31. (Previously Presented) Liu teaches wherein: the first object is a human being; step 301: the server respectively carries out three-dimensional reconstruction on a first image containing a first human body and a second image containing a second human body to obtain three-dimensional parameters of the first human body and the second human body.
and the one or more processors generate a parametric model of the human being based at least on part on features determined from the first image. step 302: and rendering the image based on the three-dimensional parameters to obtain a third image containing a third human body, wherein the form of the third human body is the same as that of the first human body, and the posture of the third human body is the same as that of the second human body.
Claim 32. (Previously Presented) Saito teaches wherein the one or more processors generate a two-dimensional image of the first object in the first pose from a point of view. Figure 1. Single view and multi-view
Claim 33. (New) Liu teaches wherein the three- dimensional occupancy RGB field comprises one or more binary values that map the first object to a location in a field of view. [page 1] acquiring a foreground image and a background image corresponding to the first image (understood to be binary: foreground – 1 or true and background – 0 or false)
Claim 34. (New) Liu teaches wherein the three-dimensional occupancy RGB field comprises one or more binary values that map the first object to a location in a field of view. [page 1] acquiring a foreground image and a background image corresponding to the first image (understood to be binary: foreground – 1 or true and background – 0 or false)
Claim 35. (New) Liu teaches wherein the three-dimensional RGB occupancy field comprises one or more binary values that map the first object to a location in a field of view. [page 1] acquiring a foreground image and a background image corresponding to the first image (understood to be binary: foreground – 1 or true and background – 0 or false)
Claim 36. (New) Liu teaches wherein the three-dimensional RGB occupancy field comprises one or more binary values that map the first object to a location in a field of view. [page 1] acquiring a foreground image and a background image corresponding to the first image (understood to be binary: foreground – 1 or true and background – 0 or false)
Claim(s) 3, 5-7, 14-16, 27-28 and 30 is/are rejected under 35 U.S.C. 103 as being unpatentable over CN110246209B to Liu et al., hereinafter, “Liu” in view of PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization to Saito et al., hereinafter, “Saito” and in further view of First Order Motion Model for Image Animation to Siarohin et al., hereinafter, “Siarohin”.
Claim 5. (Currently Amended) Liu teaches The one or more processors of claim 1, wherein: first object is a first human being; step 301: the server respectively carries out three-dimensional reconstruction on a first image containing a first human body and a second image containing a second human body to obtain three-dimensional parameters of the first human body and the second human body.
the second object is a second human being; step 302: and rendering the image based on the three-dimensional parameters to obtain a third image containing a third human body, wherein the form of the third human body is the same as that of the first human body, and the posture of the third human body is the same as that of the second human body.
Liu fails to explicitly teach the first human being is a different person than the second human being. Siarohin in the field of pose recognition in image data, teaches and the first human being is a different person than the second human being. Figure 1. Video and source image show different people
Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Liu with the teachings of Siarohin [Introduction – last paragraph] to handle high-resolution datasets to improve animation.
Claim 6. (Currently Amended) Liu fails to explicitly teach generate[[s]] a plurality of two-dimensional images of the first object from different points of view. Siarohin in the field of pose recognition in image data, teaches wherein the one or more processors generate[[s]] a plurality of two-dimensional images of the first object from different points of view. Siarohin Figure 1. The facial image of a person can be animated according to the facial expressions of another person, Generates pose image from different viewpoints based on the video to be simulated
Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Liu with the teachings of Siarohin [Introduction – last paragraph] to handle high-resolution datasets to improve animation.
Claim 7. (Currently Amended) Liu fails to explicitly teach the one or more neural networks is trained using at least a pair of image frames from a segment of video. Siarohin in the field of pose recognition in image data, teaches wherein the one or more neural networks is trained using at least a pair of image frames from a segment of video. Siarohin [3 Method] we follow a self-supervised strategy inspired from Monkey-Net [29]. For training, we employ a large collection of video sequences containing objects
Siarohin [3 Method] Our model is trained to reconstruct the training videos by combining a single frame and a learned latent representation of the motion in the video. Observing frame pairs, each extracted from the same video, it learns to encode motion as a combination of motion-specific keypoint displacements and local affine transformations. At test time we apply our model to pairs composed of the source image and of each frame of the driving video and perform image animation of the source object.
Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Liu with the teachings of Siarohin [Introduction – last paragraph] to handle high-resolution datasets to improve animation.
Claim 14. (Original) Liu fails to explicitly teach the first object and the second object represent a same person in different poses. Siarohin in the field of pose recognition in image data, teaches wherein the first object and the second object represent a same person in different poses. Siarohin Figure 1. The facial images of a person can be animated based on the facial expressions of another person
Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Liu with the teachings of Siarohin [Introduction – last paragraph] to handle high-resolution datasets to improve animation.
Claim 15. (Original) Liu fails to explicitly teach the second object is a human being; and the first object is a humanoid character. Siarohin in the field of pose recognition in image data, teaches the second object is a human being; and the first object is a humanoid character. Siarohin Figure 1. shows that the source image and video in the video contain different people, different animals. When transferring action to cartoon characters, technicians in this field tend to think of obtaining two images, the first object in the first image is a humanoid character, and the second object in the second image is a person.
Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Liu with the teachings of Siarohin [Introduction – last paragraph] to handle high-resolution datasets to improve animation.
Claim 16. (Original) Liu fails to explicitly teach the three- dimensional model is based at least in part on a plurality of images of the first object. Siarohin in the field of pose recognition in image data, teaches wherein the three- dimensional model is based at least in part on a plurality of images of the first object. Siarohin [3 Method] we follow a self-supervised strategy inspired from Monkey-Net [29]. For training, we employ a large collection of video sequences containing objects
Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Liu with the teachings of Siarohin [Introduction – last paragraph] to handle high-resolution datasets to improve animation.
Claim 27. (Previously Presented) Liu fails to explicitly teach the one or more neural networks is trained, based at least in part, on a 2-D image loss produced by providing the one or more neural networks with a pair of images from a segment of video. Siarohin in the field of pose recognition in image data, teaches wherein the one or more neural networks is trained, based at least in part, on a 2-D image loss produced by providing the one or more neural networks with a pair of images from a segment of video. Siarohin [3.3 Training Losses] We train our system in an end-to-end fashion combining several losses. First, we use the reconstruction loss based on the perceptual loss of Johnson et al. [19] using the pre-trained VGG-19 network as our main driving loss. The loss is based on implementation of Wang et al. [38]. With the input driving frame D and the corresponding reconstructed frame ˆ D, the reconstruction loss is written as: Lrec
Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Liu with the teachings of Siarohin [Introduction – last paragraph] to handle high-resolution datasets to improve animation.
Claim 28. (Previously Presented) Liu fails to explicitly teach generate a segment of video of the first object from a shifting point of view. Siarohin in the field of pose recognition in image data, teaches wherein the one or more processors generate a segment of video of the first object from a shifting point of view. Siarohin Figure 1 generates actions at different time points according to the human pose in the video, ultimately generating a video, that is, generating a video segment of the first object from a transferred viewpoint.
Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Liu with the teachings of Siarohin [Introduction – last paragraph] to handle high-resolution datasets to improve animation.
Claim 30. (Previously Presented) Liu teaches first object is a first human being; step 301: the server respectively carries out three-dimensional reconstruction on a first image containing a first human body and a second image containing a second human body to obtain three-dimensional parameters of the first human body and the second human body.
the second object is a second human being; step 302: and rendering the image based on the three-dimensional parameters to obtain a third image containing a third human body, wherein the form of the third human body is the same as that of the first human body, and the posture of the third human body is the same as that of the second human body.
Liu fails to explicitly teach the first human being is a different person than the second human being. Siarohin in the field of pose recognition in image data, teaches and the first human being is a different person than the second human being. Figure 1. Video and source image show different people
Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Liu with the teachings of Siarohin [Introduction – last paragraph] to handle high-resolution datasets to improve animation.
Claim(s) 11-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over CN110246209B to Liu et al., hereinafter, “Liu” in view of PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization to Saito et al., hereinafter, “Saito” and in further view of US 2021/0279456 A1 to Luo et al., hereinafter, “Luo”.
Claim 11. (Original) Liu fails to explicitly teach generates a 2-D feature map (2-D feature map) from the first image. Luo, in the field of pose recognition in image data, teaches wherein the computer system: generates a 2-D feature map (2-D feature map) from the first image; [0008] processing the feature map of the sample image (first image), FIG. 5
and the three-dimensional model is based at least in part on the 2-D feature map and the parametric model. [0015] processing, by using the 3D model comprised in the pose recognition model, a target human body feature map cropped from the feature map (2-D feature map) and the 2D key point parameters (parametric model)
Thus, before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Liu with the teachings of Luo [0055] to increase training efficiency.
Claim 12. (Original) Luo teaches wherein the computer system: generates a 3-D feature map from the parametric model; FIG. 5
and the three-dimensional model is based at least in part on the 3-D feature map and the 2-D feature map. FIG. 5
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DELOMIA L GILLIARD whose telephone number is (571)272-1681. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DELOMIA L GILLIARD/Primary Examiner, Art Unit 2661
Prosecution Insights