DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: #740 in fig. 7 and #820 in fig. 8. Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to because in figure 12 the last two digits are overlapping in characters #1204, #1206, #1216 and appear as #104, #106, and #126. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4-7, 9, 10, 12-15 are rejected under 35 U.S.C. 103 as being unpatentable over Wu, et al., "4D Gaussian Splatting for Real-Time Dynamic Scene Rendering," arXiv preprint arXiv:2310.08528v2, December 2023 (Hereinafter "Wu") in view of Zheng, Zerong, et al. "Structured local radiance fields for human avatar modeling." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022 (hereinafter "Zheng) .
Regarding claim 1, Wu teaches A method comprising:
receiving a video input (section 5 subsection “Real-world Datasets”) comprising a scene (Abstract, introduction paragraph 1, figs 4, 7 & 8) and a subject (Figs. 1, 3-5);
obtaining a three-dimensional (3D) reconstruction of the subject and the scene from the video input (section 2.2 paragraph 2 "Dynamic3DGS [33] models dynamic scenes by tracking the position and variance of each 3D Gaussian at each timestamp ti.", figs 3-5 7, 8);
generating a 3D Gaussian representation of each of the scene and the subject (section 3.1, fig 3-5, 7, 8);
generating a deformed 3D Gaussian representation of the subject by adapting the 3D Gaussian representation of the subject to the 3D reconstruction of the subject (fig. 3, section 4.2); and
rendering a visual output comprising at least one of the subject or the scene (fig 1, 3-5, 7 & 8, last paragraph of section 1) based at least in part on the deformed 3D Gaussian representation of the subject and the 3D Gaussian representation of the scene (fig. 3, section 4).
Wu fails to teach rendering a visual output comprising at least one of an animatable avatar of the subject or the scene
However Zheng teaches a video input (Abstract, fig 1 – “driving video”, section 2 paragraph 1 –“In contrast, our method bypasses the reconstruction step and directly learns an animatable avatar from RGB videos.”) and rendering a visual output comprising an animatable avatar of the subject (figs. 1, 2, 5-8, section 2 paragraph 1 and subsection “Animatable Human Avatars”, section 6 conclusion).
Zheng is considered analogous to the claimed invention as it is in the same field of three-dimensional animation. Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Wu with the teachings of Zheng in order to implement the improved rendering quality, speed, and storage consumption of Wu (section 5.2 paragraph 2) for use in animatable avatars.
Regarding claim 2, Wu in view of Zheng teaches the method of claim 1. Wu further teaches wherein the obtaining the 3D reconstruction of the subject and the scene comprises performing a structure-from-motion operation (section 4.3) to a sequence of frames in the video input (section 5 subsection “Real-world Datasets”) to obtain point cloud data of the scene (section 3.1 paragraph 1, Appendix pg. 12 col. 2 “The Neu3D dataset [25] includes 15 – 20 fixed camera setups, so it’s easy to get the SfM [47] point in the first frame. We utilize the dense point-cloud reconstruction and downsample it lower than 100k to avoid out of memory error.) and the 3D reconstruction of the subject (section 3.1 paragraph 1). Zheng further teaches pose estimation (Fig. 2, 4-8, tables 1 & 2, section 4.1 – novel pose synthesis/generation can be considered pose estimation).
Regarding claim 4, Wu in view of Zhen teaches the method of claim 1, Wu further teaches generating the deformed 3D Gaussian representation (fig. 3, section 4.2). Zheng further teaches wherein generating the representation comprises applying a forward deformation module (fig. 2 “Forward Skinning”, section 3 subsection “Discussion”) to facilitate learning of pose correctives (Fig. 4 desc – calculating local coordinate of posed space points) and linear skinning weights (section 3 – “we can transform node i to the posed space using linear blending skinning (LBS)”).
Regarding claim 5, Wu in view of Zheng teach the method of claim 4. Wu further teaches wherein the deformed 3D Gaussian representation is generated based at least in part on the pose correctives (section 5.3 subsection “Gaussian Deformation Decoder” – adjusting 3D gaussians to accurately model movements, section 5.4 subsection “Tracking with 3D Gaussians.”). Zheng further teaches wherein the representation is generated based at least in part on the pose correctives and the linear skinning weights (Fig. 2, fig. 4 desc., section 3).
Regarding claim 6, Wu in view of Zhen teaches the method of claim 5. Wu further teaches wherein the generating the deformed 3D Gaussian representation of the subject comprises applying the pose correctives to the 3D Gaussian representation of the subject (section 5.3 subsection “Gaussian Deformation Decoder” – adjusting 3D gaussians to accurately model movements, section 5.4 subsection “Tracking with 3D Gaussians.”).
Regarding claim 7, Wu in view of Zheng teaches The method of claim 6. Wu further teaches generating the deformed 3D Gaussian representation of the subject (fig. 3). Zheng further teaches wherein generating the representation of the subject comprises applying the linear skinning weights to the representation of the subject applied with the pose correctives (fig, 2 – forward skinning (applying linear weights) done after body pose 𝜽 applied (pose correctives)).
Apparatus claims 9, 10, 12-15 are drawn to the apparatus of the corresponding method claimed in claims 1, 2, 4-7. Therefore, the apparatus claims 9, 10, 12-15 correspond to the method claims 1, 2, 4-7, and are rejected for the same reasons of obviousness as used above.
Claims 3, 11 are rejected under 35 U.S.C. 103 as being unpatentable over Wu in view of Zheng and in further view of Kim (US 20130182894 A1).
Regarding claim 3, Wu in view of Zheng teach the method of claim 2. Wu in view of Zheng fail to teach wherein the structure-from-motion operation and the pose estimation are performed concurrently.
However Kim teaches wherein the structure-from-motion operation and the pose estimation are performed concurrently (paragraph [0173]). Kim describes estimating camera pose at the same time as a structure from motion operation estimates coordinates. While the teachings of Kim are directed towards a camera pose rather than a subject’s pose, the simultaneous execution described is analogous to what is claimed in claim 3. Furthermore, Kim is considered analogous to the claimed invention as it is in the same field of computer graphics. Therefore it would have been obvious to one of ordinary skill in the art, before the effective filing date, to incorporate the teachings of Kim with Wu and Zheng in order to perform the SfM and pose estimation operations concurrently in order to enable faster processing times.
Apparatus claim 11 is drawn to the apparatus of the corresponding method claimed in claim 3. Therefore, the apparatus claim 11 corresponds to the method claim 3, and is rejected for the same reasons of obviousness as used above.
Claims 8, 16, 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Wu in view of Zheng and in further of Kerbl et al., "3D Gaussian Splatting for Real-Time Radiance Field Rendering" SIGGRAPH 2023, volume 42, July 2023 (hereinafter "Kerbl") .
Regarding claim 8, Wu in view of Zheng teaches the method of claim 1. Zheng further teaches the visual output of the subject or the scene comprising the at least one of the animatable avatar.
Wu in view of Zheng fail to teach wherein the visual output of the subject or the scene is rendered using differentiable Gaussian rasterization.
However, Kerbl teaches wherein the visual output of the subject or the scene is rendered using differentiable Gaussian rasterization (fig. 2, section 6). Kerbl describes rendering visual output by differentially rasterizing the Gaussian representation of a given scene which can be considered analogous to differentiable Gaussian rasterization. Kerbl is considered analogous to the claimed invention as it is in the same field of three dimensional computer graphics. Therefore it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Kerbl with Wu in view of Zheng to improve rasterization speeds (Kerbl, section 8 paragraphs 1 & 2).
CRM claim 16 is drawn to the CRM of the corresponding method claimed in claim 8. Therefore, the CRM claim 16 corresponds to the method claim 8, and is rejected for the same reasons of obviousness as used above.
Regarding claim 18, Wu in view of Zheng and Kerbl teach the non-transitory computer-readable medium of claim 16.
Wu further teaches generating the deformed 3D Gaussian representation (fig. 3, section 4.2). Zheng further teaches wherein the instructions that cause the one or more processors to generate the representation of the subject further cause the one or more processors to apply a forward deformation module (fig. 2 “Forward Skinning”, section 3 subsection “Discussion”) to facilitate learning of pose correctives (Fig. 4 desc – calculating local coordinate of posed space points) and linear skinning weights (section 3 – “we can transform node i to the posed space using linear blending skinning (LBS)”).
Regarding claim 19, Wu in view of Zheng and Kerbl teach The non-transitory computer-readable medium of claim 18. Wu further teaches wherein the deformed 3D Gaussian representation is generated based at least in part on the pose correctives (section 5.3 subsection “Gaussian Deformation Decoder” – adjusting 3D gaussians to accurately model movements, section 5.4 subsection “Tracking with 3D Gaussians.”). Zheng further teaches wherein the representation is generated based at least in part on the pose correctives and the linear skinning weights (Fig. 2, fig. 4 desc., section 3).
Regarding claim 20, Wu in view of Zheng and Kerbl teach The non-transitory computer-readable medium of claim 19. Wu further teaches wherein the instructions that cause the one or more processors to generate the deformed 3D Gaussian representation of the subject further cause the one or more processors to apply the pose correctives to the 3D Gaussian representation of the subject (section 5.3 subsection “Gaussian Deformation Decoder” – adjusting 3D gaussians to accurately model movements, section 5.4 subsection “Tracking with 3D Gaussians.”). Zheng further teaches wherein the instructions that cause the one or more processors to generate the representation of the subject further cause the one or more processors to apply the linear skinning weights to representation of the subject applied with the pose correctives (fig, 2 – forward skinning (applying linear weights) done after body pose 𝜽 applied (pose correctives)).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Wu in view of Zheng and Kerbl and in further view Kim.
Regarding claim 17, Wu in view of Zheng and Kerbl teach the non-transitory computer-readable medium of claim 16. Wu in view of Zheng and Kerbl fail to teach wherein the structure-from-motion operation and the pose estimation are performed concurrently.
However Kim teaches wherein the structure-from-motion operation and the pose estimation are performed concurrently (paragraph [0173]). The motivation to combine Kim with Wu in view of Zheng and Kerbl would have been the same as that of claim 3.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Rhodin, Helge, et al. "General automatic human shape and motion capture using volumetric contour cues." European conference on computer vision. Cham: Springer International Publishing, 2016.
W. Sun, Y. Che, Y. Guo and H. Huang, "Neural Reconstruction of Relightable Human Model from Monocular Video," 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France, 2023, pp. 397-407, doi: 10.1109/ICCV51070.2023.00043.
Black (US 10,529,137 B1)
Black (US 10,679,046 B1)
Lin (US 2024/0104831 A1)
Zhao (US 2024/0311976 A1)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Aidan W McCoy whose telephone number is (571)272-5935. The examiner can normally be reached 8:00 AM-5:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571)272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AIDAN W MCCOY/Examiner, Art Unit 2611
/TAMMY PAIGE GODDARD/Supervisory Patent Examiner, Art Unit 2611