DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 21, 22, 25, and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (U.S. Publication 2021/0287007) in view of Ajani (U.S. Publication 2021/0335030).
As to claim 1, Wang discloses a computer-implemented method, comprising:
obtaining a reference three-dimensional (3D) representation of a reference image associated with a character (p. 2, section 0022; p. 3, section 0030; p. 4, sections 0038-0039; p. 6, section 0056-p. 7, section 0057; p. 7, section 0060; over a number of frames, a 3D representation of a reference image with a stick person character is obtained);
obtaining a raw 3D representation of an input frame from an input video associated with the character (p. 2, section 0022; p. 3, section 0030; p. 4, sections 0038-0039; p. 6, section 0056-p. 7, section 0057; p. 7, section 0060; a number of frames are obtained; in one of the raw frames from input video representing the 3D object, an associated character is occluding an object);
processing, using one or more neural networks, the reference 3D representation of the reference image and the raw 3D representation of the input frame to generate a fused 3D representation that recovers occluded areas from the raw 3D representation using the reference 3D representation (p. 2, section 0022; p. 3, section 0030; p. 3-4, section 0034; p. 4, sections 0038-0039; p. 6, section 0056-p. 9, section 0095; the raw input frame representation with occluded areas is fused with the overall reference 3D representation using inpainting, recovering the occluded areas);
Wang does not disclose, but Ajani discloses performing volume rendering on the fused 3D representation to generate an output two-dimensional (2D) image (figs. 7-8; p. 3, section 0040-p. 4, section 0048; a fused 3D representation made up of a 3D image and a clipping volumetric surface is used to un-occlude features and the fused 3D representation with un-occluded features is volumetrically rendered as an image on a 2D screen as shown in the figures). The motivation for this is to allow for rapid appreciation of focus anatomy and surrounding context while solving the problem of blocked focus regions (p. 1, section 0005). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Wang to perform volume rendering on the fused 3D representation to generate an output two-dimensional (2D) image in order to allow for rapid appreciation of focus anatomy and surrounding context while solving the problem of blocked focus regions as taught by Ajani.
As to claim 21, Wang discloses wherein at least one of the steps of obtaining, processing and performing are performed on a server or in a data center to generate the output 2D image, and the output 2D image is streamed to a user device (figs. 7-8; p. 3, section 0040-p. 4, section 0048; p. 13, section 0108; the computing system described can communicate/stream a result, which in this instance would be an output 2D image, to a user/client device).
As to claim 22, Ajani discloses wherein at least one of the steps of obtaining, processing and performing are performed within a cloud computing environment (p. 3, section 0029; obtaining is from a cloud/server).
As to claim 25, see the rejection to claim 1. Further, Wang discloses one or more processors and a non-transitory computer-readable medium having processor-executable instructions stored thereon, wherein the processor-executable instructions, when executed by the one or more processors, facilitate the method (p. 12, section 0101-p. 13, section 0105).
As to claim 27, see the rejections to claims 1 and 25.
Claims 2, 4, 26, and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Ajani and further in view of Deng (U.S. Publication 2022/0237862).
As to claim 2, Wang does not disclose, but Deng discloses wherein obtaining the reference 3D representation comprises:
receiving a reference image of the character using a sensor, wherein the reference image is a front novel view of the character (p. 7, sections 0092-0093; p. 10, section 0128; the process begins by receiving a character/person frontal view from a camera, which is novel since it is not simply copying a previous image; the frontal view is set to be a key/reference frame image); and
inputting the reference image into a first transformation model to transform the reference image into the reference 3D representation (p. 7, sections 0094-0101; p. 11, section 0131; the reference frontal view frame is among the frames transformed to 3D). The motivation for this is to refine a pose. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Wang and Ajani to receive a reference image of the character using a sensor, wherein the reference image is a front novel view of the character and input the reference image into a first transformation model to transform the reference image into the reference 3D representation in order to refine a pose as taught by Deng.
As to claim 4, Wang does not disclose but Deng discloses wherein obtaining the reference 3D representation further comprises:
receiving one or more additional images of the character using the sensor, wherein the one or more additional images are side views of the character (p. 7, sections 0092-0093; camera images with head rotation of 60 degrees, reading on side views, of the character are received), and
wherein inputting the reference image into the first transformation model comprises inputting the reference image and the one or more additional images into the first transformation model to transform the reference image and the one or more additional images into the reference 3D representation (p. 7, section 0094; each of the images is used to generate the 3D model).
Motivation for the combination is given in the rejection to claim 2.
As to claim 26, see the rejection to claim 2.
As to claim 28, see the rejection to claim 2.
Claim 3 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Ajani and Deng and further in view of Trevithick (“Live 3D Portrait: Real-Time Radiance Fields for Single-Image Portrait View Synthesis”).
As to claim 3, Deng discloses wherein obtaining the raw 3D representation comprises inputting the input frame into a second transformation model to transform the input frame into the raw 3D representation (p. 7, sections 0094-0101; p. 11, section 0131; multiple frames are transformed into multiple 3D representations, one of which would read on a reference representation and one of which would read on a raw representation). Motivation for the combination is given in the rejection to claim 2.
Wang in view of Ajani and Deng does not disclose, but Trevithick discloses wherein the models are Live 3D Portrait (LP3D) models (p. 12, section 1; input 2D portrait images are converted to 3D live/in real-time). The motivation for this is to not require real multiview images, saving time and cost. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Wang, Ajani, and Deng to use Live 3D Portrait (LP3D) models in order to not require real multiview images, saving time and cost as taught by Trevithick.
As to claim 5, Deng discloses a reference 3D representation and a raw 3D representation as noted above. Motivation for the combination is given in the rejection to claim 2. The combination of Wang, Ajani, and Deng does not disclose but Trevithick discloses wherein the representations are triplane representations (p. 13-14, section 3), wherein the triplane representations are data structures that have four dimensions, wherein the first and second dimensions of the triplane representations are based on a height and width of the reference image or the input frame (p. 14-15, section A2; 1st and 2nd dimensions are 2D pixel coordinates which would be on a scale based on image/frame height and width), the third dimension is associated with a set of orthogonal planes, and the fourth dimension indicates channels of feature maps for each of the set of orthogonal planes (fig. 3; p. 13-14, section 3; the triplane also indicates 3 axis-aligned orthogonal planes and 32 channels for feature maps for each plane). Motivation for the combination is given in the rejection to claim 3.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Ajani and further in view of Trevithick.
As to claim 12, Wang does not disclose but Trevithick discloses generating, using a synthetic data generator, a first training 3D representation and a second training 3D representation; and training a fusion architecture using the first and the second training 3D representations, wherein the fusion architecture comprises a first neural network and a second neural network (fig. 3; first synthetic 3D data is generated for a reference camera and second synthetic 3D data is generated for another camera; the data is fed to 2 discriminator neural networks to train; the end result of the architecture is a fused final image). Motivation for the combination is given in the rejection to claim 3.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Ajani and Trevithick and further in view of Xu (U.S. Publication 2024/0265621).
As to claim 13, Wang does not disclose, but Xu discloses wherein the synthetic data generator comprises a random vector generator (p. 2, section 0015; p. 3, section 0042; a random latent vector is generated), an animated mesh generator (p. 3, sections 0038-0040; p. 4, sections 0051-0052; p. 6, section 0068; a FLAME model is used to generated meshes for animation), and a synthetic 3D representation generator (p. 4, section 0043; a synthesizer is used to synthesize 3D representations/models of a head), and wherein generating the first training 3D representation comprises:
obtaining a random vector using the random vector generator (p. 2, section 0015; p. 3, section 0042; a random latent vector is generated);
obtaining a first animated mesh of an animated character using the animated mesh generator, wherein the first animated mesh indicates coefficients and landmarks associated with the animated character (p. 3, sections 0038-0040; p. 4, sections 0051-0052; p. 5, sections 0056-0057; p. 6, section 0068; a FLAME model is used to generate meshes of the character face for animation, including coefficients and landmarks); and
inputting the random vector and the first animated mesh into the synthetic 3D representation generator to generate the first training 3D representation (p. 4, section 0043; p. 5, section 0057-p. 6, section 0065; p. 7, section 0095; 3D training representations are synthesized using the FLAME-generated meshes and random Gaussian latent feature vectors).
The motivation for this is to support improved extrapolation to unseen novel expressions. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Wang, Ajani, and Trevithick to obtain a random vector using the random vector generator, obtain a first animated mesh of an animated character using the animated mesh generator, wherein the first animated mesh indicates coefficients and landmarks associated with the animated character, and input the random vector and the first animated mesh into the synthetic 3D representation generator to generate the first training 3D representation.
in order to support improved extrapolation to unseen novel expressions as taught by Xu.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Ajani and Trevithick and further in view of Fu (U.S. Publication 2026/0045118).
As to claim 14, Trevithick discloses 3D training representations as noted in the rejection to claim 12. Wang in view of Ajani and Trevithick does not disclose, but Fu discloses wherein the first training representation indicates a synthetic character having a first facial expression, and wherein the second training representation indicates the same synthetic character having a second facial expression that is different from the first facial expression (p. 2, sections 0023-0024; p. 2, sections 0032-0033; p. 6, sections 0086-0087; training is performed using different frames that represent different expressions of a character/user, which can be virtual/synthetic). The motivation for this is to reduce jitter. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Wang, Ajani, and Trevithick to have the first training representation indicate a synthetic character having a first facial expression, and have the second training representation indicate the same synthetic character having a second facial expression that is different from the first facial expression in order to reduce jitter as taught by Fu.
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Ajani and further in view of Kuo (U.S. Publication 2025/0157148).
As to claim 23, Wang does not disclose, but Kuo discloses wherein at least one of the steps of obtaining, processing and performing are performed for training, testing, or certifying a neural network employed in a machine, robot, or autonomous vehicle (p. 3, section 0028; p. 4, section 0036; p. 5, sections 0047-0049; processing for fusing feature map images is used for training a neural network employed in a machine with a processor). The motivation for this is to allow classification of input images. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Wang and Ajani to have at least one of the steps of obtaining, processing and performing be performed for training, testing, or certifying a neural network employed in a machine, robot, or autonomous vehicle in order to allow classification of input images as taught by Kuo.
Claim 24 is rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Ajani and further in view of Stafford (U.S. Publication 2025/0037621).
As to claim 24, Wang does not disclose, but Stafford discloses wherein at least one of the steps of obtaining, processing and performing is performed on a virtual machine comprising a portion of a graphics processing unit (p. 5, section 0053; p. 10, section 0090; a virtual machine on a GPU can implement processing to recover occluded areas). The motivation for this is to match the needs of a process to the capabilities of a processor. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Wang and Ajani to have at least one of the steps of obtaining, processing and performing be performed on a virtual machine comprising a portion of a graphics processing unit in order to match the needs of a process to the capabilities of a processor as taught by Stafford.
Conclusion
Claims 6-11 and 15-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON M RICHER whose telephone number is (571)272-7790. The examiner can normally be reached 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON M RICHER/Primary Examiner, Art Unit 2617
Prosecution Insights