DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-21 are pending.
Prior art:
D1: Kocabas et al (PACE, 2023)
D2: Xie et al (OmniControl, Apr 2024)
D3: Zhang et al (Adding Conditional Control, 2023)
D4: Sullivan et al (US20230342944A1)
D5: Li et al (CN110580740A)
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claim(s) 1-6, 12 and 17-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over D1 in view of D2 and further in view of D3.
Regarding claims 1, 17 and 20, D1 teaches a computer-implemented method, comprising:
determining an initial articulated object motion of an articulated object based on an input video comprising a plurality of frames that depict motion of the articulated object,
(D1, “We then estimate body pose information for each detected bounding box using the state-of-the-art method HybrIK. HybrIK provides body poses in the camera coordinate frame which we represent as
PNG
media_image1.png
26
158
media_image1.png
Greyscale
”, [Sec 3.1, p4:c1]; determining initial articulated body poses/motion from the frames of an input video)
wherein the input video is obtained by a non-stationary camera,
(D1, "The input to PACE is an in-the-wild RGB video I = {I1, ···, IT} with T frames captured by a moving camera.", [Sec 3, p3-c2]; determining initial body/articulated object motion frames based on an input video; obtaining the input video from a non-stationary, i.e., moving, camera)
wherein the initial articulated object motion is in a local coordinate system associated with the non-stationary camera;
(D1, "HybrIK provides body poses in the camera coordinate frame which we represent as
PNG
media_image1.png
26
158
media_image1.png
Greyscale
. The super-script c corresponds to the camera coordinate frame.", [Sec 3.1, p4-c1]; the initial object motion is in the local coordinate system of the moving camera)
determining, based on the input video, an initial camera motion in a global coordinate system that is a real-world coordinate system;
(D1, "... to a consistent world coordinate frame ... we leverage a data-driven SLAM method, namely DROID-SLAM, which uses the information of the static scene to estimate per-frame camera-to-world transforms
PNG
media_image2.png
27
97
media_image2.png
Greyscale
", [Sec 3.1, p4-c1]; determining the initial camera motion (camera-to-world transforms) in a global/world coordinate system based on the video)
generating a plurality of control signals based on the initial articulated motion object;
(D1, "HybrIK provides body poses in the camera coordinate frame which we represent as
PNG
media_image1.png
26
158
media_image1.png
Greyscale
”, [Sec 3.1, p4-c1]; D2, “Given a prompt p, such as text, and an additional spatial control signal
PNG
media_image3.png
21
84
media_image3.png
Greyscale
, our goal is to generate a human motion sequence
PNG
media_image4.png
16
74
media_image4.png
Greyscale
”, [Sec. 3, p3]; D1 teaches computing an initial articulated object motion; D2 teaches that spatial control signals (global joint locations) are the natural conditioning input for a motion diffusion model. A person of ordinary skill in the art refining D1’s initial articulated motion would have found it obvious to derive the spatial control signals required by D2 directly from that same initial articulated object motion)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate D2’s spatial-control paradigm into D1 in order to supply the control signals needed for subsequent controlled diffusion-based refinement of the initial articulated motion. The combination of D1 and D2 also teaches other enhanced capabilities.
The combination of D1 and D2 further teaches:
generating a plurality of intermediate denoised motions based on inputting the plurality of control signals into a control branch of a controlled motion denoiser, and inputting a plurality of latent motions associated with the initial articulated object motion into a motion diffusion model of the controlled motion denoiser,
(D1, “we use a variational autoencoder (VAE), which learns a latent representation z of human motion”, [Sec. 3.2, p4:c2]; “we optimize the latent codes
PNG
media_image5.png
26
69
media_image5.png
Greyscale
instead of directly optimizing the local body motion
PNG
media_image6.png
25
94
media_image6.png
Greyscale
”, [Sec. 3.2, p5:c1]; D3, "To add a ControlNet to such a pre-trained neural block, we lock (freeze) the parameters ϴ of the original block and simultaneously clone the block to a trainable copy with parameters ϴc (Figure 2b). The trainable copy takes an external conditioning vector c as input.", [Sec. 3.1, p:c1]; D2, "Inspired by classifier guidance and ControlNet, we design hybrid guidance, consisting of spatial and realism guidance ... the realism guidance uses a neural network similar to ControlNet to adjust the output to generate coherent and realistic motion.", [Sec. 2.2, p3]; "we introduce the realism guidance that outputs the residuals w.r.t. the features in each attention layer of the motion diffusion model.", [Sec. 1, p2]; D1 teaches latent motions derived from the initial articulated object motion; D3 teaches the control-branch architecture; D2 expressly applies a ControlNet-style control pathway to a motion diffusion model that accepts spatial control signals and produces refined intermediate motions. Together they teach a controlled motion denoiser in which control signals enter the control branch and latent motions enter the motion diffusion model)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the ControlNet-style control branch of D3 and its motion-diffusion realization in D2 into D1’s latent-motion refinement pipeline in order to obtain fine-grained spatial control over the intermediate denoised motions while retaining D1’s strong motion prior. The combination of D1, D2 and D3 also teaches other enhanced capabilities.
The combination of D1, D2 and D3 further teaches:
D3D3determining a global camera motion and a global articulated object motion based on the plurality of intermediate denoised motions and the initial camera motion,
(D1, "optimize the latent code z = {zΦ, zθ} and camera-to-world transforms {Rt, sTt} with correct scale s", [Sec 3.3, p5-c1]; "The final output is coherent human and camera motion in global space.", [Fig. 2, caption, p4]; D1 teaches jointly optimizing refined latent motion codes together with the initial camera-to-world transforms to obtain the final global camera motion and global articulated object motion. Having modified D1’s latent motion refinement process using the controlled denoiser of D3 and D2 as detailed above, a person of ordinary skill in the art would naturally utilize the resulting modified latent motions (i.e., the claimed intermediate denoised motions) in D1’s subsequent joint optimization step. The motivation for doing so would be to complete D1’s established motion-estimation pipeline, thereby determining the final global camera and object motions while carrying forward the benefits of the enhanced spatial control achieved in the preceding step)
wherein the global camera motion and the global articulated object motion are both in the global coordinate system; and
(D1, "Our goal is to estimate both the camera motion and the motion of all visible people in the video in a global world coordinate system.", [Sec 3], p3-c2; both motions are resolved in the global coordinate system)
outputting the global camera motion and the global articulated object motion.
(D1, "The final output is coherent human and camera motion in global space.", Fig. 2, caption; outputting the finalized global motions)
Regarding claims 2 and 18, the combination of D1, D2 and D3 teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 1, further comprising:
converting the initial articulated object motion from the local coordinate system to the global coordinate system, and
(D1, "the estimated translation τˆtc and root orientation Φˆtc must be transformed from camera coordinates to a consistent world coordinate frame.", [Sec 3.1], p4-c1; transforming/converting local object motions to a global world coordinate frame)
wherein determining the global articulated object motion based on the plurality of intermediate denoised motions comprises refining the initial articulated object motion that has been converted to the global coordinate system using the plurality of intermediate denoised motions.
(D1, "In the remainder of this paper, our goal is to refine these initial estimates via human motion priors and the background scene features, while recovering accurate global camera trajectories.", [Sec 3.1], p4-c2; refining the converted initial global object motions using the intermediate synthesized motion priors)
Regarding claims 3 and 19, the combination of D1, D2 and D3 teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 1,
wherein determining the initial camera motion is based on using a Simultaneous Localization and Mapping (SLAM) algorithm, and
(D1, "we leverage a data-driven SLAM method, namely DROID-SLAM", [Sec 3.1], p4-c1; using a SLAM algorithm to determine the camera motion)
wherein determining the initial articulated object motion is based on using a 3-dimensional (3-D) Pose Estimator.
(D1, "Camera-Space Human Pose Estimation.", [Sec 2], p4-c1; "estimate body pose information for each detected bounding box using the state-of-the-art method HybrIK", [Sec 3.1], p4-c1; using a state-of-the-art 3D Pose Estimator (HybrIK) to determine the object's initial motion)
Regarding claim 4, the combination of D1, D2 and D3 teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 1, further comprising:
training the motion diffusion model using one or more first datasets;
(D3, "LAION-5B [79] dataset that was used to train Stable Diffusion", [Sec 1], p2-c1; D1, "We initialize the latent codes using the pre-trained encoders", [Sec 3.2], p5-c1; both of D1 and D3 teaches utilizing pre-trained diffusion models/encoders initially trained on massive first datasets)
subsequent to training the motion diffusion model, freezing parameters of the trained motion diffusion model; and
(D3, "lock (freeze) the parameters Θ of the original block and simultaneously clone the block to a trainable copy", [Sec 3.1], p4-c1; freezing the main parameters of the previously trained diffusion model)
after connecting the control branch to the trained motion diffusion model, training the control branch using one or more second datasets.
(D3, "The trainable copy is connected to the locked model with zero convolution layers", [Sec 3.1], p4-c1; "train a ControlNet for the SD V2 with the same depth conditioning but only use 200k training samples", [Sec 4.3], p7-c2; training the newly connected control branch on a specific second dataset while keeping the main model frozen)
Regarding claim 5, the combination of D1, D2 and D3 teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 1, wherein generating the plurality of intermediate denoised motions comprises:
generating a first noisy latent distribution based on combining the initial articulated object motion with a noise signal;
(D3, "image diffusion algorithms progressively add noise to the image and produce a noisy image zt, where t represents the number of times noise is added.", [Sec 3.3], p5-c1; creating a noisy latent distribution by progressively combining inputs with a noise signal)
sampling the first noisy latent distribution to generate initial latent motion from the plurality of latent motions; and
(D3, "Given a set of conditions including time step t, text prompts Ct, as well as a task-specific condition Cf, image diffusion algorithms learn a network ϵθ to predict the noise added to the noisy image zt", [Sec 3.3], p5-c1; sampling and predicting from the noise distributions to recover latent information)
processing the initial latent motion according to a first control signal, from the plurality of control signals, to produce a first intermediate denoised motion, wherein the first control signal is the initial articulated object motion.
(D3, "predict the noise added to the noisy image zt with L = ... eq. (5)", [Sec 3.3]; D1, "produce the orientation Φˆt, local body pose θˆt, and joint contacts κˆt for a given time step: D: ... eq. (1)", [Sec 3.2], p4-c2; processing latent information conditioned on initial signals to produce the intermediate motion representation)
Regarding claim 6, the combination of D1, D2 and D3 teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 5, wherein generating the plurality of intermediate denoised motions further comprises:
updating the initial articulated object motion to generate one or more updated articulated object motions based on the first intermediate denoised motion;
(D1, "we adopt a multi-stage optimization pipeline, with different parameters optimized in different stages to avoid bad minima.", [Sec 3.3], p6-c1; iteratively updating the articulated object's motion through sequential optimization stages based on intermediate states)
generating one or more second noisy latent distributions based on combining the one or more updated articulated object motions with the noise signal;
(D3, "image diffusion algorithms progressively add noise to the image and produce a noisy image zt", [Sec 3.3], p5-c1; recursively generating noisy distributions over multiple steps/stages with updated data)
sampling the one or more second noisy latent distributions to generate one or more second latent motions from the plurality of latent motions; and
(D3, " Given a set of conditions including time step t, text prompts Ct, as well as a task-specific condition Cf, image diffusion algorithms learn a network ϵθ to predict the noise added to the noisy image zt", [Sec 3.3], p5-c1; iterative sampling of the latent representation during diffusion)
processing the one or more second latent motions according to one or more second control signals, from the plurality of control signals, to produce one or more second intermediate denoised motions, wherein the one or more second control signals are based on the one or more updated articulated object motions.
(D1, "In Stage-2, we incorporate the global orientation latent code zΦ to jointly adjust the subjects’ global orientation and camera. In Stage-3, we optimize the local body motion zθ as well.", [Sec 3.3], p6-c1; using updated multi-stage parameters as secondary control signals to iteratively produce refined intermediate motions)
Regarding claim 12, the combination of D1, D2 and D3 teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 1, wherein outputting the global camera motion and the global articulated object motion comprises:
using the global camera motion and the global articulated object motion to control one or more robotic systems.
(D1, "Jointly estimating global human and camera motion from dynamic RGB videos is an important problem with numerous applications in areas such as robotics", [Sec 1], p1-c2; utilizing the global motions for robotic system applications)
Regarding claim 21, the combination of D1, D2 and D3 teaches its/their respective base claim(s).
The combination further teaches the computer-implemented method of claim 1, wherein
the plurality of control signals are input into the control branch to control the motion diffusion model and
(D2, "The realism guidance takes in the same textual prompt p as the motion diffusion model, as well as the spatial control signal c.", "adds the learned feature corrections to the corresponding layers in the motion diffusion model to amend the generated motions implicitly.", [Section 3.2, p6]; the control signals are input into a trainable copy/control branch (realism guidance) to control the motion diffusion model)
the plurality of latent motions are input into the motion diffusion model to generate the plurality of intermediate denoised motions.
(D2, "At the denoising diffusion step, the model takes the text prompt and a noised motion sequence xt as input and estimates the clean motion x0.", [Figure 2 caption, p4]; "The model learns the reversed diffusion process of gradually denoising xt starting from the pure Gaussian noise xT", [Section 3.1, p4]; inputting noisy intermediate/latent motions xt into the motion diffusion model to perform iterative steps that generate intermediate denoised motions)
Claim(s) 13, 15 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over D1 in view of D2 and further in view of D3 and D4.
Regarding claim 13, the combination of D1, D2 and D3 teaches its/their respective base claim(s).
The combination does not expressly disclose but D4 teaches the computer-implemented method of claim 1, wherein at least one of the steps of obtaining, generating, determining, and outputting are performed on a server or in a data center to determine the global camera motion and the global articulated object motion, and the global camera motion and the global articulated object motion are streamed to a user device.
(D4, Fig. 6; "The low-speed expansion ports 617, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet) may be coupled to the one or more input/output devices 641. The computing device 600 may be connected to a server 653 and a rack server 655. The computing device 600 may be implemented in several different forms. For example, the computing device 600 may be implemented as part of the rack server 655.", [0087]; "The transceiver 1101 can, for example, include a transmitter enabled to transmit one or more signals over one or more types of wireless communication networks and a receiver to receive one or more signals transmitted over the one or more types of wireless communication networks.", [0099]; the motion estimation steps are executed on a rack server/data center, with results transmitted wirelessly to external devices via the transceiver => server/data center processing with outputs streamed to a user device)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of D4 into D1, D2 and D3 in order to offload the computationally intensive motion estimation and diffusion-based denoising steps to a server or data center and stream the resulting global camera and articulated object motion outputs to a user device via wireless communication. The combination of D1, D2, D3 and D4 also teaches other enhanced capabilities.
Regarding claim 15, the combination of D1, D2 and D3 teaches its/their respective base claim(s).
The combination of D1, D2, D3 and D4 further teaches the computer-implemented method of claim 1, wherein at least one of the steps of obtaining, generating, determining, and outputting are performed for training, testing, or certifying a neural network employed in a machine, robot, or autonomous vehicle.
(D4, Fig. 2; "The motion state head 130 b for motion state estimation and the motion vector head 130 b for motion vector prediction are trained jointly with the overall loss L=Lvector+0.1Lstate. In an example embodiment, in the testing phase, the estimated binary motion states are used to refine the predicted motion vectors", [0045]; "In an example, in the training phase, a pretrained segmentation backbone is used to focus on motion training and accelerate the training time.", [0055]; Fig. 7; "The autonomous device 704 may be an autonomous or semi-autonomous controlled vehicle for which the control inputs are generated by using some embodiments." [0089]; D4 teaches both a training phase and a testing phase of the motion estimation NN, deployed in an autonomous vehicle)
Regarding claim 16, the combination of D1, D2 and D3 teaches its/their respective base claim(s).
The combination of D1, D2, D3, and D4 further teaches computer-implemented method of claim 1, wherein at least one of the steps of obtaining, generating, determining, and outputting is performed on a virtual machine comprising a portion of a graphics processing unit.
(D4, "such software may be written using any of a number of suitable programming languages and/or programming or scripting tools, and also may be compiled as executable machine language code or intermediate code that is executed on a framework or virtual machine.", [0118]; "The processor 1111 can be implemented using one or more application specific integrated circuits (ASICs), central and/or graphical processing units (CPUs and/or GPUs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, embedded processor cores, electronic devices, other electronic units designed to perform the functions described herein, or a combination thereof.", [0110]; the motion estimation software is compiled as intermediate code executed on a virtual machine, and that the processor implementing those steps includes a GPU => "a virtual machine comprising a portion of a graphics processing unit.")
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over D1 in view of D2 and further in view of D3 and D5.
Regarding claim 14, the combination of D1, D2 and D3 teaches its/their respective base claim(s).
The combination does not expressly disclose but Li teaches the computer-implemented method of claim 1, wherein at least one of the steps of obtaining, generating, determining, and outputting are performed within a cloud computing environment.
(D5, Fig. 1; "Sending the local three-dimensional models and the track node information to a cloud terminal through an RPC protocol; constructing global constraints at a cloud end according to the local three-dimensional models and the track node information;", p5; performing key computational steps at a "cloud terminal"/"cloud end")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of D5 into D1, D2 and D3 in order to perform the global motion estimation and optimization steps within a cloud computing environment (cloud server/client), enabling centralized, distributed processing across multiple autonomous agents. The combination of D1, D2, D3 and D5 also teaches other enhanced capabilities.
Allowable Subject Matter
Claim(s) 7-11 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening Claim(s).
The following is a statement of reasons for the indication of allowable subject matter:
Claim(s) 7 and 10 recite(s) limitation(s) related to diffusion inpainting of known/unknown motions with COIN-SDS loss to optimize global outputs; and joint optimization using COIN-SDS loss and point-cloud-based human-scene relation loss. There are no explicit teachings to the above limitation(s) found in the prior art cited in this office action and from the prior art search.
Claim(s) 8-9 and Claim 11 depend on claims 7 and 10, respectively.
Response to Arguments
Applicant's arguments filed on 7/29/2026 with respect to one or more of the pending claims have been fully considered but are moot in view of the new ground(s) of rejection.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANXUN YANG whose telephone number is (571)272-9874. The examiner can normally be reached on MON-FRI: 8AM-5PM Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center. for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272- 1000.
/JIANXUN YANG/
Primary Examiner, Art Unit 2662 9/19/2026
Prosecution Insights