DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. CN 202210313961.2, filed on 03/28/2022.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 5, 9-12, 14-16 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ho (Spatial Relationship Preserving Character Motion Adaptation, 2010).
Regarding Claim 14, Ho teaches a device for controlling a motion of a virtual character (Abstract: “This paper presents a new method for editing and retargeting motions that involve close interactions between body parts of single or multiple articulated characters, such as dancing, wrestling, and sword fighting, or between characters and a restricted environment, such as getting into a car”; Section 6.3: “For all the retargeting examples shown in this paper, the computation required for each motion is around 1minute for an animation of 100 frames, using one core of a Core i72.67GHz CPU”; Refer to Figure 1 for a visual of virtual characters and motion retargeting. Notes: motion retargeting is the process of controlling the motion of a target character by adapting the motion of a source character through position data. The method of controlling a motion of a virtual character as described in Ho is performed via a CPU device), the device comprising:
at least one processor (Section 6.3: “For all the retargeting examples shown in this paper, the computation required for each motion is around 1minute for an animation of 100 frames, using one core of a Core i72.67GHz CPU”); and a storage apparatus (Section 6.3: “For all the retargeting examples shown in this paper, the computation required for each motion is around 1minute for an animation of 100 frames, using one core of a Core i72.67GHz CPU”. Notes: A storage apparatus is necessitated by Ho, as data processing and computation inherently requires data to be stored), configured to store at least one computer program, wherein the at least one computer program, when executed by the at least one processor, causes the at least one processor to perform:
acquiring original motion data (Section 3, Overview: “We give an overview of our method in this section. First, the data of the original characters and the motion is loaded into our system”),
wherein the original motion data is position data of a plurality of skeletal joint points (Section 4, Interaction Mesh: “In this section, we describe how we compute the interaction meshes for a given motion. We assume that the mesh characters are rigged with skeletons”; Section 4, Interaction Mesh: “The postures of the characters are represented by the positions of the joints”) of an original model in a case that the original model performs a target motion (Section 3, Overview: “We give an overview of our method in this section. First, the data of the original characters and the motion is loaded into our system”; Figure 1: “Our system can retarget motions of close interactions to characters of different morphologies. A judo interaction (red / orange pair) retargeted to characters of different sizes);
determining initial motion data of the virtual character based on the original motion data, wherein the initial motion data is initial position data of a plurality of skeletal joint points of the virtual character (Section 4, Interaction Mesh: “In this section, we describe how we compute the interaction meshes for a given motion”; Section 3, Overview: “At every morph-step, the system adapts the motion by minimizing the Laplacian deformation of the interaction meshes at all the animation frames (or a fixed window of frames at a time, according to available computing resources) and the acceleration of the bodies in these frames (see Fig. 2 for an example of the adapted result of one frame). This spacetime optimization is performed to ensure temporal coherence of the motion. The optimization is subject to various constraints, namely, bone-length constraints, collision constraints and positional constraints. Collisions are then detected between the bounding volumes. If collisions are detected, the penetration depths are evaluated and a new set of collision constraints are defined to resolve the penetrations in the next morph-step”; Refer to Algorithm 1, which outlines the motion adaptation method, where the update step includes: “update(s) the vertex locations by solving Eq. (10)” and “Compute(s) segment orientations and update virtual vertices”. Vertex locations are derived from Eq. 10, which involves the Energy term
E
L
, which is defined in Eq. 1, supported by Eq. 2 and Eq. 3. The Energy term
E
L
essentially relies on an original Laplacian coordinate
δ
j
, where Laplacian coordinates are calculated in Eq. 3. The calculation of the original Laplacian coordinate is dependent on
p
j
, where
p
j
is essentially derived from the mesh; Section 4, Interaction Mesh: “We compute the volumetric interaction mesh for every animation frame. Using the positions of joints and vertices of objects as a point cloud, we apply Delaunay tetrahedralization [Si and Gaertner2005] (see Figure 2). Note that the spatial relationships which we want to preserve are those between body parts that are in close proximity and are not occluded by other parts. Since the Delaunay tetrahedralization favors connecting such parts with edges, the Laplacian coordinates of vertices which are defined by vertex neighbor-hood will lead to mutual influence between these body parts. By the nature of Laplacian mesh editing in preserving local details, the spatial relationships of our interest will be maintained” … “Let m be the number of vertices in the interaction mesh,
p
j
i
(1 ≤ j ≤ m) be the vertices at frame i”; Refer to Figure 1, which illustrates the retargeting of motions to different characters. Notes: The output of Algorithm 1 includes the determination of the initial motion data, as Algorithm 1 outputs vertices and orientations with respect to the constraints for the purpose of motion adaptation from an input skeleton and input motion sequence (target motion));
constructing a target function using the initial motion data and the original motion data, wherein the target function is configured for calculating a similarity between the initial motion data and the original motion data (Section 5.2, Iterative Morphing: “At every morph-step, the body sizes and the positional constraints are updated, and the motions of the characters are adapted by minimizing the sum of the deformation (Eq.1), acceleration (Eq.4)and constraint energy (Eq.9) of all frames subject to the hard constraints. The adapted motion is computed by solving (Eq. 10)” Eq. 10 defines a target function which uses the initial motion data and original motion data through the energy term
E
L
, as explained in the previous limitation. Minimizing the target function Eq. 10 minimizes the
E
L
term, which describes the similarity between the original and initial motion data via the calculation of the L2 norm of the original Laplacian coordinates of the vertices of the original mesh and the Laplacian coordinates of the updated vertices in the updated mesh, as seen in Eq. 1. Notes: L2 norm describes a similarity between two terms via a distance between the two terms. L2 norm is also referred to as Euclidean norm, where the calculation of the Euclidean norm regarding two terms ex.
|
|
x
-
y
|
|
2
) is the Euclidean distance between x and y. Note that Eq. 10 is a target function as it is a goal function, as is demonstrated by the function be minimized and solved to update vertex locations in Algorithm 1);
generating a collision constraint between the plurality of skeletal joint points of the virtual character and a profile joint point of the virtual character (Section 5.1, Constraints, Collision constraints: “The collision constraints prevent penetration between the bounding volumes of the skeleton. We perform collision detection by applying the ODE library [Smith 2005] to the current configuration of the bounding volumes. Specifically, when a penetration is detected, we compute the penetration depth, directions and the point pair penetrating each other the farthest and add the following constraints: (Eq. 7) … where Ji is the Jacobian of the positions of the colliding parts with respect to the joint positions, and di is the penetration depth multiplied to the normal vectors of the penetrated surface”. Notes: profile joint point, in its broadest reasonable interpretation, is the point related to a penetration point, which is described through the normal vectors of penetrated surface), and
generating a length constraint between adjacent skeletal joint points in the plurality of skeletal joint points of the virtual character with an unchanged distance between the adjacent skeletal joint points (Section 5.1, Constraints, Bone-length constraints: “We introduce the bone-length constraints in order to morph the bone lengths (distance between ad-jacent joints) from the original scales to the target scales. In each morph-step and each animation frame, the target length le for each bone e is computed by linearly blending the original and final lengths. Then, a constraint enforcing the target length is imposed as
(
|
p
e
1
'
-
p
e
2
'
|
-
l
e
)
2
” where
p
e
1
'
),
p
e
2
'
are the end vertices of the edge e. Linearizing all the bone-length constraints result in (Eq. 5) where Bi is the Jacobian matrix and l is a vector of constant terms”. Notes: vector l is considered an unchanged distance, since it is composed of constant terms;
acquiring target motion data of the virtual character by solving a minimum distance value of the target function under the length constraint and the collision constraint, wherein the target motion data is target position data of the plurality of skeletal joint points of the virtual character (Section 5.2, Iterative Morphing: “At every morph-step, the body sizes and the positional constraints are updated, and the motions of the characters are adapted by minimizing the sum of the deformation (Eq.1), acceleration (Eq.4)and constraint energy (Eq.9) of all frames subject to the hard constraints. The adapted motion is computed by solving (Eq. 10)” Eq. 10 defines a target function which uses the initial motion data and original motion data through the energy term
E
L
, as explained in the previous limitation. Minimizing the target function Eq. 10 minimizes the
E
L
term, which describes the similarity between the original and initial motion data via the calculation of the L2 norm of the original Laplacian coordinates of the vertices of the original mesh and the Laplacian coordinates of the updated vertices in the updated mesh, as seen in Eq. 1. Notes: L2 norm describes a similarity between two terms via a distance between the two terms. L2 norm is also referred to as Euclidean norm, where the calculation of the Euclidean norm regarding two terms ex.
|
|
x
-
y
|
|
2
) is the Euclidean distance between x and y. Note that Eq. 10 is a target function as it is a goal function, as is demonstrated by the function be minimized and solved to update vertex locations in Algorithm 1; Section 5.1, Constraints, Collision constraints: “The collision constraints prevent penetration between the bounding volumes of the skeleton. We perform collision detection by applying the ODE library [Smith 2005] to the current configuration of the bounding volumes. Specifically, when a penetration is detected, we compute the penetration depth, directions and the point pair penetrating each other the farthest and add the following constraints: (Eq. 7) … where Ji is the Jacobian of the positions of the colliding parts with respect to the joint positions, and di is the penetration depth multiplied to the normal vectors of the penetrated surface”. Notes: profile joint point, in its broadest reasonable interpretation, is the point related to a penetration point, which is described through the normal vectors of penetrated surface; Section 5.1, Constraints, Bone-length constraints: “We introduce the bone-length constraints in order to morph the bone lengths (distance between ad-jacent joints) from the original scales to the target scales. In each morph-step and each animation frame, the target length le for each bone e is computed by linearly blending the original and final lengths. Then, a constraint enforcing the target length is imposed as
(
|
p
e
1
'
-
p
e
2
'
|
-
l
e
)
2
” where
p
e
1
'
),
p
e
2
'
are the end vertices of the edge e. Linearizing all the bone-length constraints result in (Eq. 5) where Bi is the Jacobian matrix and l is a vector of constant terms”; Section 5.1, Constraints, Constraint energy: “We separate the constraints in Eq.(5)-7 into soft and hard constraints…(Eq. 8) … and define a constraint energy that represents the amount of violation of the soft constraints (Eq. 9) where W is a square diagnol matrix that assigns a different weight to each constraint”. The target function Equation 10 accounts for constraints through the Constraint Energy); and
driving the virtual character to perform the target motion by controlling the plurality of skeletal joint points of the virtual character to move to positions indicated by the target position data (Figure 1: “Our system can retarget motions of close interactions to characters of different morphologies. A judo interaction (red / orange pair) retargeted to characters of different sizes”, where the retarget motions are dictated by Algorithm 1, which performs motion adaptation as pictured in Figure 1, by updating and outputting virtual vertices and segment orientations. Refer to Figure 1 for an illustration of controlling virtual characters through motion dictated by target position data obtained via Algorithm 1).
Claim 1, being similar in scope to Claim 14, is rejected under the same rationale.
Claim 15, being similar in scope to Claim 14, is rejected under the same rationale.
Regarding Claim 16, the method of Claim 1 is rejected over Ho.
Ho teaches a computer program product comprising one or more instructions, wherein the one or more instructions, when loaded and executed by a processor, cause the processor to perform the method for controlling the motion of the virtual character (Abstract: “This paper presents a new method for editing and retargeting motions that involve close interactions between body parts of single or multiple articulated characters, such as dancing, wrestling, and sword fighting, or between characters and a restricted environment, such as getting into a car”; Section 6.3: “For all the retargeting examples shown in this paper, the computation required for each motion is around 1 minute for an animation of 100 frames, using one core of a Core i72.67GHz CPU”; Refer to Figure 1 for a visual of virtual characters and motion retargeting. Notes: motion retargeting is the process of controlling the motion of a target character by adapting the motion of a source character through position data. The method of controlling a motion of a virtual character as described in Ho is performed via a CPU device. The method of Ho is necessarily a computer program product, as the method is performed via a CPU device).
Regarding Claim 20, the device according to Claim 14 is rejected over Ho.
Ho teaches the device according to Claim 14, wherein the at least one computer program, when executed by the at least one processor, causes the at least one processor to perform:
Calculating an original vector of the plurality of skeletal joint points of the original model using the original motion data (Section 5, Spacetime Deformation: Let m be the number of vertices in the interaction mesh,
p
j
i
(1 ≤ j ≤ m) be the vertices at frame i,
V
i
be a vector of size 3m that includes all
p
j
i
such that
V
i
= (
p
1
i
, · · · ,
p
m
i
),and
p
j
i
′ and
V
i
′ be the updated vectors after the deformation. Notes: the original vector is original vector is
V
i
composed of vertices
p
j
i
(1 ≤ j ≤ m)), and
calculating an initial vector of the plurality of skeletal joint points of the virtual character using the initial motion data (Section 5, Spacetime Deformation: Let m be the number of vertices in the interaction mesh,
p
j
i
(1 ≤ j ≤ m) be the vertices at frame i,
V
i
be a vector of size 3m that includes all
p
j
i
such that
V
i
= (
p
1
i
, · · · ,
p
m
i
),and
p
j
i
′ and
V
i
′ be the updated vectors after the deformation; Refer to Algorithm 1, which includes the determination of the initial motion data, as Algorithm 1 outputs vertices and orientations with respect to the constraints for the purpose of motion adaptation from an input skeleton and input motion sequence (target motion), where
V
i
′ is the vector of all updated vertices
p
j
i
);
generating a motion semantic matrix of the target motion based on a skeleton structure of the original model and a predetermined motion semantic adjacency relationship (Eq. 1 defines the energy term, in which Laplacian coordinates of the vertices of the original mesh and the updated vertices are calculated. Eq. 3 defines the calculation of the Laplacian coordinates, which sums the one ring vertices
p
l
of
p
j
for l in
N
j
multiplied to their associated weights
w
l
j
, and subtracts it from the vertex
p
j
. Notes: The broadest reasonable interpretation of a matrix is a structure with data, where w, containing
w
l
j
for l in
N
j
, is a matrix. Note that a one ring neighborhood is inclusive of the center vertex);
acquiring a first product by calculating a product of the motion semantic matrix and the original vector (Refer to Eq. 1 which defines the original Laplacian coordinates and Eq. 3, which defines how to calculate the Laplacian coordinates, where calculating the Laplacian coordinates requires multiplying the vector vertices with corresponding weight matrices), and
acquiring a second product by calculating a product of the motion semantic matrix and the initial vector (Refer to Eq. 1 which defines the Laplacian coordinates of the updated vectors and Eq. 3, which defines how to calculate the Laplacian coordinates, where calculating the Laplacian coordinates requires multiplying the vector vertices with corresponding weight matrices); and
calculating a distance between the first product and the second product as the target function (Refer to Eq. 1 in its entirety, which defines an energy function that requires the difference between the first and second product).
Claim 5, being similar in scope to Claim 20, is rejected under the same rationale.
Regarding Claim 9, the method according to Claim 5 is rejected over Ho.
Ho teaches the method of Claim 5, wherein the distance between the first product and the second product is calculated as the target function by the following formula: 0.5 x ||L x tarPos3d - L x srcPos3d||2, wherein L represents the motion semantic matrix, tarPos3d represents the initial vector of the plurality of skeletal joint points of the virtual character, srcPos3d represents the original vector of the plurality of skeletal joint points of the original model, and |.| 2 represents a two-norm distance (See Eq. 1, which describes the distance between the first and second product as specified via the two-norm distance, where the motion semantic matrix is the w term in the Laplacian coordinate term, and the original vector vertices
p
j
and initial vector vertices
p
j
i
’, which are multiplied together as specified in Eq. 3 when calculating the Laplacian coordinates).
Regarding Claim 10, the method according to Claim 5 is rejected over Ho.
Ho teaches the method according to Claim 5, wherein generating the length constraint between the adjacent skeletal joint points in the plurality of skeletal joint points of the virtual character with the unchanged distance between the adjacent skeletal joint points comprises:
calculating a distance between two skeletal joint points of each bone of the virtual character as an original length of the each bone (Section 5.1, Constraints, Bone-length constraints: “We introduce the bone-length constraints in order to morph the bone lengths (distance between ad-jacent joints) from the original scales to the target scales. In each morph-step and each animation frame, the target length le for each bone e is computed by linearly blending the original and final lengths”);
calculating a distance of vectors of the two skeletal joint points of the each bone (Section 5.1, Constraints, Bone-length constraints: “Then, a constraint enforcing the target length is imposed as
(
p
e
1
'
-
p
e
2
'
-
l
e
)
2
) where
p
e
1
'
,
p
e
2
'
are the end vertices of the edge e”); and
constructing the length constraint as follows ||tarPos3d[i] - tarPos3d[j]||-resetLength = 0, wherein resetLength represents an original length of a bone between two adjacent skeletal joint points i and j of the virtual character, tarPos3d[i] and tarPos3d[j] respectively represent vectors of the skeletal joint point i and the skeletal joint point j, and both i and j are integers greater than or equal to 0 (Section 5.1, Constraints, Bone-length constraints: “Then, a constraint enforcing the target length is imposed as
(
p
e
1
'
-
p
e
2
'
-
l
e
)
2
) where
p
e
1
'
,
p
e
2
'
are the end vertices of the edge e”. Notes:
l
e
would be 0 in a case where the bone length should not change; as Ho explains, the target length depends on the target scale, and in a scenario where the target scale is the original scale, the target length would be 0, since no morphing would be required).
Regarding Claim 11, the method according to Claim 5 is rejected over Ho.
Ho teaches the method according to Claim 5, wherein the profile joint point comprises a predetermined collision point (Section 5.1, Constraints, Collision constraints: “Specifically, when a penetration is detected, we compute the penetration depth directions and the point pair penetrating each other the farthest”), the skeletal joint points comprise a joint point subjected to the collision constraint (Refer to Eq. 7, which describes the collision constraints; Section 5.1, Constraints, Collision constraints: “where Ji is the Jacobian of the positions of the colliding parts withrespect to the joint positions, and di is the penetration depth multi-plied to the normal vectors of the penetrated surface”), and the collision constraint between the plurality of skeletal joint points of the virtual character and the profile joint point of the virtual character is generated as follows:
(tarPos3d[i] – collPos).dot(colldepth)
≤
0, wherein collPos represents a vector of the predetermined collision point, tarPos3d[i] represents a vector of ajoint point i subjected to the collision constraint, tarPos3d[i]-collPos represents a vector from the joint point i subjected to the collision constraint to the predetermined collision point, dot product .dot(colldepth) represents a projection, in a direction perpendicular to an outer contour of the virtual character, of the vector from the joint point i subjected to the collision constraint to the predetermined collision point, and i is an integer greater than or equal to 0 (Refer to Eq. 7. Notes: the value of Eq. 7 only affects the Constraint energy in Eq. 9 when there is penetration, or if the value of Eq. 7 is over 0. Therefore, the Collision constraint described by Eq. 7 is seen as being less than or equal to 0).
Regarding Claim 12, the method according to Claim 1 is rejected over Ho.
Ho teaches the method according to Claim 1, wherein acquiring the target motion data of the virtual character by solving the minimum distance value of the target function under the length constraint and the collision constraint comprises:
acquiring the target motion data of the virtual character by solving the minimum distance value of the target function under the length constraint and the collision constraint (Eq. 10 describes the minimization of the target function, where Algorithm 1 solves for Eq. 10 to output and update segment orientations and virtual vertices (virtual character after performing target motion); see Figure 1)
using a sequential quadratic programming method or a lagrangian method (Section 5.2, Iterative Morphing: “At every morph-step, the body sizes and the positional constraints are updated, and the motions of the characters are adapted by minimizing the sum of the deformation (Eq.1), acceleration (Eq.4) and constraint energy (Eq.9) of all frames subject to the hard constraints. The adapted motion is computed by solving: (Eq. 10) where n is the number of frames, V′i is the set of new vertex positions at frame i, λi are the Lagrange multipliers and w∆ is a constant weight (we use 0.2). Note that EA is not defined for the first and last frames, hence we set them to zero. The spacetime optimization problem in Eq.(10) can be solved by differentiating it with respect to V′i and λi, and solving the following linear equation: (Eq. 11)”; Refer to Algorithm 1).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2, 3, 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Ho, in view of Cao (US 20210375045 A1).
Regarding Claim 17, the device according to Claim 14 is rejected over Ho.
Ho teaches the device according to Claim 14, wherein the at least one computer program, when executed by the at least one processor, causes the at least one processor to perform:
setting joint points, wherein the joint points comprise the profile joint point (Section 5.1, Collision constraints: “Specifically, when a penetration is detected, we compute the penetration depth, directions and the point pair penetrating each other the farthest”) and
the plurality of skeletal joint points of the virtual character (Section 4, Interaction Mesh: “We assume that the mesh characters are rigged with skeletons”; Section 6.2, Motion Adaptation in a Constrained Environment: “The interaction mesh is composed of the vertices of the environment and the skeleton joints and end effectors”; Figure 1: “Our system can retarget motions of close interactions to characters of different morphologies. A judo interaction (red / orange pair) retargeted to characters of different sizes”, where the retarget motions are dictated by Algorithm 1, which performs motion adaptation as pictured in Figure 1, by updating and outputting virtual vertices and segment orientations. Refer to Figure 1 for an illustration of controlling virtual characters through motion dictated by target position data obtained via Algorithm 1.).
Ho does not explicitly teach setting the plurality of skeletal joint points of the original model.
However, Cao teaches setting the plurality of skeletal joint points of a model (Paragraph [0005]: “The first purpose of the invention is to propose a system for digitalizing body shape of human body shape under clothing based on machine learning techniques and optimal algorithms on RGB image data. In which, machine learning techniques are used to: first, classify and segment clothing region; second, estimate skeleton joint locations and postures”. Notes: a model, in its broadest reasonable interpretation, is a representation of an entity, such as a person).
Ho and Cao are considered analogous in the art with respect to the estimation of skeleton joint points. A person having ordinary skill in the art would be motivated to utilize joint location estimation to reduce manual labor in setting joint points corresponding to a model before the use of the joint points in data processing.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the setting of the profile joint point and plurality of joint points for a virtual character of Ho with the setting of the joint points of the original character of Cao; Doing so would yield the predictable result of comprehensively setting all joint points utilized in the method of Ho.
Claim 2, being similar in scope to Claim 17, is rejected under the same rationale.
Regarding Claim 18, the device according to Claim 14 is rejected over Ho.
Ho teaches the device according to Claim 14, wherein the at least one computer program, when executed by the at least one processor, causes the at least one processor to perform:
Acquiring the position data of the plurality of skeletal joint points of the original model as the original motion data (Section 4, Interaction Mesh: “In this section, we describe how we compute the interaction meshes for a given motion. We assume that the mesh characters are rigged with skeletons”; Section 4, Interaction Mesh: “The postures of the characters are represented by the positions of the joints”; Section 3, Overview: “We give an overview of our method in this section. First, the data of the original characters and the motion is loaded into our system”; Figure 1: “Our system can retarget motions of close interactions to characters of different morphologies. A judo interaction (red / orange pair) retargeted to characters of different sizes”.
Ho does not teach collecting an image of the original model, and acquiring, by performing joint point identification on the image, the position data of the plurality of skeletal joint points of the original model.
However, Cao teaches collecting an image of the original model, and acquiring, by performing joint point identification on the image, the position data of the plurality of skeletal joint points of the original model (Paragraph [0005]: “The first purpose of the invention is to propose a system for digitalizing body shape of human body shape under clothing based on machine learning techniques and optimal algorithms on RGB image data. In which, machine learning techniques are used to: first, classify and segment clothing region; second, estimate skeleton joint locations and postures”. Notes: a model, in its broadest reasonable interpretation, is a representation of an entity, such as a person).
Ho and Cao are considered analogous in the art with respect to the estimation of skeleton joint points. A person having ordinary skill in the art would be motivated to utilize joint location estimation to reduce manual labor in setting joint points corresponding to a model before the use of the joint points in data processing.
Therefore, it would have been obvious to a person having ordinary skill in the art to combine the acquiring of position data of the plurality of skeletal joint points of the original model as original motion data of Ho with the use of joint point identification on an image of the original model to obtain position data of the skeletal joint points of Cao; Doing so would yield the predictable result of obtaining skeletal joint points as motion data via image analysis, improving the efficiency of setting skeletal joint points and obtaining corresponding motion data.
Claim 3, being similar in scope to Claim 18, is rejected under the same rationale.
Claims 4 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Ho, in view of Daz3D ().
Regarding Claim 19, the device according to Claim 14 is rejected over Ho.
Ho teaches the device according to Claim 14, wherein the at least one computer program, when executed by the at least one processor, causes the at least one processor to perform:
calculating rotation data of a bone between every two adjacent skeletal joint points based on position data of the every two adjacent skeletal joint points in the plurality of skeletal joint points of the original model in the original motion data (Section 4, Interaction Mesh: “The orientation of some body segments cannot be computed only from the positions of joints bounding that segment. For example, the joint positions of the elbow and the wrist are insufficient to confirm the rotation around the forearm. In order to compute such orientations, we sample one extra virtual vertex on the surface of each bounding volume”);
Ho does not explicitly teach acquiring the initial motion data of the virtual character by transplanting rotation data of each bone in the original model to a bone, corresponding to the each bone, of the virtual character as rotation data of the bone of the virtual character, although it is heavily implicit through the nature of motion adaptation between characters.
However, Daz3D teaches acquiring the initial motion data of the virtual character by transplanting rotation data of each bone in the original model to a bone, corresponding to the each bone, of the virtual character as rotation data of the bone of the virtual character (Post by freenomon: “basically I want to retarget rotation data of an imported fbx node to finger joints of my character. I can do this easy in maya via sdk (set driven keys). But I am not familiar with daz scriting commads. I can record rotation data of a mocap prop tracker and bring that in to daz as fbx. Then I want to assign that rotation data to drive the finger joints (open and close finger poses). Will need to retarget the rotation data to chosen finger joints with different multiplied values (strengths). I can already do it in maya. hope it makes sense”; Answer by Richard Haseltine: “Yes, I would suggest trying BVH. If that doesn't work - you essentially have a rigged figure, or at least a skeleton, and want to transfer a hand pose from that to a Genesis figure so that it looks the same? Script could certainly get the rotations from one figure and apply the transformed rptation to another”; Answer by Richard Haseltine: “Ah, so not actually retargeting (which takes account, as I understand it, of different bone legths and zero positions) but simply copying across? Yes, you can do that with a script. I would get the selected node list (from DzScene), sort it into bones and (hopefully) just one prop. Then you can take the prop and use node.getXRotControl() etc. to get the rotation proeprties, then get the value from each, getXRotControl() (etc.) for each bone, setValue( rot) on each”).
Ho and Daz3D are considered analogous in the art with respect to the use of motion data between different models. Acquiring motion data by transplanting rotation data of each bone in an original model to a corresponding bone in another model is well known in the art, as is evident in Daz3D, and is commonly done for having differently sized or proportioned models perform a target motion.
Therefore, it would have been obvious to a person having ordinary skill in the art to combine the calculation of rotation data a bone between each adjacent skeletal joint point of Ho with the transplanting of the rotation data of each bone in the original model to a corresponding bone in the virtual character of Daz3D; Doing so would yield the predictable result of acquiring initial motion data of the virtual character.
Claim 4, being similar in scope to Claim 19, is rejected under the same rationale.
Allowable Subject Matter
Claims 6-8 and 21 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 6-8 are deemed to be allowable subject matter dependent on rejected dependent Claim 5, and ultimately dependent on rejected independent Claim 1.
Claim 21 is deemed to be allowable subject matter dependent on rejected dependent Claim 20, and ultimately dependent on rejected independent Claim 14.
Regarding Claim 6, While generating a motion semantic matrix of the target motion based on the skeleton structure of the original model and the predetermined motion semantic adjacency relationship is known in the art as taught by Ho (Eq. 1 defines the energy term, in which Laplacian coordinates of the vertices of the original mesh and the updated vertices are calculated. Eq. 3 defines the calculation of the Laplacian coordinates, which sums the one ring vertices
p
l
of
p
j
for l in
N
j
multiplied to their associated weights
w
l
j
, and subtracts it from the vertex
p
j
. Notes: The broadest reasonable interpretation of a matrix is a structure with data, where w, containing
w
l
j
for l in
N
j
, is a matrix. Note that a one ring neighborhood is inclusive of the center vertex), there is no prior art that explicitly teaches acquiring a joint point adjacency matrix as defined in Claim 6.
Regarding Claim 7, While Ho broadly teaches acquiring a weighted value via calculating a product of the motion semantic matrix and original and initial vectors (Eq. 1 defines the energy term, in which Laplacian coordinates of the vertices of the original mesh and the updated vertices are calculated. Eq. 3 defines the calculation of the Laplacian coordinates, which sums the one ring vertices
p
l
of
p
j
for l in
N
j
multiplied to their associated weights
w
l
j
, and subtracts it from the vertex
p
j
; Refer to Eq. 1 which defines the original Laplacian coordinates and Eq. 3, which defines how to calculate the Laplacian coordinates, where calculating the Laplacian coordinates requires multiplying the vector vertices with corresponding weight matrices; Refer to Eq. 1 which defines the Laplacian coordinates of the updated vectors and Eq. 3, which defines how to calculate the Laplacian coordinates, where calculating the Laplacian coordinates requires multiplying the vector vertices with corresponding weight matrices. Notes: The broadest reasonable interpretation of a matrix is a structure with data, where w, containing
w
l
j
for l in
N
j
, is a matrix. Note that a one ring neighborhood is inclusive of the center vertex), Ho does not teach doing so with the joint point adjacency matrix as defined in Claim 6.
Regarding Claim 8, While Ho teaches calculating weight where the weigh involves calculating a reciprocal of the distance between the adjacent joint points in the form of the motion semantic matrix, Ho does not teach doing so with the joint point adjacency matrix as defined in Claim 6.
Claims 6-8, upon being rewritten into independent Claim 1 along with base Claim 5, which Claims 6-8 depend upon, would be considered allowable.
Claim 21, upon being rewritten into independent Claim 14 along with base Claim 20, which Claim 21 depends upon, would be considered allowable.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAYMOND CHUN LAM LI whose telephone number is (571)272-5124. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAYMOND CHUN LAM LI/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights