DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
2. Applicant's arguments filed on 09/08/2026, with respect to the 103 rejection have been fully considered but are moot in view of the new grounds of rejection.
In light of the current Office Action, the Examiner respectfully submits that independent claims 1, 9 and 16 are rejected in view of newly discovered reference(s) to Chen et al. (US-2014/0035901-A1).
Examiner notes that independent claims 1, 9 and 16 have been amended to include new limitation. Examiner finds these limitations to be unpatentable as can be found in below detail action.
3. On page 12 of Applicant's Remarks, the Applicant argues that the corresponding dependent claims are not taught by the prior art, insomuch as they depend from claims that are not taught by the prior art. Examiner respectfully disagrees with these arguments, for the reasons discussed above.
Information Disclosure Statement
4. The information disclosure statements (IDS) submitted on the following dates are in compliance with the provisions of 37 CFR 1.97 and are being considered by the Examiner: 09/08/2026.
Claim Rejections - 35 USC § 103
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 1-2, 9 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (machine translation of CN-104021584-A with citation below, hereinafter “Zhang”) in view of Chen (“Chen”) [US-2014/0035901-A1], further in view of Stern (“Stern”) [US-4,600,919-A]
Regarding claim 1, Zhang discloses a method for an animation of a skeleton, the method (Zhang- ¶0002, at least discloses an method for implementation skeletal skinning animation) comprising:
receiving keyframes representing movement skeletal motion stored in the character model file, perform interpolation calculations between two adjacent keyframes to determine the new position and orientation of each bone at a certain moment; ¶0013, at least discloses The keyframe information includes a sequence of keyframes that constitute the bone's motion);
receiving a mesh representing a skin of the character (Zhang- ¶0009, at least discloses Create a skeletal model and a mesh skinning model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of bones describes the structure of the character. The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of the bones; ¶0032, at least discloses The mesh skinning model, which is the character's skin, is a deformable mesh that changes under the influence of the skeleton);
from the keyframes, generating skeletal animation data representing movement of the skeleton (Zhang- ¶0009-0013, at least disclose A. Create a skeletal model and a mesh skin model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of the skeletons describes the structure of the character. The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] C. Based on the keyframe sequence of skeletal motion stored in the character model file, perform interpolation calculation between two adjacent keyframes to determine the new position and orientation of each bone at a certain moment […] The keyframe information includes a sequence of keyframes that constitute the bone’s motion; Fig. 1 and ¶0032, at least disclose Step S101: Create the skeletal model and a mesh skinning model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of bones describes the structure of the character. The mesh skinning model, which is the character’s skin, is a deformable mesh that changes under the influence of the skeleton; ¶0156, at least discloses the calculation can be used to obtain the interpolation matrix of the skeleton during movement, rotation, and scaling);
binding the mesh to the skeleton to obtain mesh binding data (Zhang- ¶0009-0012, at least discloses A. Create a skeletal model and a mesh skin model that constitute the character model […] The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] B. Determine the bones that affect the mesh vertices on the mesh skinning model, and determine their influence weights based on the geometric and physical relationships between the bones and the vertexes […] D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone indexes and corresponding influence weights stored in each vertex of the mesh skinning model and influence the vertex, and realize the skeletal skinning animation [binding a mesh to the skeleton to obtain mesh binding data]; Fig. 1 and ¶0037, at least disclose Step S104: Calculate the new positions and orientations of each vertex in the world coordinate system according to the bone index and corresponding influence weight stored in each vertex of the mesh skin model [binding a mesh], and realize the skeletal skinning animation); and
outputting the skeletal animation data and the mesh binding data (Zhang- ¶0016, at least discloses Step A includes creating the skeletal model and the mesh skinning model that constitute the character model using either Poser or 3ds MAX software; ¶0038, at least discloses the vertex of the model is defined in the model coordinate system, and for the game engine, the vertex position in the model coordinate system is converted into the world coordinate system for rendering […] by attaching each vertex to the corresponding bone, it can be rendered into an animation), the skeletal animation data and the mesh binding data providing a basis for animation by deforming the mesh in accordance with the skeletal animation data (Zhang- ¶0009-0012, at least disclose A. Create a skeletal model and a mesh skin model that constitute the character model […] The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone indexes and corresponding influence weights stored in each vertex of the mesh skinning model and influence the vertex, and realize the skeletal skinning animation; Fig. 1 and ¶0032, at least disclose Step S101: Create the skeletal model and a mesh skinning model that constitute the character model […] The mesh skinning model, which is the character’s skin, is a deformable mesh that changes under the influence of the skeleton [deforming the mesh in accordance with the skeletal animation data]).
Zhang does not explicitly disclose precomputing an animation of a character having a skeleton; receiving keyframes representing movement of the character; precomputing an animation of the character from the keyframes and the mesh, the precomputing comprising: the skeletal animation data comprising a matrix representing a transformation of the skeleton, wherein a particular dimension of the matrix represents frames of the precomputed animation of the character; providing a basis for subsequent animation of the character at runtime.
However, Chen discloses
precomputing an animation of a character having a skeleton (Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime […] In contrast, although the deformation graph may be generated (in block 112) at runtime, alternatively this may be considered a pre-processing step and may be performed in advance; ¶0057-0058, at least disclose As the user has freedom to adopt different body positions, the user can influence or directly control which parts of their body are mapped to particular parts of the object being animated. In order to assist the user in this, a graphical user interface (GUI) may be provided which provides visual feedback to the user, for example by displaying both the tracked skeleton and the object (or more particularly, the mesh of the object) […] the GUI is arranged such that the user appears to walk up to the object within the GUI. In some examples, lines or links 602 may be shown in the GUI 600 between a joint 604 on the skeleton 605 and the k-nearest nodes in the graph representing the object mesh 606 as shown in FIG. 6 […] As shown in FIG. 6, the GUI comprises multiple lines (e.g. k lines) radiating from each joint in the skeleton to points on the object mesh which correspond to the nodes in the deformation graph);
receiving keyframes representing movement of the character (Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0082, at least discloses the methods may also be used for quickly previewing an animation before doing more sophisticated animation using a rigging or keyframe system);
precomputing an animation of the character from the keyframes and the mesh (Chen- ¶0001, at least discloses First a model of an object will be generated in the form of a 3D mesh […] at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0004, at least discloses Methods of animating objects using the human body are described. In an embodiment, a deformation graph is generated from a mesh which describes the object […] The transformations are then applied to the mesh to generate an animation of the object which corresponds to the captured motion; ¶0082, at least discloses the methods may also be used for quickly previewing an animation before doing more sophisticated animation using a rigging or keyframe system);
providing a basis for subsequent animation of the character at runtime (Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang to incorporate the teachings of Chen, and apply driving the bones of the character and subsequent transformation of both the graph and the mesh are performed in real-time at runtime into the Zhang’s teachings for receiving keyframes representing movement of the character; receiving a mesh representing a skin of the character; precomputing an animation of the character from the keyframes and the mesh, the precomputing comprising: providing a basis for subsequent animation of the character at runtime by deforming the mesh in accordance with the skeletal animation data.
Doing so would provide the optimal transformations on the deformation graph that are smooth, feature-preserving and satisfy the user's motion constraints.
The prior art does not explicitly disclose, but Stern discloses
the skeletal animation data comprising a matrix representing a transformation of the skeleton (Stern- col 7, lines 30-37, at least discloses matrix transformations [matrix representing a transformation] are employed to transform the limb vectors in a given local coordinate system into equivalent vectors in the joint coordinate system of next highest priority. The procedure is repeated for successively higher priority joint coordinate systems until the limb vectors are expressed in terms of the world coordinate system; col 7, lines 51-65, at least discloses The matrix transformation which translates a point (x,y,z) to a new point (x',y',z') is known to be: ##EQU4## where mx, my and mz are the components of the translation or move in the x, y and z directions respectively, and the 1 in each fourth column is the homogeneous coordinate; Fig. 5 and col 9, lines 36-42, at least disclose If the figure skeletons are to be displayed, a basic orthogonal coordinate system representation (i.e., three mutually orthogonal lines), can be multiplied by the concatenated matrix transformation, so as to obtain properly oriented and scaled local coordinate axes representation at each joint position, as shown in the small sketch near block 516), wherein a particular dimension of the matrix represents frames of the animation (Stern- col 6, lines 26 to col 7, line 24, at least disclose FIGS. 4A and 4B are useful in understanding the manner in which the joints and limbs are represented and manipulated. In FIG. 4A, there is shown the coordinate axes for an exemplary joint, Ji, and a limb, Li, (shown in dashed line) of vectors associated with the joint. The matrix values for the joint Ji is as follows: ##EQU2## As seen in FIG. 4A, the origin of the local coordinate system for Ji is at the point (10, 10, 10) in the main or "world" coordinate system (which, in this example, is the coordinate system of next higher priority in the hierarchy). Also, the x, y, and z axes are aligned with the axes of the world coordinate system, so the relative rotation angles in the second row of the matrix are all 0°. Finally, it is assumed that, for this example, the scale factors for the frame are all unity […] Reference can now be made to FIG. 4B which represents the same joint Ji and limb Li, but with the position, orientation, and size and shape of the limb being different by virtue of modifications in the matrix values of the examplary joint. In particular, the matrix for the joint of FIG. 4B is as follows: ##EQU3## In this case, the origin of the local coordinate axes (i.e., the joint position) is seen to be at the coordinates (10, 15, 10), which correspond to the values in the first row of the matrix. Also, it is seen that the y, z axes are rotated by 45° around the x axis as compared to the orientations of the y, z axes in the parent "world" coordinate system. Accordingly, the second row of values has a 45° rotational angle indicated in the x column, and 0° rotational angles indicated for rotation around the y and z axes. Regarding scaling, the limb of FIG. 4A is seen to be doubled in length or "stretched" along the z axis (only). Thus a scale factor of 2 is indicated in the z column of the third row of the matrix [particular dimension of the matrix]; col 10, lines 43-66, at least discloses Block 723 is then entered, this block representing the storage of values from the interpolation curve in the transformation matrices of the frames being in-betweened. In particular, the selected one of the nine values in the parametric matrix described above for each joint is stored for each of the in-between frames, based on the value of such parameter component taken from the cubic interpolation curve […] It can be noted, as represented by the block 729, that the limbs associated with each joint are obtained in conjunction with the joint in each frame of the sequence, the motion of the limb being completely defined as described above, by the matrix transformation values of the associated joint; col 11, lines 9-14, at least discloses when a generated sequence of animation is displayed, the operator can modify the motion of a limb during a continuous display of the sequence […] at each frame of a sequence being displayed, the x, y and z components of a selected parameter (move, rotation or scale factor) of a selected joint are incremented by an amount which is proportional to the operator selection of x, y and z values using the joystick);
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen to incorporate the teachings of Stern, and apply the matrix transformation into the Zhang/Chen’s teachings for generating skeletal animation data representing movement of the skeleton, the skeletal animation data comprising a matrix representing a transformation of the skeleton, wherein a particular dimension of the matrix represents frames of the precomputed animation of the character.
Doing so the animator has virtually complete artistic freedom and control over the resulting film; i.e. anything that is drawn can be made to move in a desired fashion so natural-looking motion can generally be achieved.
Regarding claim 2, Zhang in view of Chen and Stern, discloses the method of claim 1, and further discloses wherein generating the skeletal animation data (see Claim 1 rejection for detailed analysis) represents the transformation from a first keyframe to a second keyframe of the animation for each of multiple joints of the skeleton (Zhang- ¶0013, at least disclose the keyframe information includes a sequence of keyframes constituting the motion of the bones, each keyframe holding a transformation matrix for each bone at that moment with respect to the parent bone coordinate system, and the position and orientation information of the bones; ¶0035, at least discloses according to the key frame sequence of the bone motion stored in the character model file, interpolation calculation is carried out between two adjacent key frames, and the new position and the new orientation of each bone at a certain moment are determined).
Regarding claim 9, Zhang discloses a method (Zhang- ¶0002, at least discloses an method for implementation skeletal skinning animation) comprising:
receiving animation of a character having a skeleton and a skin (Zhang- ¶0032, at least discloses Create the skeletal model and mesh skinning model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of bones describes the structure of the character. The mesh skinning model, which is the character's skin, is a deformable mesh that changes under the influence of the skeleton) animation including:
skeletal animation data representing movement of joints connecting bones of the skeleton (Zhang- ¶0009-0013, at least disclose A. Create a skeletal model and a mesh skin model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of the skeletons describes the structure of the character. The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] C. Based on the keyframe sequence of skeletal motion stored in the character model file, perform interpolation calculation between two adjacent keyframes to determine the new position and orientation of each bone at a certain moment […] The keyframe information includes a sequence of keyframes that constitute the bone’s motion; Fig. 1 and ¶0032, at least disclose Step S101: Create the skeletal model and a mesh skinning model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of bones describes the structure of the character. The mesh skinning model, which is the character’s skin, is a deformable mesh that changes under the influence of the skeleton; ¶0156, at least discloses the calculation can be used to obtain the interpolation matrix of the skeleton during movement, rotation, and scaling)skeletal animation datafrom keyframes representing movement (Zhang- ¶0011, at least discloses Based on the keyframe sequence of skeletal motion stored in the character model file, perform interpolation calculations between two adjacent keyframes to determine the new position and orientation of each bone at a certain moment; ¶0013, at least discloses The keyframe information includes a sequence of keyframes that constitute the bone's motion)
;
mesh binding data representing a binding of a mesh to the skeleton (Zhang- ¶0009-0012, at least discloses A. Create a skeletal model and a mesh skin model that constitute the character model […] The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] B. Determine the bones that affect the mesh vertices on the mesh skinning model, and determine their influence weights based on the geometric and physical relationships between the bones and the vertexes […] D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone indexes and corresponding influence weights stored in each vertex of the mesh skinning model and influence the vertex, and realize the skeletal skinning animation [binding a mesh to the skeleton to obtain mesh binding data]; Fig. 1 and ¶0037, at least disclose Step S104: Calculate the new positions and orientations of each vertex in the world coordinate system according to the bone index and corresponding influence weight stored in each vertex of the mesh skin model [binding a mesh], and realize the skeletal skinning animation), the mesh representing the skin of the character (Zhang- ¶0009, at least discloses Create a skeletal model and a mesh skinning model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of bones describes the structure of the character. The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of the bones; ¶0032, at least discloses The mesh skinning model, which is the character's skin, is a deformable mesh that changes under the influence of the skeleton); and
animating the character by performing deformation of the mesh based on the skeletal animation data and the mesh binding data (Zhang- ¶0009-0012, at least disclose A. Create a skeletal model and a mesh skin model that constitute the character model […] The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone indexes and corresponding influence weights stored in each vertex of the mesh skinning model and influence the vertex, and realize the skeletal skinning animation [animation of the mesh]; ¶0016, at least discloses Step A includes creating the skeletal model and the mesh skinning model that constitute the character model using either Poser or 3ds MAX software; Fig. 1 and ¶0032, at least disclose Step S101: Create the skeletal model and a mesh skinning model that constitute the character model […] The mesh skinning model, which is the character’s skin, is a deformable mesh that changes under the influence of the skeleton [deformation of the mesh]; Fig. 1 and ¶0037, at least disclose Step S104: Calculate the new positions and orientations of each vertex in the world coordinate system according to the bone index and corresponding influence weight stored in each vertex of the mesh skin model, and realize the skeletal skinning animation).
Zhang does not explicitly disclose receiving a precomputed animation of a character having a skeleton and a skin, the precomputed animation including: precomputed skeletal animation data; the precomputed skeletal animation data having been precomputed from keyframes representing movement of the character and comprising a matrix representing a transformation of the skeleton, wherein a particular dimension of the matrix represents frames of the precomputed animation of the character, and precomputed mesh binding data; performing runtime deformation of the mesh based on the precomputed skeletal animation data and the precomputed mesh binding data.
However, Chen discloses
receiving a precomputed animation of a character having a skeleton and a skin, the precomputed animation including: precomputed skeletal animation data (Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime […] In contrast, although the deformation graph may be generated (in block 112) at runtime, alternatively this may be considered a pre-processing step and may be performed in advance [precomputed]; ¶0057-0058, at least disclose As the user has freedom to adopt different body positions, the user can influence or directly control which parts of their body are mapped to particular parts of the object being animated. In order to assist the user in this, a graphical user interface (GUI) may be provided which provides visual feedback to the user, for example by displaying both the tracked skeleton and the object (or more particularly, the mesh of the object) […] the GUI is arranged such that the user appears to walk up to the object within the GUI. In some examples, lines or links 602 may be shown in the GUI 600 between a joint 604 on the skeleton 605 and the k-nearest nodes in the graph representing the object mesh 606 as shown in FIG. 6 […] As shown in FIG. 6, the GUI comprises multiple lines (e.g. k lines) radiating from each joint in the skeleton to points on the object mesh which correspond to the nodes in the deformation graph; ¶0067, at least discloses The deformation defined by multiple nodes is combined linearly as in linear blend skinning (LBS));
the precomputed skeletal animation data having been precomputed from keyframes representing movement of the character (Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime […] In contrast, although the deformation graph may be generated (in block 112) at runtime, alternatively this may be considered a pre-processing step and may be performed in advance [precomputed]; ¶0082, at least discloses the methods may also be used for quickly previewing an animation before doing more sophisticated animation using a rigging or keyframe system);
animating the character by performing runtime deformation of the mesh (Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime. The attachment of the skeleton to the graph is performed once, while the transformations (in blocks 118-120) are performed per frame (e.g. at a frame rate of 30 Hz). In contrast, although the deformation graph may be generated (in block 112) at runtime, alternatively this may be considered a pre-processing step and may be performed in advance).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang to incorporate the teachings of Chen, and apply receiving a precomputed animation of a character and the pre-processing step performed in advance into animation of a character having a skeleton and a skin, skeletal animation data and mesh binding data, as taught by Zhang, for receiving a precomputed animation of a character having a skeleton and a skin, the precomputed animation including: precomputed skeletal animation data representing movement of joints connecting bones of the skeleton, the precomputed skeletal animation data having been precomputed from keyframes representing movement of the character, and precomputed mesh binding data representing a binding of a mesh to the skeleton, the mesh representing the skin of the character; and animating the character by performing runtime deformation of the mesh based on the precomputed skeletal animation data and the precomputed mesh binding data.
Doing so would provide the optimal transformations on the deformation graph that are smooth, feature-preserving and satisfy the user's motion constraints.
The prior art does not explicitly disclose, but Stern discloses
the skeletal animation data comprising a matrix representing a transformation of the skeleton (Stern- col 7, lines 30-37, at least discloses matrix transformations [matrix representing a transformation] are employed to transform the limb vectors in a given local coordinate system into equivalent vectors in the joint coordinate system of next highest priority. The procedure is repeated for successively higher priority joint coordinate systems until the limb vectors are expressed in terms of the world coordinate system; col 7, lines 51-65, at least discloses The matrix transformation which translates a point (x,y,z) to a new point (x',y',z') is known to be: ##EQU4## where mx, my and mz are the components of the translation or move in the x, y and z directions respectively, and the 1 in each fourth column is the homogeneous coordinate; Fig. 5 and col 9, lines 36-42, at least disclose If the figure skeletons are to be displayed, a basic orthogonal coordinate system representation (i.e., three mutually orthogonal lines), can be multiplied by the concatenated matrix transformation, so as to obtain properly oriented and scaled local coordinate axes representation at each joint position, as shown in the small sketch near block 516), wherein a particular dimension of the matrix represents frames of the animation of the skeleton (Stern- col 6, lines 26 to col 7, line 24, at least disclose FIGS. 4A and 4B are useful in understanding the manner in which the joints and limbs are represented and manipulated. In FIG. 4A, there is shown the coordinate axes for an exemplary joint, Ji, and a limb, Li, (shown in dashed line) of vectors associated with the joint. The matrix values for the joint Ji is as follows: ##EQU2## As seen in FIG. 4A, the origin of the local coordinate system for Ji is at the point (10, 10, 10) in the main or "world" coordinate system (which, in this example, is the coordinate system of next higher priority in the hierarchy). Also, the x, y, and z axes are aligned with the axes of the world coordinate system, so the relative rotation angles in the second row of the matrix are all 0°. Finally, it is assumed that, for this example, the scale factors for the frame are all unity […] Reference can now be made to FIG. 4B which represents the same joint Ji and limb Li, but with the position, orientation, and size and shape of the limb being different by virtue of modifications in the matrix values of the examplary joint. In particular, the matrix for the joint of FIG. 4B is as follows: ##EQU3## In this case, the origin of the local coordinate axes (i.e., the joint position) is seen to be at the coordinates (10, 15, 10), which correspond to the values in the first row of the matrix. Also, it is seen that the y, z axes are rotated by 45° around the x axis as compared to the orientations of the y, z axes in the parent "world" coordinate system. Accordingly, the second row of values has a 45° rotational angle indicated in the x column, and 0° rotational angles indicated for rotation around the y and z axes. Regarding scaling, the limb of FIG. 4A is seen to be doubled in length or "stretched" along the z axis (only). Thus a scale factor of 2 is indicated in the z column of the third row of the matrix [particular dimension of the matrix]; col 10, lines 43-66, at least discloses Block 723 is then entered, this block representing the storage of values from the interpolation curve in the transformation matrices of the frames being in-betweened. In particular, the selected one of the nine values in the parametric matrix described above for each joint is stored for each of the in-between frames, based on the value of such parameter component taken from the cubic interpolation curve […] It can be noted, as represented by the block 729, that the limbs associated with each joint are obtained in conjunction with the joint in each frame of the sequence, the motion of the limb being completely defined as described above, by the matrix transformation values of the associated joint; col 11, lines 9-14, at least discloses when a generated sequence of animation is displayed, the operator can modify the motion of a limb during a continuous display of the sequence […] at each frame of a sequence being displayed, the x, y and z components of a selected parameter (move, rotation or scale factor) of a selected joint are incremented by an amount which is proportional to the operator selection of x, y and z values using the joystick).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen to incorporate the teachings of Stern, and apply the matrix transformation into the Zhang/Chen’s teachings in order the skeletal animation data comprising a matrix representing a transformation of the skeleton, wherein a particular dimension of the matrix represents frames of the precomputed animation of the character.
Doing so the animator has virtually complete artistic freedom and control over the resulting film; i.e. anything that is drawn can be made to move in a desired fashion so natural-looking motion can generally be achieved.
Regarding claim 15, Zhang in view of Chen and Stern, discloses the method of claim 9, and further discloses the method performed on a graphics processing unit (Zhang- ¶0164, at least discloses the new matrix is passed to the GPU).
7. Claims 16-17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Chen, further in view of Stern, still further in view of Li et al., (“Li”) [US-2019/0206145-A1]
Regarding claim 16, Zhang discloses a system (Zhang- ¶0033, at least discloses 3ds MAX is a three-dimensional animation rendering and production software developed by Discreet corporation (later incorporated by Autodesk corporation) based on PC systems) comprising:
a graphics processing unit (Zhang- ¶0164, at least discloses a GPU);
wherein the processing unit is configured to:
receive an animation of a character having a skeleton and a skin (Zhang- ¶0032, at least discloses Create the skeletal model and mesh skinning model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of bones describes the structure of the character. The mesh skinning model, which is the character's skin, is a deformable mesh that changes under the influence of the skeleton), animation including:
skeletal animation data representing movement of joints connecting bones of the skeleton (see Claim 9 rejection for detailed analysis);
mesh binding data representing a binding of a mesh to the skeleton (see Claim 9 rejection for detailed analysis), the mesh representing the skin of the character (see Claim 9 rejection for detailed analysis); and
transfer the skeletal animation data and the mesh binding data to the graphics processing unit (Zhang- ¶0163-0164, at least discloses calculating the world matrix of each bone at any given time, we can calculate the world matrix of the parent skeleton, and then calculate the world matrix of the child bone. The MESH file stores this vertex information, i.e., MESH skin information, including vertex position, normal vectors, texture material, and the bones that affect the vertex and their impact weights […] Once the new matrix is obtained, the new matrix for each bone can be calculated sequentially. The new matrix is passed to GPU as a bone array, and the new position and orientation of the vertex in the world coordinate are calculated based on the weights of the bones that affect the vertex), and wherein the graphics processing unit is configured to:
receive the skeletal animation data and the mesh binding data (Zhang- ¶0009, at least discloses A. Create a skeletal model and a mesh skin model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of the skeletons describes the structure of the character. The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones; Fig. 1 and ¶0032, at least disclose Step S101: Create the skeletal model and a mesh skinning model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of bones describes the structure of the character. The mesh skinning model, which is the character’s skin, is a deformable mesh that changes under the influence of the skeleton) ; and
animate the character by performing deformation of the mesh based on the skeletal animation data and the mesh binding data (see Claim 9 rejection for detailed analysis).
Zhang does not explicitly disclose a central processing unit; receive a precomputed animation of a character having a skeleton and a skin, the precomputed animation including: precomputed skeletal animation data, the precomputed skeletal animation data having been precomputed from keyframes representing movement of the character and comprising a matrix representing a transformation of the skeleton, wherein a particular dimension of the matrix represents frames of the precomputed animation of the character; precomputed mesh binding data; performing runtime deformation of the mesh based on the precomputed skeletal animation data and the precomputed mesh binding data.
However, Chen discloses
a central processing unit (Chen- Fig. 3 and ¶0029, at least disclose Computing-based device 300 comprises one or more processors 302 which may be microprocessors, controllers or any other suitable type of processors for processing computer executable instructions to control the operation of the device in order to perform the animation methods described herein);
receive a precomputed animation of a character having a skeleton and a skin, the precomputed animation including: precomputed skeletal animation data (Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime […] In contrast, although the deformation graph may be generated (in block 112) at runtime, alternatively this may be considered a pre-processing step and may be performed in advance [precomputed]; ¶0057-0058, at least disclose As the user has freedom to adopt different body positions, the user can influence or directly control which parts of their body are mapped to particular parts of the object being animated. In order to assist the user in this, a graphical user interface (GUI) may be provided which provides visual feedback to the user, for example by displaying both the tracked skeleton and the object (or more particularly, the mesh of the object) […] the GUI is arranged such that the user appears to walk up to the object within the GUI. In some examples, lines or links 602 may be shown in the GUI 600 between a joint 604 on the skeleton 605 and the k-nearest nodes in the graph representing the object mesh 606 as shown in FIG. 6 […] As shown in FIG. 6, the GUI comprises multiple lines (e.g. k lines) radiating from each joint in the skeleton to points on the object mesh which correspond to the nodes in the deformation graph; ¶0067, at least discloses The deformation defined by multiple nodes is combined linearly as in linear blend skinning (LBS)), the precomputed skeletal animation data having been precomputed from keyframes representing movement of the character (Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime […] In contrast, although the deformation graph may be generated (in block 112) at runtime, alternatively this may be considered a pre-processing step and may be performed in advance [precomputed]; ¶0082, at least discloses the methods may also be used for quickly previewing an animation before doing more sophisticated animation using a rigging or keyframe system);
performing runtime deformation of the mesh (Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime. The attachment of the skeleton to the graph is performed once, while the transformations (in blocks 118-120) are performed per frame (e.g. at a frame rate of 30 Hz). In contrast, although the deformation graph may be generated (in block 112) at runtime, alternatively this may be considered a pre-processing step and may be performed in advance);
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang to incorporate the teachings of Chen, and apply receiving a precomputed animation of a character and the pre-processing step performed in advance into animation of a character having a skeleton and a skin, skeletal animation data and mesh binding data, as taught by Zhang, for receive a precomputed animation of a character having a skeleton and a skin, the precomputed animation including: precomputed skeletal animation data, the precomputed skeletal animation data having been precomputed from keyframes representing movement of the character and performing runtime deformation of the mesh based on the precomputed skeletal animation data.
The same motivation that was utilized in the rejection of claim 9 applies equally to this claim.
The prior art does not explicitly disclose, but Stern discloses
the skeletal animation data comprising a matrix representing a transformation of the skeleton (see Claim 9 rejection for detailed analysis), wherein a particular dimension of the matrix represents frames of the animation of the character (see Claim 9 rejection for detailed analysis);
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen to incorporate the teachings of Stern, and apply the matrix transformation into the Zhang/Chen’s teachings in order the skeletal animation data comprising a matrix representing a transformation of the skeleton, wherein a particular dimension of the matrix represents frames of the precomputed animation of the character.
The same motivation that was utilized in the rejection of claim 9 applies equally to this claim.
The prior art does not explicitly disclose, but Li disclosed
precomputed mesh binding data (Li- ¶0057, at least discloses Skinning means attaching (binding) a vertex in a mesh to a skeleton, and each vertex may be controlled by a plurality of skeletons; ¶0263, at least discloses coordinates of each vertex in a world coordinate system are calculated based on binding between the skeleton and the vertex in the mesh, to render the vertex and finally to obtain a displayed virtual object);
receive the skeletal animation data and the mesh binding data from the central processing unit (Li- Fig. 28B and ¶0280-0302, at least disclose 501: The CPU loads model data of an avatar accessory. The model data of the avatar accessory may include network data (including vertex data) of a first mesh model and skeleton information of a first virtual framework. Each mesh in a mesh model is generally of a triangle or another polygon. The mesh data includes the vertex data (a vertex table) and index data […] The skeleton information includes the number of all skeletons in the first virtual framework and specific information of each skeleton […] 502: The CPU obtains model data of a character (that is, a virtual object). The model data of the character may include skeleton information of a second virtual framework, vertex data of a second mesh model, and mounting correction information of the virtual accessory for the virtual object. For the skeleton information and the vertex data, refer to the descriptions in 501, and details are not described herein again. The mounting correction information may exist as an additional node (a mounting point) […] One mounting point may be bound to one or more skeletons. There is a parent-child relationship between a mounting point and a skeleton to which the mounting point is bound, and the mounting point is a child node. The parent-child relationship is similar to a parent-child between skeletons).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Stern to incorporate the teachings of Li, and apply the CPU and GPU into the Zhang/Stern’s teachings in order to receive the skeletal animation data and the mesh binding data from the central processing unit.
Doing so would harmoniously display a simulation object composited with an accessory, so that a user has desirable experience and visual enjoyment.
Li further disclosed
transfer the skeletal animation data and the mesh binding data to the graphics processing unit (Li- Fig. 28A and ¶0311-0321, at least disclose in an animation key frame, a transform matrix of each skeleton in the key frame relative to a coordinate system of a parent skeleton may be indicated. Alternatively, the animation key frame may record rotation, translation, and zoom of each joint in a joint state relative to a bound pose […] a transform matrix that corresponds to the jth skeleton in the second virtual framework in the current frame and that is in the world space is indicated by using matrix[j] […] The CPU transmits shading-related data of the character and the avatar accessory to a GPU […] the shading-related data may include vertex data of the character and the avatar accessory, matrix[i], matrix[j]r, vertex texture data of the first mesh model, and vertex texture data of the second mesh model. During actual running, the foregoing data may be transmitted in batches. For example, the vertex data of the character, matrix[i], and the vertex texture data of the second mesh model may be first transmitted, and then the vertex data of the avatar accessory, matrix[j]t, and the vertex texture data of the first mesh model are transmitted).
Regarding claim 17, Zhang in view of Chen, Stern and Li, discloses the system of claim 16, and further discloses wherein the central processing unit (see Claim 16 rejection for detailed analysis) is configured to:
instruct the graphics processing unit (see Claim 16 rejection for detailed analysis) to generate multiple parallel animations of the character according to the precomputed skeletal animation data and the precomputed mesh binding data (Zhang- ¶0016, at least discloses Step A includes creating the skeletal model and the mesh skinning model that constitute the character model using either Poser or 3ds MAX software; ¶0038, at least discloses the vertex of the model is defined in the model coordinate system, and for the game engine, the vertex position in the model coordinate system is converted into the world coordinate system for rendering […] by attaching each vertex to the corresponding bone, it can be rendered into an animation; Li- ¶0005, at least discloses the skin of the simulation object may change when the sizes of the skeletal model and the avatar are simultaneously changed).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang to incorporate the teachings of Stern and Li, and apply the change when the sizes of the skeletal model and the avatar are simultaneously changed into the Zhang’s teachings in order to instruct the graphics processing unit to generate multiple parallel animations of the character according to the precomputed skeletal animation data and the precomputed mesh binding data.
The same motivation that was utilized in the rejection of claim 16 applies equally to this claim.
Regarding claim 20, Zhang in view of Chen, Stern and Li, discloses the system of claim 16, and further discloses wherein the central processing unit (see Claim 16 rejection for detailed analysis) is configured to:
at runtime, instruct the graphics processing unit (see Claim 16 rejection for detailed analysis) to perform different portions of the precomputed animation on a per-joint basis in response to received user inputs (Zhang- ¶0037-0038, at least disclose step S104, calculating new positions and new orientations of the vertexes under the world coordinate system according to the bone indexes influencing the vertexes and the corresponding influence weights stored in the vertexes on the mesh skin model, and realizing the bone skin animation […] the vertices can then be rendered as animations by simply attaching them to the corresponding bones; Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime […] In contrast, although the deformation graph may be generated (in block 112) at runtime, alternatively this may be considered a pre-processing step and may be performed in advance; Li- ¶0322-0336, at least disclose each process of preparing data and instructing the GPU to perform shading is referred to as a draw call […] 508: Calculate a coordinate value of each mesh vertex in the second mesh model in the current time based on the transform matrix set of the second virtual framework in the current time. For a mesh vertex s in the second mesh model, a coordinate value thereof may be calculated).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen/Stern to incorporate the teachings of Li, and apply the GPU into the Zhang/Chen/Stern’s teachings in order to instruct the graphics processing unit to perform multiple other animations on a per-joint basis in response to received user inputs.
The same motivation that was utilized in the rejection of claim 16 applies equally to this claim.
8. Claims 3-5 and 11-14 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Chen, further in view of Stern, further in view of Jiang et al. (machine translation of CN-111738935-A with citation below, hereinafter “Jiang”)
Regarding claim 3, Zhang in view of Chen and Stern, discloses the method of claim 2, and further disclosed wherein the skeletal animation data (see Claim 1 rejection for detailed analysis) comprises textures (Zhang- ¶0013, at least discloses Each vertex includes its position, normal vector, texture material, and the bones affecting that vertex and their influence weights).
The prior art does not explicitly disclose, but Jiang discloses
each texture comprising rows representing the frames of the animation and columns representing individual bones (Jiang- ¶0009, at least disclose Store all animation frame data of the animation segment that needs to generate afterimages in a texture. Then, in the Vertex Shader (VS), retrieve the bone transformation matrix information of the corresponding animation frame from the texture based on the recorded frame number of the afterimage animation and perform skinning calculations. Here, the animation frame data refers to the set of matrices storing the rotation, translation, and pose data of all bones at a certain moment when playing the skeletal animation. The texture is equivalent to a two-dimensional array, and a floating-point number can be uniquely identified by the row index and column index; ¶0053-0054, at least disclose The pose data here refers to the transformation matrix of each bone. That is, the transformation matrix of each bone represents the absolute transformation matrix relative to world space, not the transformation matrix relative to the parent bone. The bone transformation matrix is generally a 4x4 square matrix. However, since the 4th row is fixed at (0,0,0,1), only the first three rows are stored during data storage. These three rows correspond to three pixels in the pose graph (each pixel can store four floating-point numbers). The motion data for each frame contains the transformation matrix of all bones in the skeleton. One or more rows of pixels in the pose graph (related to the number of bones and the resolution of the pose graph) can be used to store the transformation matrix data of all bones in a frame. The transformation matrix data of other frames are also stored sequentially in the pose graph).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen/Stern to incorporate the teachings of Jiang, and apply the texture is equivalent to a two-dimensional array with the row index and column index into the Zhang/Chen/Stern’s teachings in order the skeletal animation data comprises textures having rows representing frames and columns representing individual bones.
Doing so would solve the problem of slow animation creation equipment operation speed due to low skinning calculation efficiency in related technologies.
Regarding claim 4, Zhang in view of Chen, Stern and Jiang, discloses the method of claim 3, and further disclosed wherein the mesh binding data represents influence of individual joints on corresponding vertices of the mesh (Zhang- ¶0010-0012, at least disclose B. Determine the bones that affect the mesh vertices on the mesh skinning model, and determine their influence weights based on the geometric and physical relationships between the bones and the vertices […] D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone index and corresponding influence weight stored in each vertex of the mesh skinning model, and realize the skeletal skinning animation; ¶0037, at least discloses Step S104: Calculate the new position and orientation of each vertex in the world coordinate system according to the bone index and corresponding influence weight stored in each vertex of the mesh skinning model, and realize the skeletal skinning animation).
Regarding claim 5, Zhang in view of Chen, Stern and Jiang, discloses the method of claim 4, and further disclosed wherein the mesh binding data comprises a joint influence index array (Zhang- ¶0012-0013, at least disclose D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone index and corresponding influence weight stored in each vertex of the mesh skinning model, and realize the skeletal skinning animation […] Each vertex includes its position, normal vector, texture material, and the bones affecting that vertex and their influence weights. The bone information includes all the bones that make up the bone model. All bones are organized into a tree according to parent-child relationships, with the root representing the entire skeleton. Each node, including leaf nodes, represents a bone. Each bone includes its transformation matrix in the parent bone's coordinate system, which determines its position within the parent bone's coordinate system. The keyframe information includes a sequence of keyframes that constitute the bone's motion. Each keyframe stores the transformation matrix of each bone relative to the parent bone's coordinate system at that moment, as well as the bone's position and orientation information; ¶0015, at least discloses Step D specifically includes: according to the bone index and corresponding influence weight stored in each vertex of the mesh skinning model, based on the formula: new position of vertex = world matrix of each bone * inverse matrix of Tpos matrix of each bone, world matrix of each bone = world matrix of Tpos matrix of each bone * interpolated matrix * world matrix of parent bone, the accurate position of vertex is obtained by average weighting according to which bones affect the vertex, thus realizing bone skinning animation; ¶0164, at least discloses The new matrix is passed to the GPU as a bone array, and the new position and orientation of the vertex in world coordinates are calculated based on the weights of the bones that affect the vertex; Jiang- ¶0009, at least discloses The texture is equivalent to a two-dimensional array, and a floating-point number can be uniquely identified by the row index and column index).
Regarding claim 11, Zhang in view of Chen and Stern, discloses the method of claim 9, wherein the precomputed skeletal animation data comprises textures (see Claim 3 rejection for detailed analysis), and further discloses each texture having rows representing individual frames of the precomputed animation and columns representing individual bones of the skeleton (see Claim 3 rejection for detailed analysis).
Regarding claim 12, Zhang in view of Chen and Stern, discloses the method of claim 11, and further discloses wherein the precomputed mesh binding data represents influence of individual joints on corresponding vertices of the mesh (see Claim 4 rejection for detailed analysis).
Regarding claim 13, Zhang in view of Chen and Stern, discloses the method of claim 11, and discloses the method further comprising:
at runtime, selectively applying the precomputed skeletal animation data based on received input (Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime).
The prior art does not explicitly disclose a subset of the rows of the skeletal animation data based on received input, resulting in replaying a subset of the animation of the character represented by the subset of rows.
However, Jiang discloses
a subset of the rows of the skeletal animation data based on received input, resulting in replaying a subset of the animation of the character represented by the subset of rows (Jiang- ¶0053-0054, at least disclose The pose data here refers to the transformation matrix of each bone. That is, the transformation matrix of each bone represents the absolute transformation matrix relative to world space, not the transformation matrix relative to the parent bone. The bone transformation matrix is generally a 4x4 square matrix. However, since the 4th row is fixed at (0,0,0,1), only the first three rows are stored during data storage. These three rows correspond to three pixels in the pose graph (each pixel can store four floating-point numbers). The motion data for each frame contains the transformation matrix of all bones in the skeleton. One or more rows of pixels in the pose graph (related to the number of bones and the resolution of the pose graph) can be used to store the transformation matrix data of all bones in a frame).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen/Stern to incorporate the teachings of Jiang, and apply moving in real-time and one or more rows of pixels related to the number of bones into the Zhang/Chen/Stern’s teachings for selectively applying a subset of the rows of the skeletal animation data based on input received at runtime.
Doing so would be desirable to have features that leverage mixed reality to enrich these person-to-person interactions.
Regarding claim 14, Zhang in view of Chen and Stern, discloses the method of claim 11, and discloses the method further comprising:
at runtime, selectively applying of the precomputed skeletal animation data based on received input (Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime).
The prior art does not explicitly disclose selectively applying a subset of the columns of the skeletal animation data based on input received at runtime, resulting in movement of a portion of a character corresponding to a subset of the bones represented by the subset of the columns.
However, Jiang discloses
a subset of the columns of the skeletal animation data based on input received at runtime, resulting in movement of a portion of a character corresponding to a subset of the bones represented by the subset of the columns (Jiang- ¶0009, at least discloses Store all animation frame data of the animation segment that needs to generate afterimages in a texture. Then, in the Vertex Shader (VS), retrieve the bone transformation matrix information of the corresponding animation frame from the texture based on the recorded frame number of the afterimage animation and perform skinning calculations. Here, the animation frame data refers to the set of matrices storing the rotation, translation, and pose data of all bones at a certain moment when playing the skeletal animation. The texture is equivalent to a two-dimensional array, and a floating-point number can be uniquely identified by the row index and column index; Jiang- ¶0053-0054, at least disclose The pose data here refers to the transformation matrix of each bone. That is, the transformation matrix of each bone represents the absolute transformation matrix relative to world space, not the transformation matrix relative to the parent bone. The bone transformation matrix is generally a 4x4 square matrix. However, since the 4th row is fixed at (0,0,0,1), only the first three rows are stored during data storage).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen/Stern to incorporate the teachings of Jiang, and apply column index into the Zhang/Chen/Stern’s teachings for selectively applying a subset of the columns of the skeletal animation data based on input received at runtime, resulting in movement of a portion of a character corresponding to a subset of the bones represented by the subset of the columns.
Doing so would be desirable to have features that leverage mixed reality to enrich these person-to-person interactions.
9. Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Chen, further in view of Stern, further in view of Jiang et al. (machine translation of CN-111738935-A with citation below, hereinafter “Jiang”), still further in view of Wu et al., (“Wu”) [US-2024/0307779-A1]
Regarding claim 6, Zhang in view of Chen, Stern and Jiang, discloses the method of claim 5, and discloses the method further comprising:
receiving a second mesh representing a second skin of a second charactera mesh skin model that constitute the character model […] The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] The mesh skinning information is the character's polygonal model, composed of triangular faces. Each triangle has three indices pointing to the model's vertex table, which identify the three vertices);
precomputing a second animation of the second character by binding the second mesh to the skeleton to obtain second mesh binding data (Zhang- ¶0009-0013, at least disclose A. Create a skeletal model and a mesh skin model that constitute the character model […] The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone indexes and corresponding influence weights stored in each vertex of the mesh skinning model and influence the vertex, and realize the skeletal skinning animation […] The mesh skinning information is the character's polygonal model, composed of triangular faces. Each triangle has three indices pointing to the model's vertex table, which identify the three vertices; ¶0038, at least discloses by attaching each vertex to the corresponding bone, it can be rendered into an animation; Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime […] In contrast, although the deformation graph may be generated (in block 112) at runtime, alternatively this may be considered a pre-processing step and may be performed in advance); and
outputting the second mesh binding data, the skeletal animation data and the second mesh binding data providing a basis for subsequent animation of the second character at runtime by deforming the second mesh in accordance with the skeletal animation data (Zhang- ¶0009-0013, at least disclose A. Create a skeletal model and a mesh skin model that constitute the character model […] The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone indexes and corresponding influence weights stored in each vertex of the mesh skinning model and influence the vertex, and realize the skeletal skinning animation […] The mesh skinning information is the character's polygonal model, composed of triangular faces. Each triangle has three indices pointing to the model's vertex table, which identify the three vertices; ¶0016, at least discloses Step A includes creating the skeletal model and the mesh skinning model that constitute the character model using either Poser or 3ds MAX software; Fig. 1 and ¶0032, at least disclose Step S101: Create the skeletal model and a mesh skinning model that constitute the character model […] The mesh skinning model, which is the character’s skin, is a deformable mesh that changes under the influence of the skeleton; ¶0038, at least discloses the vertex of the model is defined in the model coordinate system, and for the game engine, the vertex position in the model coordinate system is converted into the world coordinate system for rendering […] by attaching each vertex to the corresponding bone, it can be rendered into an animation; Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime).
The prior art does not explicitly disclose, but Wu discloses
a second skin of a second character that shares the skeleton with the character (Wu- ¶0028, at least discloses Animation Retargeting: it is a function that allows animation to be reused between virtual character models that share the same skeleton framework resource but have greatly different proportions. The retargeting can prevent a skeleton framework that generates the animation from losing a proportion or deforming unnecessarily when using animations from virtual character models with different profiles Fig. 3 and ¶0062-0063, at least disclose virtual human models share a set of calculation manners, virtual reptile models share another set of calculation manners, and a position of a foot key point is obtained by a foot joint of the human skeleton framework through the preset calculation manner […] ) In addition, since in a model space, all skeleton joints or key point positions share a coordinate system of this space, and in a joint space, a coordinate system is established on a parent joint and a child joint position coordinate depends on the parent joint, the position data of the skeleton framework end is defined in the model space rather than the joint space).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen/Stern/Jiang to incorporate the teachings of Wu, and apply the virtual character models that share the same skeleton framework resource into the Zhang/Chen/Stern/Jiang’s teachings in order the animation and the another animation representing different characters that share the skeleton.
Doing so would prevent a skeleton framework that generates the animation from losing a proportion or deforming unnecessarily when using animations from virtual character models with different profiles.
10. Claims 21-23 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Chen, further in view of Stern, still further in view of Jiang et al. (machine translation of CN-111738935-A with citation below, hereinafter “Jiang”), still further in view of Nimura et al. (“Nimura”) [US-2003/0043154-A1]
Regarding claim 21, Zhang in view of Chen and Stern, discloses the method of claim 11, and discloses the method further comprising:
at runtimeof the precomputed skeletal animation data (Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime); and
at runtime of the precomputed skeletal animation dataleast discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime).
The prior art does not explicitly disclose responsive to receiving first input indicating a first button has been pushed on a video game controller, replaying a first subset of the precomputed animation involving movement of part of a character through a first range of motion by applying a first subset of the rows of the textures of the precomputed skeletal animation data; and responsive to receiving second input indicating a second button has been pushed on the video game controller, replaying a second subset of the precomputed animation involving movement of the part of the character through a second range of motion by applying a second subset of the rows of the textures of the precomputed skeletal animation data, the second range of motion being different from the first range of motion.
However, Jiang discloses
applying a first subset of the rows of the textures of the precomputed skeletal animation data (Jiang- ¶0053-0054, at least disclose The pose data here refers to the transformation matrix of each bone. That is, the transformation matrix of each bone represents the absolute transformation matrix relative to world space, not the transformation matrix relative to the parent bone. The bone transformation matrix is generally a 4x4 square matrix. However, since the 4th row is fixed at (0,0,0,1), only the first three rows are stored during data storage. These three rows correspond to three pixels in the pose graph (each pixel can store four floating-point numbers). The motion data for each frame contains the transformation matrix of all bones in the skeleton. One or more rows of pixels in the pose graph (related to the number of bones and the resolution of the pose graph) can be used to store the transformation matrix data of all bones in a frame); and
applying a second subset of the rows of the textures of the precomputed skeletal animation data (Jiang- ¶0053-0054, at least disclose The pose data here refers to the transformation matrix of each bone. That is, the transformation matrix of each bone represents the absolute transformation matrix relative to world space, not the transformation matrix relative to the parent bone. The bone transformation matrix is generally a 4x4 square matrix. However, since the 4th row is fixed at (0,0,0,1), only the first three rows are stored during data storage. These three rows correspond to three pixels in the pose graph (each pixel can store four floating-point numbers). The motion data for each frame contains the transformation matrix of all bones in the skeleton. One or more rows of pixels in the pose graph (related to the number of bones and the resolution of the pose graph) can be used to store the transformation matrix data of all bones in a frame).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen/Stern to incorporate the teachings of Jiang, and apply the texture is equivalent to a two-dimensional array with the row index and column index into the Zhang/Chen/Stern’s teachings for applying a first subset of the rows of the textures of the precomputed skeletal animation data; applying a second subset of the rows of the textures of the precomputed skeletal animation data.
Doing so would solve the problem of slow animation creation equipment operation speed due to low skinning calculation efficiency in related technologies.
Nimura discloses
responsive to receiving first input indicating a first button has been pushed on a video game controller (Nimura- Fig. 1 and ¶0089, at least disclose the motion processing section 112 comprises a first motion control section 114 (a main-side motion control section), a second motion control section 116 (a sub-side motion control section), and a motion blend section 118; ¶0205, at least discloses With this standing motion script, standing motion data is first selected (step S 1). A motion script switching condition is then determined, based on manipulation input (manipulation data from operating buttons or joystick) and status information for the model object (the current situation of the model object) (step S2); ¶0259, at least discloses Manipulation data from a game controller 942 (such as a joystick, buttons, casing, or a bat-shaped or handgun-shaped controller)), replaying a first subset of the animation involving movement of part of a character through a first range of motion (Nimura- ¶0050, at least discloses selecting motion data for a model object and controlling motion of the model object by a plurality of motion control sections [range of motion], based on motion scripts in which are defined the motion data to be selected and motion script switching conditions; ¶0054-0055, at least disclose the plurality of motion control sections to operate in parallel, enabling a motion blend of a plurality of sets of motion data selected by this plurality of motion control sections. It is therefore possible to implement motion control by kind of a motion script obtained by combining first motion scripts of the first motion control section [first range of motion] and second motion scripts of the second motion control section, by way of example. This enables the implementation of the representation of a wide range of motion scripts with a small amount of data […] Note that the motion data selected by each of the first and second motion control sections could be obtained by blending a plurality of sets of motion data; ¶0058, at least discloses first group of the motion scripts may be allocated to a first motion control section of the plurality of motion control sections; ¶0061, at least discloses This would make it possible for the first motion control section to operate in parallel with the second motion control section, even after the second motion control section has started operating. This therefore makes it possible to represent motions such that a first motion is played (replayed) by the first motion section, then a blend of first and second motions is played by the first and second motion control sections, and finally the motion play returns to the first motion of the first motion control section); and
responsive to receiving second input indicating a second button has been pushed on the video game controller (Nimura- Fig. 1 and ¶0089, at least disclose the motion processing section 112 comprises a first motion control section 114 (a main-side motion control section), a second motion control section 116 (a sub-side motion control section), and a motion blend section 118; ¶0205, at least discloses With this standing motion script, standing motion data is first selected (step S 1). A motion script switching condition is then determined, based on manipulation input (manipulation data from operating buttons or joystick) and status information for the model object (the current situation of the model object) (step S2); ); ¶0259, at least discloses Manipulation data from a game controller 942 (such as a joystick, buttons, casing, or a bat-shaped or handgun-shaped controller)), replaying a second subset of the animation involving movement of the part of the character through a second range of motion controlling motion of the model object by a plurality of motion control sections [range of motion], based on motion scripts in which are defined the motion data to be selected and motion script switching conditions; ¶0054-0055, at least disclose the plurality of motion control sections to operate in parallel, enabling a motion blend of a plurality of sets of motion data selected by this plurality of motion control sections. It is therefore possible to implement motion control by kind of a motion script obtained by combining first motion scripts of the first motion control section [first range of motion] and second motion scripts of the second motion control section, by way of example. This enables the implementation of the representation of a wide range of motion scripts with a small amount of data […] Note that the motion data selected by each of the first and second motion control sections could be obtained by blending a plurality of sets of motion data; ¶0058, at least discloses a second group of the motion scripts may be allocated to a second motion control section of the motion control sections; ¶0061, at least discloses This would make it possible for the first motion control section to operate in parallel with the second motion control section, even after the second motion control section has started operating. This therefore makes it possible to represent motions such that a first motion is played (replayed) by the first motion section, then a blend of first and second motions is played by the first and second motion control sections, and finally the motion play returns to the first motion of the first motion control section),
the second range of motion being different from the first range of motion (Nimura- ¶0089-0090, at least disclose the motion processing section 112 comprises a first motion control section 114 (a main-side motion control section), a second motion control section 116 (a sub-side motion control section), and a motion blend section 118. Note that the configuration could be such that three or more motion control sections are provided […] In this case, the first and second motion control sections 114 and 116 select motion data for a model object, based on a motion script (a script that defines a processing sequence for motion control) stored in a motion script storage section 178; ¶0097, at least discloses The blend data that is the objective of the blending done by the motion blend section 118 is selected by the first and second motion control sections 114 and 116. In this case, the first and second motion control sections 114 and 116 could select the motion data created by the blend processing of the motion blend section 118, and the thus-selected motion data could then be blended again by the motion blend section 118; Fig. 5 and ¶0139, at least discloses A series of transitional motions of the model object (the motion data MA) is shown at C 1 to C5 in FIG. 5. Similarly, a series of attack motions of the model object (the motion data MB) is shown at D1 to D5 of FIG. 5.; Also see fig. 8).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen/Stern to incorporate the teachings of Jiang and Nimura, and apply first motion control section, a second motion control section into the Zhang/Chen/Stern’s teachings at runtime and in responsive to receiving first input indicating a first button has been pushed on a video game controller, replaying a first subset of the precomputed animation involving movement of part of the character through a first range of motion by applying a first subset of the rows of the textures of the precomputed skeletal animation data; and at runtime and responsive to receiving second input indicating a second button has been pushed on the video game controller, replaying a second subset of the precomputed animation involving movement of the part of the character through a second range of motion by applying a second subset of the rows of the textures of the precomputed skeletal animation data, the second range of motion being different from the first range of motion.
Doing so would enable players to enjoy a fighting game, as an example, each player uses a game controller (manipulation section) to manipulate his or her own character, to enjoy a game of combat against an enemy character manipulated by another player or a computer.
Regarding claim 22, Zhang in view of Chen, Stern, Jiang and Nimura, discloses the method of claim 21, and further discloses wherein the precomputed skeletal animation data represents movement of the character through a third range of motion that encompasses the first range of motion and the second range of motion (Nimura- Fig. 1 and ¶0089, at least disclose the motion processing section 112 comprises a first motion control section 114 (a main-side motion control section), a second motion control section 116 (a sub-side motion control section), and a motion blend section 118 [third range of motion]. Note that the configuration could be such that three or more motion control sections are provided.).
Regarding claim 23, Zhang in view of Chen and Stern, discloses the method of claim 11, and discloses the method further comprising:
at runtime of the precomputed skeletal animation data corresponding to bones of the character (Chen- ¶0001, at least discloses at runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0009, at least discloses Create a skeletal model and a mesh skinning model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of bones describes the structure of the character. The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of the bones; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime); and
at runtime of the precomputed skeletal animation data corresponding to bones of the characterat runtime, motion capture data or an inverse kinematics engine drives the bones of the character, which then transforms the mesh; ¶0009, at least discloses Create a skeletal model and a mesh skinning model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of bones describes the structure of the character. The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of the bones; ¶0026, at least discloses the attachment of the skeleton to the graph and subsequent transformation of both the graph and the mesh (blocks 114-120) are performed in real-time at runtime).
The prior art does not explicitly disclose responsive to receiving first input indicating a first button has been pushed on a video game controller, replaying a first subset of the precomputed animation involving movement of a first part of the character by applying a first subset of the columns of the textures of the precomputed skeletal animation data that move the first part of the character; and responsive to receiving second input indicating a second button has been pushed on the video game controller, replaying a second subset of the precomputed animation involving movement of a second part of the character by applying a second subset of the columns of the textures of the precomputed skeletal animation data corresponding to bones that move the second part of the character, the first part of the character being different from the second part of the character.
However, Jiang discloses
applying a first subset of the columns of the textures of the precomputed skeletal animation data corresponding to bones that move the first part of the character (Jiang- ¶0009, at least discloses Store all animation frame data of the animation segment that needs to generate afterimages in a texture. Then, in the Vertex Shader (VS), retrieve the bone transformation matrix information of the corresponding animation frame from the texture based on the recorded frame number of the afterimage animation and perform skinning calculations. Here, the animation frame data refers to the set of matrices storing the rotation, translation, and pose data of all bones at a certain moment when playing the skeletal animation. The texture is equivalent to a two-dimensional array, and a floating-point number can be uniquely identified by the row index and column index; Jiang- ¶0053-0054, at least disclose The pose data here refers to the transformation matrix of each bone. That is, the transformation matrix of each bone represents the absolute transformation matrix relative to world space, not the transformation matrix relative to the parent bone. The bone transformation matrix is generally a 4x4 square matrix. However, since the 4th row is fixed at (0,0,0,1), only the first three rows are stored during data storage); and
applying a second subset of the columns of the textures of the precomputed skeletal animation data corresponding to bones that move the second part of the character (Jiang- ¶0009, at least discloses Store all animation frame data of the animation segment that needs to generate afterimages in a texture. Then, in the Vertex Shader (VS), retrieve the bone transformation matrix information of the corresponding animation frame from the texture based on the recorded frame number of the afterimage animation and perform skinning calculations. Here, the animation frame data refers to the set of matrices storing the rotation, translation, and pose data of all bones at a certain moment when playing the skeletal animation. The texture is equivalent to a two-dimensional array, and a floating-point number can be uniquely identified by the row index and column index; Jiang- ¶0053-0054, at least disclose The pose data here refers to the transformation matrix of each bone. That is, the transformation matrix of each bone represents the absolute transformation matrix relative to world space, not the transformation matrix relative to the parent bone. The bone transformation matrix is generally a 4x4 square matrix. However, since the 4th row is fixed at (0,0,0,1), only the first three rows are stored during data storage).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen/Stern to incorporate the teachings of Jiang, and apply the texture is equivalent to a two-dimensional array with the row index and column index into the Zhang/Chen/Stern’s teachings for applying a first subset of the columns of the textures of the precomputed skeletal animation data corresponding to bones; applying a second subset of the columns of the textures of the precomputed skeletal animation data corresponding to bones.
Doing so would solve the problem of slow animation creation equipment operation speed due to low skinning calculation efficiency in related technologies
Nimura discloses
responsive to receiving first input indicating a first button has been pushed on a video game controller (Nimura- Fig. 1 and ¶0089, at least disclose the motion processing section 112 comprises a first motion control section 114 (a main-side motion control section), a second motion control section 116 (a sub-side motion control section), and a motion blend section 118; ¶0205, at least discloses With this standing motion script, standing motion data is first selected (step S 1). A motion script switching condition is then determined, based on manipulation input (manipulation data from operating buttons or joystick) and status information for the model object (the current situation of the model object) (step S2); ¶0259, at least discloses Manipulation data from a game controller 942 (such as a joystick, buttons, casing, or a bat-shaped or handgun-shaped controller)), replaying a first subset of the precomputed animation involving movement of a first part of the character (Nimura- ¶0050, at least discloses selecting motion data for a model object and controlling motion of the model object by a plurality of motion control sections [parts of the character], based on motion scripts in which are defined the motion data to be selected and motion script switching conditions; ¶0054-0055, at least disclose the plurality of motion control sections to operate in parallel, enabling a motion blend of a plurality of sets of motion data selected by this plurality of motion control sections. It is therefore possible to implement motion control by kind of a motion script obtained by combining first motion scripts of the first motion control section [first part of the character] and second motion scripts of the second motion control section, by way of example. This enables the implementation of the representation of a wide range of motion scripts with a small amount of data […] Note that the motion data selected by each of the first and second motion control sections could be obtained by blending a plurality of sets of motion data; ¶0058, at least discloses first group of the motion scripts may be allocated to a first motion control section of the plurality of motion control sections [Wingdings font/0xE0] “first motion control section” suggests first part of the character; ¶0061, at least discloses This would make it possible for the first motion control section to operate in parallel with the second motion control section, even after the second motion control section has started operating. This therefore makes it possible to represent motions such that a first motion is played (replayed) by the first motion section, then a blend of first and second motions is played by the first and second motion control sections, and finally the motion play returns to the first motion of the first motion control section); and
responsive to receiving second input indicating a second button has been pushed on the video game controller (Nimura- Fig. 1 and ¶0089, at least disclose the motion processing section 112 comprises a first motion control section 114 (a main-side motion control section), a second motion control section 116 (a sub-side motion control section), and a motion blend section 118; ¶0205, at least discloses With this standing motion script, standing motion data is first selected (step S 1). A motion script switching condition is then determined, based on manipulation input (manipulation data from operating buttons or joystick) and status information for the model object (the current situation of the model object) (step S2); ); ¶0259, at least discloses Manipulation data from a game controller 942 (such as a joystick, buttons, casing, or a bat-shaped or handgun-shaped controller)), replaying a second subset of the precomputed animation involving movement of a second part of the character (Nimura- ¶0050, at least discloses selecting motion data for a model object and controlling motion of the model object by a plurality of motion control sections [parts of the character], based on motion scripts in which are defined the motion data to be selected and motion script switching conditions; ¶0054-0055, at least disclose the plurality of motion control sections to operate in parallel, enabling a motion blend of a plurality of sets of motion data selected by this plurality of motion control sections. It is therefore possible to implement motion control by kind of a motion script obtained by combining first motion scripts of the first motion control section and second motion scripts of the second motion control section [second part of the character], by way of example. This enables the implementation of the representation of a wide range of motion scripts with a small amount of data […] Note that the motion data selected by each of the first and second motion control sections could be obtained by blending a plurality of sets of motion data [Wingdings font/0xE0] “first and second motion control sections” suggests first part of the character and second part of the character; ¶0058, at least discloses a second group of the motion scripts may be allocated to a second motion control section of the motion control sections; ¶0061, at least discloses This would make it possible for the first motion control section to operate in parallel with the second motion control section, even after the second motion control section has started operating. This therefore makes it possible to represent motions such that a first motion is played (replayed) by the first motion section, then a blend of first and second motions is played by the first and second motion control sections, and finally the motion play returns to the first motion of the first motion control section),
the first part of the character being different from the second part of the character (Nimura- ¶0089-0090, at least disclose the motion processing section 112 comprises a first motion control section 114 (a main-side motion control section), a second motion control section 116 (a sub-side motion control section), and a motion blend section 118. Note that the configuration could be such that three or more motion control sections are provided […] In this case, the first and second motion control sections 114 and 116 select motion data for a model object, based on a motion script (a script that defines a processing sequence for motion control) stored in a motion script storage section 178; ¶0097, at least discloses The blend data that is the objective of the blending done by the motion blend section 118 is selected by the first and second motion control sections 114 and 116. In this case, the first and second motion control sections 114 and 116 could select the motion data created by the blend processing of the motion blend section 118, and the thus-selected motion data could then be blended again by the motion blend section 118 [Wingdings font/0xE0] “the first and second motion control sections” suggests the first part of the character being different from the second part of the character; Fig. 5 and ¶0139, at least discloses A series of transitional motions of the model object (the motion data MA) is shown at C 1 to C5 in FIG. 5. Similarly, a series of attack motions of the model object (the motion data MB) is shown at D1 to D5 of FIG. 5.; Also see fig. 8).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen/Stern to incorporate the teachings of Jiang and Nimura, and apply first motion control section, a second motion control section into the Zhang/Chen/Stern’s teachings at runtime and responsive to receiving first input indicating a first button has been pushed on a video game controller, replaying a first subset of the precomputed animation involving movement of a first part of the character by applying a first subset of the columns of the textures of the precomputed skeletal animation data corresponding to bones that move the first part of the character; and at runtime and responsive to receiving second input indicating a second button has been pushed on the video game controller, replaying a second subset of the precomputed animation involving movement of a second part of the character by applying a second subset of the columns of the textures of the precomputed skeletal animation data corresponding to bones that move the second part of the character, the first part of the character being different from the second part of the character.
Doing so would enable players to enjoy a fighting game, as an example, each player uses a game controller (manipulation section) to manipulate his or her own character, to enjoy a game of combat against an enemy character manipulated by another player or a computer.
11. Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Chen, further in view of Stern, still further in view of Li, still further in view of Horsman et al., (“Horsman”) [US-2020/0035021-A1]
Regarding claim 18, Zhang in view of Chen, Stern and Li, discloses the system of claim 16, and further discloses wherein the precomputed skeletal animation data includes one or more textures (Zhang- ¶0013, at least discloses Each vertex includes its position, normal vector, texture material, and the bones affecting that vertex and their influence weights), and the prior art does not explicitly disclose, but Horsman discloses wherein the precomputed mesh binding data (see Claim 16 rejection for detailed analysis) includes one or more UV maps (Horsman- ¶0012, at least discloses receiving a sequence of UV mapped meshes of a humanoid character comprising different camera positions, stabilizing a topology of the sequence of UV mapped meshes, wherein the stabilizing results in a stabilized mesh, and synthesizing a group of stabilized meshes comprising the stabilized mesh, wherein the synthesizing results in a synthesized topology texture map sequence; ¶0060, at least discloses a mesh and MVE can be used to generate a mesh with UV's and texture. Processing Service 150 can further perform Skin Binding Generation 162. Skin Binding Generation 162 can bind a skin or a surface texture of a human, humanoid rig, or other object to a skeletonization associated with the human, humanoid rig, or object; ¶0067, at least discloses mesh: with UV mapping or with an MVE, stabilized mesh (sequence of meshes with stable UVs/vertex-face relationships), texture sequence (png), SSDR Sequence, etc; ¶0072, at least discloses Keyframes can comprise a mesh vertex, normal and triangle index stream, SSDR bone bindings/initial pose, and a humanoid bone binding/initial pose […] A humanoid bone binding exists for each keyframe and assumes the mesh has already been transformed into the pose for frame n, and that is then used to transform the vertex again based on the dynamic humanoid skeleton pose for the currently displayed frame; Fig. 9 and ¶0080, at least discloses a mesh of a sequence of UV mapped meshes can comprise different topologies and UV layouts).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen/Stern/Li to incorporate the teachings of Horsman, and apply the UV mapped meshes of a humanoid character into the Zhang/Chen/Stern/Li’s teachings in order the mesh binding data includes one or more UV maps.
Doing so would complete end-to-end experience involving capture of a volumetric performance and ultimate distribution to consumers of that volumetric performance.
12. Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Chen, further in view of Stern, still further in view of Li, still further in view of Horsman, still further in view of McBeth (“McBeth”) [US-2019/0108681-A1]
Regarding claim 19, Zhang in view of Chen, Stern, Li and Horsman, discloses the system of claim 18, and further discloses wherein the graphics processing unit (see Claim 16 rejection for detailed analysis) is configured to:
the mesh based on the one or more textures (Zhang- ¶0013, at least discloses Each vertex includes its position, normal vector, texture material, and the bones affecting that vertex and their influence weights) and the one or more UV maps to generate the animation (Horsman- ¶0012, at least discloses receiving a sequence of UV mapped meshes of a humanoid character comprising different camera positions, stabilizing a topology of the sequence of UV mapped meshes, wherein the stabilizing results in a stabilized mesh, and synthesizing a group of stabilized meshes comprising the stabilized mesh, wherein the synthesizing results in a synthesized topology texture map sequence).
The prior art does not explicitly disclose, but McBeth discloses
offset vertex positions of the mesh (McBeth- ¶0043, at least discloses At block 404, mixed reality device 104 can place the floating customization at an offset from the point determined at block 402, where the offset is defined as part of the customization. For example, a floating animal familiar may be positioned slightly down and to the left of the individual's head, while a halo may be positioned directly above the individual's head).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen/Stern/Li/Horsman to incorporate the teachings of McBeth, and apply the offset into the Zhang/Chen/Stern/Li/Horsman’s teachings for offsetting vertex positions of the mesh based on the skeletal animation data.
Doing so would be desirable to have features that leverage mixed reality to enrich these person-to-person interactions.
13. Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Chen, further in view of Stern, still further in view of McBeth (“McBeth”) [US-2019/0108681-A1]
Regarding claim 10, Zhang in view of Chen and Stern, discloses the method of claim 9, and the prior art does not explicitly disclose, but McBeth discloses wherein animating the character comprises:
offsetting vertex positions of the mesh based on the precomputed skeletal animation data and the precomputed mesh binding data (McBeth- ¶0043, at least discloses At block 404, mixed reality device 104 can place the floating customization at an offset from the point determined at block 402, where the offset is defined as part of the customization. For example, a floating animal familiar may be positioned slightly down and to the left of the individual's head, while a halo may be positioned directly above the individual's head).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Chen/Stern to incorporate the teachings of McBeth, and apply the offset into the Zhang/Chen/Stern’s teachings for offsetting vertex positions of the mesh based on the skeletal animation data.
Doing so would be desirable to have features that leverage mixed reality to enrich these person-to-person interactions.
Conclusion
14. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. They are recited in the attached PTO-892 form.
15. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL LE whose telephone number is (571)272-5330. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL LE/Primary Examiner, Art Unit 2614
