DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
2. Applicant’s amendments filed on 02/17/2026 have been entered. Claims 1 1-3, 6-9, 11, 13, 14 and 16 have been amended. Claim 21 have been added. Claims 1-21 are pending in this application, with claims 1, 9 and 16 being independent.
Response to Arguments
3. Applicant's arguments filed on 02/17/2026, with respect to the 103 rejection have been fully considered but are moot in view of the new grounds of rejection.
In light of the current Office Action, the Examiner respectfully submits that independent claims 1, 9 and 16 are rejected in view of newly discovered reference(s) to Stern (US-4,600,919-A).
Examiner notes that independent claims 1, 9 and 16 have been amended to include new limitation. Examiner finds these limitations to be unpatentable as can be found in below detail action.
4. On page 9 of Applicant's Remarks, the Applicant argues that the corresponding dependent claims are not taught by the prior art, insomuch as they depend from claims that are not taught by the prior art. Examiner respectfully disagrees with these arguments, for the reasons discussed above.
Claim Rejections - 35 USC § 103
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 1-2, 9-12 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (machine translation of CN-104021584-A with citation below, hereinafter “Zhang”) in view of Stern (“Stern”) [US-4,600,919-A], further in view of McBeth (“McBeth”) [US-2019/0108681-A1]
Regarding claim 1, Zhang discloses a method (Zhang- ¶0002, at least discloses an method for implementation skeletal skinning animation), comprising:
generating skeletal animation data representing movement of a skeleton having joints connecting bones (Zhang- ¶0009-0013, at least disclose A. Create a skeletal model and a mesh skin model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of the skeletons describes the structure of the character. The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] C. Based on the keyframe sequence of skeletal motion stored in the character model file, perform interpolation calculation between two adjacent keyframes to determine the new position and orientation of each bone at a certain moment […] The keyframe information includes a sequence of keyframes that constitute the bone’s motion; Fig. 1 and ¶0032, at least disclose Step S101: Create the skeletal model and a mesh skinning model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of bones describes the structure of the character. The mesh skinning model, which is the character’s skin, is a deformable mesh that changes under the influence of the skeleton; ¶0156, at least discloses the calculation can be used to obtain the interpolation matrix of the skeleton during movement, rotation, and scaling);
binding a mesh to the skeleton to obtain mesh binding data (Zhang- ¶0009-0012, at least discloses A. Create a skeletal model and a mesh skin model that constitute the character model […] The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] B. Determine the bones that affect the mesh vertices on the mesh skinning model, and determine their influence weights based on the geometric and physical relationships between the bones and the vertexes […] D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone indexes and corresponding influence weights stored in each vertex of the mesh skinning model and influence the vertex, and realize the skeletal skinning animation [binding a mesh to the skeleton to obtain mesh binding data]; Fig. 1 and ¶0037, at least disclose Step S104: Calculate the new positions and orientations of each vertex in the world coordinate system according to the bone index and corresponding influence weight stored in each vertex of the mesh skin model [binding a mesh], and realize the skeletal skinning animation); and
outputting the skeletal animation data and the mesh binding data (Zhang- ¶0016, at least discloses Step A includes creating the skeletal model and the mesh skinning model that constitute the character model using either Poser or 3ds MAX software; ¶0038, at least discloses the vertex of the model is defined in the model coordinate system, and for the game engine, the vertex position in the model coordinate system is converted into the world coordinate system for rendering […] by attaching each vertex to the corresponding bone, it can be rendered into an animation), the skeletal animation data and the mesh binding data providing a basis for deformation of the mesh to generate an animation of the mesh (Zhang- ¶0009-0012, at least disclose A. Create a skeletal model and a mesh skin model that constitute the character model […] The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone indexes and corresponding influence weights stored in each vertex of the mesh skinning model and influence the vertex, and realize the skeletal skinning animation; Fig. 1 and ¶0032, at least disclose Step S101: Create the skeletal model and a mesh skinning model that constitute the character model […] The mesh skinning model, which is the character’s skin, is a deformable mesh that changes under the influence of the skeleton [deformation of the mesh to generate an animation of the mesh]).
Zhang does not explicitly disclose the skeletal animation data comprising a matrix representing a transformation of the skeleton, wherein a particular dimension of the matrix represents frames of an animation of the skeleton; a basis for runtime deformation of the mesh.
However, Stern discloses
the skeletal animation data comprising a matrix representing a transformation of the skeleton (Stern- col 7, lines 30-37, at least discloses matrix transformations [matrix representing a transformation] are employed to transform the limb vectors in a given local coordinate system into equivalent vectors in the joint coordinate system of next highest priority. The procedure is repeated for successively higher priority joint coordinate systems until the limb vectors are expressed in terms of the world coordinate system; col 7, lines 51-65, at least discloses The matrix transformation which translates a point (x,y,z) to a new point (x',y',z') is known to be: ##EQU4## where mx, my and mz are the components of the translation or move in the x, y and z directions respectively, and the 1 in each fourth column is the homogeneous coordinate; Fig. 5 and col 9, lines 36-42, at least disclose If the figure skeletons are to be displayed, a basic orthogonal coordinate system representation (i.e., three mutually orthogonal lines), can be multiplied by the concatenated matrix transformation, so as to obtain properly oriented and scaled local coordinate axes representation at each joint position, as shown in the small sketch near block 516), wherein a particular dimension of the matrix represents frames of an animation of the skeleton (Stern- col 6, lines 26 to col 7, line 24, at least disclose FIGS. 4A and 4B are useful in understanding the manner in which the joints and limbs are represented and manipulated. In FIG. 4A, there is shown the coordinate axes for an exemplary joint, Ji, and a limb, Li, (shown in dashed line) of vectors associated with the joint. The matrix values for the joint Ji is as follows: ##EQU2## As seen in FIG. 4A, the origin of the local coordinate system for Ji is at the point (10, 10, 10) in the main or "world" coordinate system (which, in this example, is the coordinate system of next higher priority in the hierarchy). Also, the x, y, and z axes are aligned with the axes of the world coordinate system, so the relative rotation angles in the second row of the matrix are all 0°. Finally, it is assumed that, for this example, the scale factors for the frame are all unity […] Reference can now be made to FIG. 4B which represents the same joint Ji and limb Li, but with the position, orientation, and size and shape of the limb being different by virtue of modifications in the matrix values of the examplary joint. In particular, the matrix for the joint of FIG. 4B is as follows: ##EQU3## In this case, the origin of the local coordinate axes (i.e., the joint position) is seen to be at the coordinates (10, 15, 10), which correspond to the values in the first row of the matrix. Also, it is seen that the y, z axes are rotated by 45° around the x axis as compared to the orientations of the y, z axes in the parent "world" coordinate system. Accordingly, the second row of values has a 45° rotational angle indicated in the x column, and 0° rotational angles indicated for rotation around the y and z axes. Regarding scaling, the limb of FIG. 4A is seen to be doubled in length or "stretched" along the z axis (only). Thus a scale factor of 2 is indicated in the z column of the third row of the matrix [particular dimension of the matrix]; col 10, lines 43-66, at least discloses Block 723 is then entered, this block representing the storage of values from the interpolation curve in the transformation matrices of the frames being in-betweened. In particular, the selected one of the nine values in the parametric matrix described above for each joint is stored for each of the in-between frames, based on the value of such parameter component taken from the cubic interpolation curve […] It can be noted, as represented by the block 729, that the limbs associated with each joint are obtained in conjunction with the joint in each frame of the sequence, the motion of the limb being completely defined as described above, by the matrix transformation values of the associated joint; col 11, lines 9-14, at least discloses when a generated sequence of animation is displayed, the operator can modify the motion of a limb during a continuous display of the sequence […] at each frame of a sequence being displayed, the x, y and z components of a selected parameter (move, rotation or scale factor) of a selected joint are incremented by an amount which is proportional to the operator selection of x, y and z values using the joystick);
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang to incorporate the teachings of Stern, and apply the matrix transformation into the Zhang’s teachings for generating skeletal animation data representing movement of a skeleton having joints connecting bones, the skeletal animation data comprising a matrix representing a transformation of the skeleton, wherein a particular dimension of the matrix represents frames of an animation of the skeleton.
Doing so the animator has virtually complete artistic freedom and control over the resulting film; i.e. anything that is drawn can be made to move in a desired fashion so natural-looking motion can generally be achieved.
The prior art does not explicitly disclose, but McBeth discloses
a basis for runtime deformation of the mesh (McBeth- ¶0037, at least discloses Mixed reality device 104 can then attach the 3D mesh of the wearable customization at an appropriate point on the skeletal animation rig and skin the customization mesh on the rig such that, when the rig moves, the customization mesh will move with the rig and deform/change its shape in a correct manner (block 308) […] the vertices of the customization mesh will be weighted based on certain joints/bones of the skeletal animation rig so that, when those joint/bones move, the customization mesh will deform accordingly; ¶0040, at least discloses Upon presenting the rendered mesh on display 108, mixed reality device 104 can determine updated joint data for individual 110 (block 314), update the positioning/pose of the skeletal animation rig based on the updated joint data (block 316), and return to block 310 to re-render/re-display the customization mesh. Finally, device 104 can repeat blocks 310-316 in a continuous loop, which will cause the wearable customization to move/deform in real-time in display 108 in accordance with the movements of the skeletal animation rig (and thus, the movements of individual 110); Fig. 5 and ¶0051, at least disclose computing device 500 includes one or more processors).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Stern to incorporate the teachings of McBeth, and apply the deform in real-time in accordance with the movements of the skeletal animation rig into the Zhang/Stern’s teachings for outputting the skeletal animation data and the mesh binding data, the skeletal animation data and the mesh binding data providing a basis for runtime deformation of the mesh to generate an animation.
Doing so would be desirable to have features that leverage mixed reality to enrich these person-to-person interactions.
Regarding claim 2, Zhang in view of Stern and McBeth, discloses the method of claim 1, and further discloses wherein generating the skeletal animation data (see Claim 1 rejection for detailed analysis) represents the transformation from a first keyframe to a second keyframe of the animation for each of the joints (Zhang- ¶0013, at least disclose the keyframe information includes a sequence of keyframes constituting the motion of the bones, each keyframe holding a transformation matrix for each bone at that moment with respect to the parent bone coordinate system, and the position and orientation information of the bones; ¶0035, at least discloses according to the key frame sequence of the bone motion stored in the character model file, interpolation calculation is carried out between two adjacent key frames, and the new position and the new orientation of each bone at a certain moment are determined).
Regarding claim 9, Zhang discloses a method (Zhang- ¶0002, at least discloses an method for implementation skeletal skinning animation) comprising:
obtaining skeletal animation data representing movement of a skeleton having joints connecting bones (Zhang- ¶0009-0013, at least disclose A. Create a skeletal model and a mesh skin model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of the skeletons describes the structure of the character. The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] C. Based on the keyframe sequence of skeletal motion stored in the character model file, perform interpolation calculation between two adjacent keyframes to determine the new position and orientation of each bone at a certain moment […] The keyframe information includes a sequence of keyframes that constitute the bone’s motion; Fig. 1 and ¶0032, at least disclose Step S101: Create the skeletal model and a mesh skinning model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of bones describes the structure of the character. The mesh skinning model, which is the character’s skin, is a deformable mesh that changes under the influence of the skeleton; ¶0156, at least discloses the calculation can be used to obtain the interpolation matrix of the skeleton during movement, rotation, and scaling);
obtaining mesh binding data representing a binding of a mesh to the skeleton (Zhang- ¶0009-0012, at least discloses A. Create a skeletal model and a mesh skin model that constitute the character model […] The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] B. Determine the bones that affect the mesh vertices on the mesh skinning model, and determine their influence weights based on the geometric and physical relationships between the bones and the vertexes […] D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone indexes and corresponding influence weights stored in each vertex of the mesh skinning model and influence the vertex, and realize the skeletal skinning animation [binding a mesh to the skeleton to obtain mesh binding data]; Fig. 1 and ¶0037, at least disclose Step S104: Calculate the new positions and orientations of each vertex in the world coordinate system according to the bone index and corresponding influence weight stored in each vertex of the mesh skin model [binding a mesh], and realize the skeletal skinning animation); and
generating an animation of the mesh by performing deformation of the mesh based on the skeletal animation data and the mesh binding data (Zhang- ¶0009-0012, at least disclose A. Create a skeletal model and a mesh skin model that constitute the character model […] The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone indexes and corresponding influence weights stored in each vertex of the mesh skinning model and influence the vertex, and realize the skeletal skinning animation [animation of the mesh]; ¶0016, at least discloses Step A includes creating the skeletal model and the mesh skinning model that constitute the character model using either Poser or 3ds MAX software; Fig. 1 and ¶0032, at least disclose Step S101: Create the skeletal model and a mesh skinning model that constitute the character model […] The mesh skinning model, which is the character’s skin, is a deformable mesh that changes under the influence of the skeleton [deformation of the mesh]; Fig. 1 and ¶0037, at least disclose Step S104: Calculate the new positions and orientations of each vertex in the world coordinate system according to the bone index and corresponding influence weight stored in each vertex of the mesh skin model, and realize the skeletal skinning animation).
Zhang does not explicitly disclose the skeletal animation data corresponding to a matrix representing a transformation of the skeleton, wherein a particular dimension of the matrix represents frames of an animation of the skeleton; performing runtime deformation of the mesh.
However, Stern discloses
the skeletal animation data corresponding to a matrix representing a transformation of the skeleton (Stern- col 7, lines 30-37, at least discloses matrix transformations [matrix representing a transformation] are employed to transform the limb vectors in a given local coordinate system into equivalent vectors in the joint coordinate system of next highest priority. The procedure is repeated for successively higher priority joint coordinate systems until the limb vectors are expressed in terms of the world coordinate system; col 7, lines 51-65, at least discloses The matrix transformation which translates a point (x,y,z) to a new point (x',y',z') is known to be: ##EQU4## where mx, my and mz are the components of the translation or move in the x, y and z directions respectively, and the 1 in each fourth column is the homogeneous coordinate; Fig. 5 and col 9, lines 36-42, at least disclose If the figure skeletons are to be displayed, a basic orthogonal coordinate system representation (i.e., three mutually orthogonal lines), can be multiplied by the concatenated matrix transformation, so as to obtain properly oriented and scaled local coordinate axes representation at each joint position, as shown in the small sketch near block 516), wherein a particular dimension of the matrix represents frames of an animation of the skeleton (Stern- col 6, lines 26 to col 7, line 24, at least disclose FIGS. 4A and 4B are useful in understanding the manner in which the joints and limbs are represented and manipulated. In FIG. 4A, there is shown the coordinate axes for an exemplary joint, Ji, and a limb, Li, (shown in dashed line) of vectors associated with the joint. The matrix values for the joint Ji is as follows: ##EQU2## As seen in FIG. 4A, the origin of the local coordinate system for Ji is at the point (10, 10, 10) in the main or "world" coordinate system (which, in this example, is the coordinate system of next higher priority in the hierarchy). Also, the x, y, and z axes are aligned with the axes of the world coordinate system, so the relative rotation angles in the second row of the matrix are all 0°. Finally, it is assumed that, for this example, the scale factors for the frame are all unity […] Reference can now be made to FIG. 4B which represents the same joint Ji and limb Li, but with the position, orientation, and size and shape of the limb being different by virtue of modifications in the matrix values of the examplary joint. In particular, the matrix for the joint of FIG. 4B is as follows: ##EQU3## In this case, the origin of the local coordinate axes (i.e., the joint position) is seen to be at the coordinates (10, 15, 10), which correspond to the values in the first row of the matrix. Also, it is seen that the y, z axes are rotated by 45° around the x axis as compared to the orientations of the y, z axes in the parent "world" coordinate system. Accordingly, the second row of values has a 45° rotational angle indicated in the x column, and 0° rotational angles indicated for rotation around the y and z axes. Regarding scaling, the limb of FIG. 4A is seen to be doubled in length or "stretched" along the z axis (only). Thus a scale factor of 2 is indicated in the z column of the third row of the matrix [particular dimension of the matrix]; col 10, lines 43-66, at least discloses Block 723 is then entered, this block representing the storage of values from the interpolation curve in the transformation matrices of the frames being in-betweened. In particular, the selected one of the nine values in the parametric matrix described above for each joint is stored for each of the in-between frames, based on the value of such parameter component taken from the cubic interpolation curve […] It can be noted, as represented by the block 729, that the limbs associated with each joint are obtained in conjunction with the joint in each frame of the sequence, the motion of the limb being completely defined as described above, by the matrix transformation values of the associated joint; col 11, lines 9-14, at least discloses when a generated sequence of animation is displayed, the operator can modify the motion of a limb during a continuous display of the sequence […] at each frame of a sequence being displayed, the x, y and z components of a selected parameter (move, rotation or scale factor) of a selected joint are incremented by an amount which is proportional to the operator selection of x, y and z values using the joystick);
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang to incorporate the teachings of Stern, and apply the matrix transformation into the Zhang’s teachings for obtaining skeletal animation data representing movement of a skeleton having joints connecting bones, the skeletal animation data corresponding to a matrix representing a transformation of the skeleton, wherein a particular dimension of the matrix represents frames of an animation of the skeleton.
Doing so the animator has virtually complete artistic freedom and control over the resulting film; i.e. anything that is drawn can be made to move in a desired fashion so natural-looking motion can generally be achieved.
The prior art does not explicitly disclose, but McBeth discloses
performing runtime deformation of the mesh (McBeth- ¶0037, at least discloses Mixed reality device 104 can then attach the 3D mesh of the wearable customization at an appropriate point on the skeletal animation rig and skin the customization mesh on the rig such that, when the rig moves, the customization mesh will move with the rig and deform/change its shape in a correct manner (block 308) […] the vertices of the customization mesh will be weighted based on certain joints/bones of the skeletal animation rig so that, when those joint/bones move, the customization mesh will deform accordingly; ¶0040, at least discloses Upon presenting the rendered mesh on display 108, mixed reality device 104 can determine updated joint data for individual 110 (block 314), update the positioning/pose of the skeletal animation rig based on the updated joint data (block 316), and return to block 310 to re-render/re-display the customization mesh. Finally, device 104 can repeat blocks 310-316 in a continuous loop, which will cause the wearable customization to move/deform in real-time in display 108 in accordance with the movements of the skeletal animation rig (and thus, the movements of individual 110); Fig. 5 and ¶0051, at least disclose computing device 500 includes one or more processors).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Stern to incorporate the teachings of McBeth, and apply the deform in real-time in accordance with the movements of the skeletal animation rig into the Zhang/Stern’s teachings for generating an animation by performing runtime deformation of the mesh based on the skeletal animation data and the mesh binding data.
Doing so would be desirable to have features that leverage mixed reality to enrich these person-to-person interactions.
Regarding claim 10, Zhang in view of Stern and McBeth, discloses the method of claim 9, and further discloses wherein generating the animation comprises:
offsetting vertex positions of the mesh based on the skeletal animation data (McBeth- ¶0043, at least discloses At block 404, mixed reality device 104 can place the floating customization at an offset from the point determined at block 402, where the offset is defined as part of the customization. For example, a floating animal familiar may be positioned slightly down and to the left of the individual's head, while a halo may be positioned directly above the individual's head).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Stern to incorporate the teachings of McBeth, and apply the offset into the Zhang/Stern’s teachings for offsetting vertex positions of the mesh based on the skeletal animation data.
The same motivation that was utilized in the rejection of claim 9 applies equally to this claim.
Regarding claim 11, Zhang in view of Stern and McBeth, discloses the method of claim 9, wherein the skeletal animation data comprises textures (see Claim 3 rejection for detailed analysis), and further discloses each texture having rows representing individual frames of the animation and columns representing individual bones of the skeleton (see Claim 3 rejection for detailed analysis).
Regarding claim 12, Zhang in view of Stern and McBeth, discloses the method of claim 11, and further discloses wherein the mesh binding data represents influence of individual joints on corresponding vertices of the mesh (see Claim 4 rejection for detailed analysis).
Regarding claim 15, Zhang in view of Stern and McBeth, discloses the method of claim 9, and further discloses the method performed on a graphics processing unit (Zhang- ¶0164, at least discloses the new matrix is passed to the GPU).
7. Claims 16-17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Stern, further in view of McBeth, still further in view of Li et al., (“Li”) [US-2019/0206145-A1]
Regarding claim 16, Zhang discloses a system (Zhang- ¶0033, at least discloses 3ds MAX is a three-dimensional animation rendering and production software developed by Discreet corporation (later incorporated by Autodesk corporation) based on PC systems) comprising:
a graphics processing unit (Zhang- ¶0164, at least discloses a GPU); and
wherein the processing unit is configured to:
obtain skeletal animation data representing movement of a skeleton having joints connecting bones (see Claim 9 rejection for detailed analysis);
obtain mesh binding data representing a binding of a mesh to the skeleton (see Claim 9 rejection for detailed analysis); and
transfer the skeletal animation data and the mesh binding data to the graphics processing unit (Zhang- ¶0163-0164, at least discloses calculating the world matrix of each bone at any given time, we can calculate the world matrix of the parent skeleton, and then calculate the world matrix of the child bone. The MESH file stores this vertex information, i.e., MESH skin information, including vertex position, normal vectors, texture material, and the bones that affect the vertex and their impact weights […] Once the new matrix is obtained, the new matrix for each bone can be calculated sequentially. The new matrix is passed to GPU as a bone array, and the new position and orientation of the vertex in the world coordinate are calculated based on the weights of the bones that affect the vertex), and wherein the graphics processing unit is configured to:
receive the skeletal animation data and the mesh binding data (Zhang- ¶0009, at least discloses A. Create a skeletal model and a mesh skin model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of the skeletons describes the structure of the character. The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones; Fig. 1 and ¶0032, at least disclose Step S101: Create the skeletal model and a mesh skinning model that constitute the character model. The skeletal model consists of several bones organized in a certain hierarchy, and the hierarchy of bones describes the structure of the character. The mesh skinning model, which is the character’s skin, is a deformable mesh that changes under the influence of the skeleton) ; and
generate an animation of the mesh by performing deformation of the mesh based on the skeletal animation data and the mesh binding data (see Claim 9 rejection for detailed analysis).
Zhang does not explicitly disclose a central processing unit; the skeletal animation data corresponding to a matrix representing a transformation of the skeleton, wherein a particular dimension of the matrix represents frames of an animation of the skeleton; performing runtime deformation of the mesh.
However, Stern discloses
the skeletal animation data corresponding to a matrix representing a transformation of the skeleton (see Claim 9 rejection for detailed analysis), wherein a particular dimension of the matrix represents frames of an animation of the skeleton (see Claim 9 rejection for detailed analysis);
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang to incorporate the teachings of Stern, and apply the matrix transformation into the Zhang’s teachings for obtaining skeletal animation data representing movement of a skeleton having joints connecting bones, the skeletal animation data corresponding to a matrix representing a transformation of the skeleton, wherein a particular dimension of the matrix represents frames of an animation of the skeleton.
Doing so the animator has virtually complete artistic freedom and control over the resulting film; i.e. anything that is drawn can be made to move in a desired fashion so natural-looking motion can generally be achieved.
The prior art does not explicitly disclose, but McBeth discloses
performing runtime deformation of the mesh (see Claim 9 rejection for detailed analysis).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Stern to incorporate the teachings of McBeth, and apply the deform in real-time in accordance with the movements of the skeletal animation rig into the Zhang/Stern’s teachings in order to generate an animation by performing runtime deformation of the mesh based on the skeletal animation data and the mesh binding data.
Doing so would be desirable to have features that leverage mixed reality to enrich these person-to-person interactions.
The prior art does not explicitly disclose, but Li disclosed
a central processing unit (Li- Fig. 28B and ¶0280, at least disclose a skeleton animation scenario, and is completed through interaction between a CPU and a GPU in a terminal shown in FIG. 1);
receive the skeletal animation data and the mesh binding data from the central processing unit (Li- Fig. 28B and ¶0280-0302, at least disclose 501: The CPU loads model data of an avatar accessory. The model data of the avatar accessory may include network data (including vertex data) of a first mesh model and skeleton information of a first virtual framework. Each mesh in a mesh model is generally of a triangle or another polygon. The mesh data includes the vertex data (a vertex table) and index data […] The skeleton information includes the number of all skeletons in the first virtual framework and specific information of each skeleton […] 502: The CPU obtains model data of a character (that is, a virtual object). The model data of the character may include skeleton information of a second virtual framework, vertex data of a second mesh model, and mounting correction information of the virtual accessory for the virtual object. For the skeleton information and the vertex data, refer to the descriptions in 501, and details are not described herein again. The mounting correction information may exist as an additional node (a mounting point) […] One mounting point may be bound to one or more skeletons. There is a parent-child relationship between a mounting point and a skeleton to which the mounting point is bound, and the mounting point is a child node. The parent-child relationship is similar to a parent-child between skeletons).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Stern/McBeth to incorporate the teachings of Li, and apply the CPU and GPU into the Zhang/Stern/McBeth’s teachings in order to receive the skeletal animation data and the mesh binding data from the central processing unit.
Doing so would harmoniously display a simulation object composited with an accessory, so that a user has desirable experience and visual enjoyment.
Li further disclosed
transfer the skeletal animation data and the mesh binding data to the graphics processing unit (Li- Fig. 28A and ¶0311-0321, at least disclose in an animation key frame, a transform matrix of each skeleton in the key frame relative to a coordinate system of a parent skeleton may be indicated. Alternatively, the animation key frame may record rotation, translation, and zoom of each joint in a joint state relative to a bound pose […] a transform matrix that corresponds to the jth skeleton in the second virtual framework in the current frame and that is in the world space is indicated by using matrix[j] […] The CPU transmits shading-related data of the character and the avatar accessory to a GPU […] the shading-related data may include vertex data of the character and the avatar accessory, matrix[i], matrix[j]r, vertex texture data of the first mesh model, and vertex texture data of the second mesh model. During actual running, the foregoing data may be transmitted in batches. For example, the vertex data of the character, matrix[i], and the vertex texture data of the second mesh model may be first transmitted, and then the vertex data of the avatar accessory, matrix[j]t, and the vertex texture data of the first mesh model are transmitted).
Regarding claim 17, Zhang in view of Stern, McBeth and Li, discloses the system of claim 16, and further discloses wherein the central processing unit (see Claim 16 rejection for detailed analysis) is configured to:
instruct the graphics processing unit (see Claim 16 rejection for detailed analysis) to generate multiple parallel animations of a particular character according to the skeletal animation data and the mesh binding data (McBeth- ¶0020, at least discloses This is particularly useful in social applications where such personal expression in an inherent part of the experience, as well as in video games where users may want to assume the identities of in-game characters; ¶0034, at least discloses Starting with block 302, mixed reality device 104 can determine real-time joint data for individual 110, where the joints are points on individual 110's body that map to corresponding joints on a humanoid skeletal animation rig. This real-time joint data indicates the current location and pose of individual 110's body […] The skeletal animation rig, which comprises joints/bones and weights that define how the movement of one joint/bone affects other joints/bones, can be predefined by an animator or may be generated by the skeletal tracking software itself; ¶0059, at least discloses Steps described as sequential may be executed in parallel; Li- ¶0005, at least discloses the skin of the simulation object may change when the sizes of the skeletal model and the avatar are simultaneously changed).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang to incorporate the teachings of Stern, McBeth and Li, and apply the humanoid skeletal animation rig and the simulation object into the Zhang’s teachings in order to instruct the graphics processing unit to generate multiple parallel animations of a particular character according to the skeletal animation data and the mesh binding data.
The same motivation that was utilized in the rejection of claim 16 applies equally to this claim.
Regarding claim 20, Zhang in view of Stern, McBeth and Li, discloses the system of claim 16, and further discloses wherein the central processing unit (see Claim 16 rejection for detailed analysis) is configured to:
instruct the graphics processing unit (see Claim 16 rejection for detailed analysis) to perform multiple other animations on a per-joint basis in response to received user inputs (Zhang- ¶0037-0038, at least disclose step S104, calculating new positions and new orientations of the vertexes under the world coordinate system according to the bone indexes influencing the vertexes and the corresponding influence weights stored in the vertexes on the mesh skin model, and realizing the bone skin animation […] the vertices can then be rendered as animations by simply attaching them to the corresponding bones; Li- ¶0322-0336, at least disclose each process of preparing data and instructing the GPU to perform shading is referred to as a draw call […] 508: Calculate a coordinate value of each mesh vertex in the second mesh model in the current time based on the transform matrix set of the second virtual framework in the current time. For a mesh vertex s in the second mesh model, a coordinate value thereof may be calculated).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Stern/McBeth to incorporate the teachings of Li, and apply the GPU into the Zhang/Stern/McBeth’s teachings in order to instruct the graphics processing unit to perform multiple other animations on a per-joint basis in response to received user inputs.
The same motivation that was utilized in the rejection of claim 16 applies equally to this claim.
8. Claims 3-6 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Stern, further in view of McBeth, further in view of Jiang et al. (machine translation of CN-111738935-A with citation below, hereinafter “Jiang”)
Regarding claim 3, Zhang in view of Stern and McBeth, discloses the method of claim 2, and further disclosed wherein the skeletal animation data (see Claim 1 rejection for detailed analysis) comprises textures (Zhang- ¶0013, at least discloses Each vertex includes its position, normal vector, texture material, and the bones affecting that vertex and their influence weights).
The prior art does not explicitly disclose, but Jiang discloses
each texture comprising rows representing frames of the animation and columns representing individual bones (Jiang- ¶0009, at least disclose Store all animation frame data of the animation segment that needs to generate afterimages in a texture. Then, in the Vertex Shader (VS), retrieve the bone transformation matrix information of the corresponding animation frame from the texture based on the recorded frame number of the afterimage animation and perform skinning calculations. Here, the animation frame data refers to the set of matrices storing the rotation, translation, and pose data of all bones at a certain moment when playing the skeletal animation. The texture is equivalent to a two-dimensional array, and a floating-point number can be uniquely identified by the row index and column index; ¶0053-0054, at least disclose The pose data here refers to the transformation matrix of each bone. That is, the transformation matrix of each bone represents the absolute transformation matrix relative to world space, not the transformation matrix relative to the parent bone. The bone transformation matrix is generally a 4x4 square matrix. However, since the 4th row is fixed at (0,0,0,1), only the first three rows are stored during data storage. These three rows correspond to three pixels in the pose graph (each pixel can store four floating-point numbers). The motion data for each frame contains the transformation matrix of all bones in the skeleton. One or more rows of pixels in the pose graph (related to the number of bones and the resolution of the pose graph) can be used to store the transformation matrix data of all bones in a frame. The transformation matrix data of other frames are also stored sequentially in the pose graph).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Stern/McBeth to incorporate the teachings of Jiang, and apply the texture is equivalent to a two-dimensional array with the row index and column index into the Zhang/Stern/McBeth’s teachings in order the skeletal animation data comprises textures having rows representing frames and columns representing individual bones.
Doing so would solve the problem of slow animation creation equipment operation speed due to low skinning calculation efficiency in related technologies.
Regarding claim 4, Zhang in view of Stern, McBeth and Jiang, discloses the method of claim 3, and further disclosed wherein the mesh binding data represents influence of individual joints on corresponding vertices of the mesh (Zhang- ¶0010-0012, at least disclose B. Determine the bones that affect the mesh vertices on the mesh skinning model, and determine their influence weights based on the geometric and physical relationships between the bones and the vertices […] D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone index and corresponding influence weight stored in each vertex of the mesh skinning model, and realize the skeletal skinning animation; ¶0037, at least discloses Step S104: Calculate the new position and orientation of each vertex in the world coordinate system according to the bone index and corresponding influence weight stored in each vertex of the mesh skinning model, and realize the skeletal skinning animation).
Regarding claim 5, Zhang in view of Stern, McBeth and Jiang, discloses the method of claim 4, and further disclosed wherein the mesh binding data comprises a joint influence index array (Zhang- ¶0012-0013, at least disclose D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone index and corresponding influence weight stored in each vertex of the mesh skinning model, and realize the skeletal skinning animation […] Each vertex includes its position, normal vector, texture material, and the bones affecting that vertex and their influence weights. The bone information includes all the bones that make up the bone model. All bones are organized into a tree according to parent-child relationships, with the root representing the entire skeleton. Each node, including leaf nodes, represents a bone. Each bone includes its transformation matrix in the parent bone's coordinate system, which determines its position within the parent bone's coordinate system. The keyframe information includes a sequence of keyframes that constitute the bone's motion. Each keyframe stores the transformation matrix of each bone relative to the parent bone's coordinate system at that moment, as well as the bone's position and orientation information; ¶0015, at least discloses Step D specifically includes: according to the bone index and corresponding influence weight stored in each vertex of the mesh skinning model, based on the formula: new position of vertex = world matrix of each bone * inverse matrix of Tpos matrix of each bone, world matrix of each bone = world matrix of Tpos matrix of each bone * interpolated matrix * world matrix of parent bone, the accurate position of vertex is obtained by average weighting according to which bones affect the vertex, thus realizing bone skinning animation; ¶0164, at least discloses The new matrix is passed to the GPU as a bone array, and the new position and orientation of the vertex in world coordinates are calculated based on the weights of the bones that affect the vertex; Jiang- ¶0009, at least discloses The texture is equivalent to a two-dimensional array, and a floating-point number can be uniquely identified by the row index and column index).
Regarding claim 6, Zhang in view of Stern, McBeth and Jiang, discloses the method of claim 5, and discloses the method further comprising:
binding a second mesh to the skeleton to obtain second mesh binding data (Zhang- ¶0009-0013, at least disclose A. Create a skeletal model and a mesh skin model that constitute the character model […] The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone indexes and corresponding influence weights stored in each vertex of the mesh skinning model and influence the vertex, and realize the skeletal skinning animation […] The mesh skinning information is the character's polygonal model, composed of triangular faces. Each triangle has three indices pointing to the model's vertex table, which identify the three vertices; ¶0038, at least discloses by attaching each vertex to the corresponding bone, it can be rendered into an animation); and
outputting the second mesh binding data, the skeletal animation data and the second mesh binding data providing a basis for runtime deformation of the second mesh resulting in a second animation (Zhang- ¶0009-0013, at least disclose A. Create a skeletal model and a mesh skin model that constitute the character model […] The mesh skinning model is the skin of the character, which is a deformable mesh that changes under the influence of bones […] D. Calculate the new position and orientation of each vertex in the world coordinate system based on the bone indexes and corresponding influence weights stored in each vertex of the mesh skinning model and influence the vertex, and realize the skeletal skinning animation […] The mesh skinning information is the character's polygonal model, composed of triangular faces. Each triangle has three indices pointing to the model's vertex table, which identify the three vertices; ¶0016, at least discloses Step A includes creating the skeletal model and the mesh skinning model that constitute the character model using either Poser or 3ds MAX software; Fig. 1 and ¶0032, at least disclose Step S101: Create the skeletal model and a mesh skinning model that constitute the character model […] The mesh skinning model, which is the character’s skin, is a deformable mesh that changes under the influence of the skeleton; ¶0038, at least discloses the vertex of the model is defined in the model coordinate system, and for the game engine, the vertex position in the model coordinate system is converted into the world coordinate system for rendering […] by attaching each vertex to the corresponding bone, it can be rendered into an animation; McBeth- ¶0037, at least discloses Mixed reality device 104 can then attach the 3D mesh of the wearable customization at an appropriate point on the skeletal animation rig and skin the customization mesh on the rig such that, when the rig moves, the customization mesh will move with the rig and deform/change its shape in a correct manner (block 308) […] the vertices of the customization mesh will be weighted based on certain joints/bones of the skeletal animation rig so that, when those joint/bones move, the customization mesh will deform accordingly; ¶0040, at least discloses Upon presenting the rendered mesh on display 108, mixed reality device 104 can determine updated joint data for individual 110 (block 314), update the positioning/pose of the skeletal animation rig based on the updated joint data (block 316), and return to block 310 to re-render/re-display the customization mesh. Finally, device 104 can repeat blocks 310-316 in a continuous loop, which will cause the wearable customization to move/deform in real-time in display 108 in accordance with the movements of the skeletal animation rig (and thus, the movements of individual 110)).
Regarding claim 13, Zhang in view of Stern and McBeth, discloses the method of claim 11, and discloses the method further comprising:
selectively applying the skeletal animation data based on input received at runtime (McBeth- ¶0037, at least discloses Mixed reality device 104 can then attach the 3D mesh of the wearable customization at an appropriate point on the skeletal animation rig and skin the customization mesh on the rig such that, when the rig moves, the customization mesh will move with the rig and deform/change its shape in a correct manner (block 308) […] the vertices of the customization mesh will be weighted based on certain joints/bones of the skeletal animation rig so that, when those joint/bones move, the customization mesh will deform accordingly; ¶0040, at least discloses Upon presenting the rendered mesh on display 108, mixed reality device 104 can determine updated joint data for individual 110 (block 314), update the positioning/pose of the skeletal animation rig based on the updated joint data (block 316), and return to block 310 to re-render/re-display the customization mesh. Finally, device 104 can repeat blocks 310-316 in a continuous loop, which will cause the wearable customization to move/deform in real-time in display 108 in accordance with the movements of the skeletal animation rig (and thus, the movements of individual 110).
The prior art does not explicitly disclose a subset of the rows of the skeletal animation data based on input received at runtime, resulting in replaying a subset of the animation of the mesh represented by the subset of rows.
However, Jiang discloses
a subset of the rows of the skeletal animation data based on input received at runtime, resulting in replaying a subset of the animation of the mesh represented by the subset of rows (Jiang- ¶0053-0054, at least disclose The pose data here refers to the transformation matrix of each bone. That is, the transformation matrix of each bone represents the absolute transformation matrix relative to world space, not the transformation matrix relative to the parent bone. The bone transformation matrix is generally a 4x4 square matrix. However, since the 4th row is fixed at (0,0,0,1), only the first three rows are stored during data storage. These three rows correspond to three pixels in the pose graph (each pixel can store four floating-point numbers). The motion data for each frame contains the transformation matrix of all bones in the skeleton. One or more rows of pixels in the pose graph (related to the number of bones and the resolution of the pose graph) can be used to store the transformation matrix data of all bones in a frame).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Stern/McBeth to incorporate the teachings of Jiang, and apply moving in real-time and one or more rows of pixels related to the number of bones into the Zhang/Stern/McBeth’s teachings for selectively applying a subset of the rows of the skeletal animation data based on input received at runtime.
Doing so would be desirable to have features that leverage mixed reality to enrich these person-to-person interactions.
Regarding claim 14, Zhang in view of Stern and McBeth, discloses the method of claim 11, and discloses the method further comprising: selectively applying of the skeletal animation data based on input received at runtime (McBeth- ¶0037, at least discloses Mixed reality device 104 can then attach the 3D mesh of the wearable customization at an appropriate point on the skeletal animation rig and skin the customization mesh on the rig such that, when the rig moves, the customization mesh will move with the rig and deform/change its shape in a correct manner (block 308) […] the vertices of the customization mesh will be weighted based on certain joints/bones of the skeletal animation rig so that, when those joint/bones move, the customization mesh will deform accordingly; ¶0040, at least discloses Upon presenting the rendered mesh on display 108, mixed reality device 104 can determine updated joint data for individual 110 (block 314), update the positioning/pose of the skeletal animation rig based on the updated joint data (block 316), and return to block 310 to re-render/re-display the customization mesh. Finally, device 104 can repeat blocks 310-316 in a continuous loop, which will cause the wearable customization to move/deform in real-time in display 108 in accordance with the movements of the skeletal animation rig (and thus, the movements of individual 110)).
The prior art does not explicitly disclose selectively applying a subset of the columns of the skeletal animation data based on input received at runtime, resulting in movement of a portion of a character corresponding to a subset of the bones represented by the subset of the columns.
However, Jiang discloses
a subset of the columns of the skeletal animation data based on input received at runtime, resulting in movement of a portion of a character corresponding to a subset of the bones represented by the subset of the columns (Jiang- ¶0009, at least discloses Store all animation frame data of the animation segment that needs to generate afterimages in a texture. Then, in the Vertex Shader (VS), retrieve the bone transformation matrix information of the corresponding animation frame from the texture based on the recorded frame number of the afterimage animation and perform skinning calculations. Here, the animation frame data refers to the set of matrices storing the rotation, translation, and pose data of all bones at a certain moment when playing the skeletal animation. The texture is equivalent to a two-dimensional array, and a floating-point number can be uniquely identified by the row index and column index; Jiang- ¶0053-0054, at least disclose The pose data here refers to the transformation matrix of each bone. That is, the transformation matrix of each bone represents the absolute transformation matrix relative to world space, not the transformation matrix relative to the parent bone. The bone transformation matrix is generally a 4x4 square matrix. However, since the 4th row is fixed at (0,0,0,1), only the first three rows are stored during data storage).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Stern/McBeth to incorporate the teachings of Jiang, and apply column index into the Zhang/Stern/McBeth’s teachings for selectively applying a subset of the columns of the skeletal animation data based on input received at runtime, resulting in movement of a portion of a character corresponding to a subset of the bones represented by the subset of the columns.
Doing so would be desirable to have features that leverage mixed reality to enrich these person-to-person interactions.
9. Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Stern, further in view of McBeth, still further in view of Jiang et al. (machine translation of CN-111738935-A with citation below, hereinafter “Jiang”), still further in view of Nimura et al. (“Nimura”) [US-2003/0043154-A1]
Regarding claim 21, Zhang in view of Stern and McBeth, discloses the method of claim 11, and discloses the method further comprising:
applying a first subset of the rows of the textures (Jiang- ¶0053-0054, at least disclose The pose data here refers to the transformation matrix of each bone. That is, the transformation matrix of each bone represents the absolute transformation matrix relative to world space, not the transformation matrix relative to the parent bone. The bone transformation matrix is generally a 4x4 square matrix. However, since the 4th row is fixed at (0,0,0,1), only the first three rows are stored during data storage. These three rows correspond to three pixels in the pose graph (each pixel can store four floating-point numbers). The motion data for each frame contains the transformation matrix of all bones in the skeleton. One or more rows of pixels in the pose graph (related to the number of bones and the resolution of the pose graph) can be used to store the transformation matrix data of all bones in a frame); and
applying a second subset of the rows of the texturestransformation matrix of each bone. That is, the transformation matrix of each bone represents the absolute transformation matrix relative to world space, not the transformation matrix relative to the parent bone. The bone transformation matrix is generally a 4x4 square matrix. However, since the 4th row is fixed at (0,0,0,1), only the first three rows are stored during data storage. These three rows correspond to three pixels in the pose graph (each pixel can store four floating-point numbers). The motion data for each frame contains the transformation matrix of all bones in the skeleton. One or more rows of pixels in the pose graph (related to the number of bones and the resolution of the pose graph) can be used to store the transformation matrix data of all bones in a frame).
The prior art does not explicitly disclose responsive to receiving first input indicating a first button has been pushed on a video game controller, replaying a first subset of the animation involving movement of part of a character through a first range of motion by applying a first subset of the rows of the textures; and responsive to receiving second input indicating a second button has been pushed on the video game controller, replaying a second subset of
the animation involving movement of the part of the character through a second range of motion by applying a second subset of the rows of the textures, the second range of motion being different from the first range of motion.
However, Nimura discloses
responsive to receiving first input indicating a first button has been pushed on a video game controller (Nimura- Fig. 1 and ¶0089, at least disclose the motion processing section 112 comprises a first motion control section 114 (a main-side motion control section), a second motion control section 116 (a sub-side motion control section), and a motion blend section 118; ¶0205, at least discloses With this standing motion script, standing motion data is first selected (step S 1). A motion script switching condition is then determined, based on manipulation input (manipulation data from operating buttons or joystick) and status information for the model object (the current situation of the model object) (step S2); ¶0259, at least discloses Manipulation data from a game controller 942 (such as a joystick, buttons, casing, or a bat-shaped or handgun-shaped controller)), replaying a first subset of the animation involving movement of part of a character through a first range of motion (Nimura- ¶0050, at least discloses selecting motion data for a model object and controlling motion of the model object by a plurality of motion control sections [range of motion], based on motion scripts in which are defined the motion data to be selected and motion script switching conditions; ¶0054-0055, at least disclose the plurality of motion control sections to operate in parallel, enabling a motion blend of a plurality of sets of motion data selected by this plurality of motion control sections. It is therefore possible to implement motion control by kind of a motion script obtained by combining first motion scripts of the first motion control section [first range of motion] and second motion scripts of the second motion control section, by way of example. This enables the implementation of the representation of a wide range of motion scripts with a small amount of data […] Note that the motion data selected by each of the first and second motion control sections could be obtained by blending a plurality of sets of motion data; ¶0058, at least discloses first group of the motion scripts may be allocated to a first motion control section of the plurality of motion control sections; ¶0061, at least discloses This would make it possible for the first motion control section to operate in parallel with the second motion control section, even after the second motion control section has started operating. This therefore makes it possible to represent motions such that a first motion is played (replayed) by the first motion section, then a blend of first and second motions is played by the first and second motion control sections, and finally the motion play returns to the first motion of the first motion control section;); and
responsive to receiving second input indicating a second button has been pushed on the video game controller (Nimura- Fig. 1 and ¶0089, at least disclose the motion processing section 112 comprises a first motion control section 114 (a main-side motion control section), a second motion control section 116 (a sub-side motion control section), and a motion blend section 118; ¶0205, at least discloses With this standing motion script, standing motion data is first selected (step S 1). A motion script switching condition is then determined, based on manipulation input (manipulation data from operating buttons or joystick) and status information for the model object (the current situation of the model object) (step S2); ); ¶0259, at least discloses Manipulation data from a game controller 942 (such as a joystick, buttons, casing, or a bat-shaped or handgun-shaped controller)), replaying a second subset of the animation involving movement of the part of the character through a second range of motion controlling motion of the model object by a plurality of motion control sections [range of motion], based on motion scripts in which are defined the motion data to be selected and motion script switching conditions; ¶0054-0055, at least disclose the plurality of motion control sections to operate in parallel, enabling a motion blend of a plurality of sets of motion data selected by this plurality of motion control sections. It is therefore possible to implement motion control by kind of a motion script obtained by combining first motion scripts of the first motion control section [first range of motion] and second motion scripts of the second motion control section, by way of example. This enables the implementation of the representation of a wide range of motion scripts with a small amount of data […] Note that the motion data selected by each of the first and second motion control sections could be obtained by blending a plurality of sets of motion data; ¶0058, at least discloses a second group of the motion scripts may be allocated to a second motion control section of the motion control sections; ¶0061, at least discloses This would make it possible for the first motion control section to operate in parallel with the second motion control section, even after the second motion control section has started operating. This therefore makes it possible to represent motions such that a first motion is played (replayed) by the first motion section, then a blend of first and second motions is played by the first and second motion control sections, and finally the motion play returns to the first motion of the first motion control section),
the second range of motion being different from the first range of motion (Nimura- ¶0089-0090, at least disclose the motion processing section 112 comprises a first motion control section 114 (a main-side motion control section), a second motion control section 116 (a sub-side motion control section), and a motion blend section 118. Note that the configuration could be such that three or more motion control sections are provided […] In this case, the first and second motion control sections 114 and 116 select motion data for a model object, based on a motion script (a script that defines a processing sequence for motion control) stored in a motion script storage section 178; ¶0097, at least discloses The blend data that is the objective of the blending done by the motion blend section 118 is selected by the first and second motion control sections 114 and 116. In this case, the first and second motion control sections 114 and 116 could select the motion data created by the blend processing of the motion blend section 118, and the thus-selected motion data could then be blended again by the motion blend section 118; Fig. 5 and ¶0139, at least discloses A series of transitional motions of the model object (the motion data MA) is shown at C 1 to C5 in FIG. 5. Similarly, a series of attack motions of the model object (the motion data MB) is shown at D1 to D5 of FIG. 5.; Also see fig. 8).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Stern/McBeth/Jiang to incorporate the teachings of Nimura, and apply first motion control section, a second motion control section into the Zhang/Stern/McBeth/Jiang’s teachings in responsive to receiving first input indicating a first button has been pushed on a video game controller, replaying a first subset of the animation involving movement of part of a character through a first range of motion by applying a first subset of the rows of the textures; and responsive to receiving second input indicating a second button has been pushed on the video game controller, replaying a second subset of the animation involving movement of the part of the character through a second range of motion by applying a second subset of the rows of the textures, the second range of motion being different from the first range of motion.
Doing so would enable players to enjoy a fighting game, as an example, each player uses a game controller (manipulation section) to manipulate his or her own character, to enjoy a game of combat against an enemy character manipulated by another player or a computer.
10. Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Stern, further in view of McBeth, still further in view of Jiang, still further in view of Wu et al., (“Wu”) [US-2024/0307779-A1]
Regarding claim 7, Zhang in view of Stern, McBeth and Jiang, discloses the method of claim 6, and the prior art does not explicitly disclose, but Wu discloses the mesh and the second mesh representing different characters that share the skeleton (Wu- ¶0028, at least discloses Animation Retargeting: it is a function that allows animation to be reused between virtual character models that share the same skeleton framework resource but have greatly different proportions. The retargeting can prevent a skeleton framework that generates the animation from losing a proportion or deforming unnecessarily when using animations from virtual character models with different profiles Fig. 3 and ¶0062-0063, at least disclose virtual human models share a set of calculation manners, virtual reptile models share another set of calculation manners, and a position of a foot key point is obtained by a foot joint of the human skeleton framework through the preset calculation manner […] ) In addition, since in a model space, all skeleton joints or key point positions share a coordinate system of this space, and in a joint space, a coordinate system is established on a parent joint and a child joint position coordinate depends on the parent joint, the position data of the skeleton framework end is defined in the model space rather than the joint space).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Stern/McBeth/Jiang to incorporate the teachings of Wu, and apply the virtual character models that share the same skeleton framework resource into the Zhang/Stern/McBeth/Jiang’s teachings in order the animation and the another animation representing different characters that share the skeleton.
Doing so would prevent a skeleton framework that generates the animation from losing a proportion or deforming unnecessarily when using animations from virtual character models with different profiles.
Regarding claim 8, Zhang in view of Stern, McBeth and Jiang, discloses the method of claim 6, and the prior art does not explicitly disclose, but Wu discloses the mesh and the second mesh representing a particular character at different levels of detail (Wu- ¶0028, at least discloses Animation Retargeting: it is a function that allows animation to be reused between virtual character models that share the same skeleton framework resource but have greatly different proportions. The retargeting can prevent a skeleton framework that generates the animation from losing a proportion or deforming unnecessarily when using animations from virtual character models with different profiles Fig. 3 and ¶0062-0063, at least disclose virtual human models share a set of calculation manners, virtual reptile models share another set of calculation manners, and a position of a foot key point is obtained by a foot joint of the human skeleton framework through the preset calculation manner […] ) In addition, since in a model space, all skeleton joints or key point positions share a coordinate system of this space, and in a joint space, a coordinate system is established on a parent joint and a child joint position coordinate depends on the parent joint, the position data of the skeleton framework end is defined in the model space rather than the joint space; ¶0073-0075, at least disclose before skinning the object skeleton of the first afterimage model based on the reference pose information to obtain the target afterimage object, multiple LOD models can be obtained, wherein each LOD model contains a different number of triangles; the target LOD model among the multiple LOD models is determined as the first afterimage model, wherein the target LOD model is the LOD model with the fewest number of triangles among the multiple LOD models […] A Levels of Detail (LOD) model contains multiple images with different levels of detail. The level of detail refers to the number of vertices and triangles.).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Stern/McBeth/Jiang to incorporate the teachings of Wu, and apply the multiple images with different levels of detail into the Zhang/Stern/McBeth/Jiang’s teachings in order the animation and the another animation representing a particular character at different levels of detail.
Doing so would prevent a skeleton framework that generates the animation from losing a proportion or deforming unnecessarily when using animations from virtual character models with different profiles.
11. Claims 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Stern, further in view of McBeth, still further in view of Li et al., (“Li”) [US-2019/0206145-A1], still further in view of Horsman et al., (“Horsman”) [US-2020/0035021-A1]
Regarding claim 18, Zhang in view of Stern, McBeth and Li, discloses the system of claim 16, and further discloses wherein the skeletal animation data includes one or more textures (Zhang- ¶0013, at least discloses Each vertex includes its position, normal vector, texture material, and the bones affecting that vertex and their influence weights), and the prior art does not explicitly disclose, but Horsman discloses wherein the mesh binding data (see Claim 16 rejection for detailed analysis) includes one or more UV maps (Horsman- ¶0012, at least discloses receiving a sequence of UV mapped meshes of a humanoid character comprising different camera positions, stabilizing a topology of the sequence of UV mapped meshes, wherein the stabilizing results in a stabilized mesh, and synthesizing a group of stabilized meshes comprising the stabilized mesh, wherein the synthesizing results in a synthesized topology texture map sequence; ¶0060, at least discloses a mesh and MVE can be used to generate a mesh with UV's and texture. Processing Service 150 can further perform Skin Binding Generation 162. Skin Binding Generation 162 can bind a skin or a surface texture of a human, humanoid rig, or other object to a skeletonization associated with the human, humanoid rig, or object; ¶0067, at least discloses mesh: with UV mapping or with an MVE, stabilized mesh (sequence of meshes with stable UVs/vertex-face relationships), texture sequence (png), SSDR Sequence, etc; ¶0072, at least discloses Keyframes can comprise a mesh vertex, normal and triangle index stream, SSDR bone bindings/initial pose, and a humanoid bone binding/initial pose […] A humanoid bone binding exists for each keyframe and assumes the mesh has already been transformed into the pose for frame n, and that is then used to transform the vertex again based on the dynamic humanoid skeleton pose for the currently displayed frame; Fig. 9 and ¶0080, at least discloses a mesh of a sequence of UV mapped meshes can comprise different topologies and UV layouts).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Zhang/Stern/McBeth/Li to incorporate the teachings of Horsman, and apply the UV mapped meshes of a humanoid character into the Zhang/Stern/McBeth/Li’s teachings in order the mesh binding data includes one or more UV maps.
Doing so would complete end-to-end experience involving capture of a volumetric performance and ultimate distribution to consumers of that volumetric performance.
Regarding claim 19, Zhang in view of McBeth, Li and Horsman, discloses the system of claim 18, and further discloses wherein the graphics processing unit (see Claim 16 rejection for detailed analysis) is configured to: offset vertex positions of the mesh (McBeth- ¶0043, at least discloses At block 404, mixed reality device 104 can place the floating customization at an offset from the point determined at block 402, where the offset is defined as part of the customization. For example, a floating animal familiar may be positioned slightly down and to the left of the individual's head, while a halo may be positioned directly above the individual's head) based on the one or more textures (Zhang- ¶0013, at least discloses Each vertex includes its position, normal vector, texture material, and the bones affecting that vertex and their influence weights) and the one or more UV maps to generate the animation (Horsman- ¶0012, at least discloses receiving a sequence of UV mapped meshes of a humanoid character comprising different camera positions, stabilizing a topology of the sequence of UV mapped meshes, wherein the stabilizing results in a stabilized mesh, and synthesizing a group of stabilized meshes comprising the stabilized mesh, wherein the synthesizing results in a synthesized topology texture map sequence).
Conclusion
12. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
13. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL LE whose telephone number is (571)272-5330. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL LE/Primary Examiner, Art Unit 2614
Prosecution Insights