DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on September 12th, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
This is in response to applicant’s amendment/response filed on 07/15/2026 which have been entered and made of record.
Applicant’s arguments regarding claim rejections under 35 U.S.C. 112(b) have been fully considered and are persuasive. Previous 112(b) rejections for claims 1-20 have been withdrawn.
Applicant’s arguments regarding claim rejections under 35 U.S.C. 103 have been fully considered but they are not persuasive.
Applicant argues As acknowledged in the Office action regarding original Claim 8, Han and Peters fail to disclose or suggest motion information of the dynamic 3D avatar data of Claim 1. Naka merely discloses motion data per se. While Naka may separately disclose certain types of motion data, it does so in a completely different context. Naka fails to provide any teaching on how such motion data interacts with volumetric data, particularly regarding data segmentation or adaptive division for transmission efficiency.
Even if Han, Peters, and Naka can be combined by a skilled person in the pertinent art, the assumed combination of cited references fails to disclose or suggest a specific relationship between the first data element including the volumetric data and the second data element including the motion information of the dynamic 3D avatar data of Claim 1. Specifically, none of cited references teach that the volumetric data element (included in the first data element of Claim 1) is divided into the plurality of sub-data elements by referring to the motion information of the dynamic 3D avatar data (included in the second data element of Claim 1) for encoding and transmitting of the first data element.
The core of the present invention lies in the inter-data correlation during the data division process. Specifically, amended Claim 1 requires that the first data element (including volumetric data) is divided into a plurality of sub-data elements by referring to information of the second data element (including motion information of the dynamic 3D avatar data). In the prior art, geometry/attribute data (Han) and motion data (Naka) are treated as separate, unrelated data streams processed in isolation. There is absolutely no motivation, suggestion, or rationale in Han, Peter, or Naka to utilize the structural or temporal information of motion information of the dynamic 3D avatar data as a reference metric to dynamically divide volumetric data into sub-elements. Without the benefit of hindsight based on the Applicant's own disclosure, a person of ordinary skill in the art would not have arrived at the configuration where motion information of the dynamic 3D avatar data serves as the baseline reference for partitioning volumetric avatar data.
Examiner respectably disagrees. Han teaches an encoding geometry and attribute data(Page 12 Para. 5). The point cloud encoding unit can encode the point cloud data together into a single bitstream or into separate bitstreams (Page 14, Para. 2). The point cloud data can be data associated with an avatar (Page 28 Para. 4-5). Hans briefly teaches motion for the avatar can be acquired as points on a fixed screen (Page 38, Para. 1) and skeleton values are applied to avatars to express movement in avatars(Page 38, Para. 4). While Hans does not teach skeleton and the motion of the skeleton in depth, Naka does.
Naka teaches encoding and transmitting avatar data (Col. 2 Lines 30-41). Naka teaches generating 3-DCG data streams representing 3-dimensional virtual spaces including avatar data. (Col. 4 Lines 54-67 to Col. 5 Lines 1-20). The 3-DCG is generated according to a stream authoring command inputted by the user (Col. 4 Lines 54-67). The stream authoring divides the data streams into arbitrary lengths for synchronization between three different data streams(Col. 10 Lines 66-67 to Col. 11 Lines 1-9). The three different data streams making up the avatar data are the skeletal data stream Ss, motion data stream Sm, and audio data stream Sa (Col. 5 Lines 53-67 to Col. 6 Lines 1-3). The conditions used to divide the data streams into arbitrary lengths are determined arbitrarily and can be a number of joints and degree of freedom, or by duration of motion and generation time of sound(Col. 10 Lines 66-67 to Col. 11 Lines 1-9).The skeletal data stream Ss includes data to define hierarchical relation of the skeletal structure, the initial states of the hierarchical data, states of joints, shape corresponding to the skeleton, and relations corresponding to the skeletal structure (Col. 6 Lines 22-39, Fig. 6). The motion data stream sm contains information about the skeletal structures initial values, and the motion values associated with the joints, (Col. 8 Lines 59-67 to Col. 9 Lines 1-10, Fig. 7 and Fig. 11). Both the motion data stream sm and skeletal data stream ss contain information about the skeletal structure. Different joints of the skeletal structure move differently and have specific restrictive conditions based on the skeleton (Col. 7 Lines 40-54). Once the streams have been generated they are transmitted to the 3-DCG data stream transmitting (Col. 11 Lines 15-41), where the PCS encodes a 3-DCG data stream DS (Col. 5 Lines 1-20). Thus, Naka teaches the first data element(Skeletal Data Stream Ss, Col. 5 Lines 53-67 to Col. 6 Lines 1-21) is divided into the plurality of sub-data elements(Joints and Shape Corresponding to Skeleton Col. 6 Lines 22-39 and the Stream Divided into an Arbitrary Data Length Col. 10 Lines 66-67 to Col. 11 Lines 1-9) based on the motion information (Duration of Motion) of the dynamic 3D avatar data included in the second data element(Motion Data Stream Sm, Col. 5 Lines 53-67 to Col. 6 Lines 1-21), for the first encoding and transmission (Col. 4 Lines 54-67 to Col. 5 Lines 1-20 and Fig. 1 and Fig. 2).
In response to applicant's argument that the examiner's conclusion of obviousness is based upon improper hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971).
In response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, the Han’s reference is directed towards encoding/decoding streams of point cloud data, where the point cloud data contains three-dimensional media data which includes avatar data. The point cloud data is formatted in Polygon file format (PLY) or stand triangle format (Han et al. Page 12 Para. 5) The Naka’s reference is directed towards encoding/decoding streams of 3-dimensional virtual spaces which include avatar data. The 3-dimesional virtual space data is formatted in Virtual Reality Modeling Language (VRML, Naka et al. Col. 1 Lines 39-67 to Col. 2 Lines 1-4) Both references encode 3-dimensional avatar data. As well as the PLY format can be converted to VRML and VRML to PLY. Thus, the data of either reference can be formatted in either of the formats and then encoded.
Regarding the remaining arguments applicant argues with respect to the amended claim language, which is fully addressed in the prior art rejections set forth below.
Claim Objections
Claim 18 is objected to because of the following informalities:
Claim 18 depends from canceled claim 17. Examiner is interpreting claim 18 to depend from claim 10, which canceled claim 17 depended from. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 10-16 and 18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 10, states the following limitation “…wherein the first data element is divided into the plurality of sub-data elements based on the motion information of the dynamic 3D avatar data included in the second data element, for the first encoding and transmission.” Examiner is unsure what applicant is referring to by “for the first encoding and transmission” as there is no mention of first encoding and transmission in claim 10. Did applicant mean to claim, “for the first reception and decoding” instead as claim 10’s obtaining limitations are directed to “reception and decoding” or is “the” merely a typo and should be “a” to state, “for a first encoding and transmission”. Examiner is interpreting the claim limitation to be a typo and therefore stating “ for a first encoding and transmission”. Thus, the claim will be examined as best understood by the Examiner.
Claims 11-16 and 18 inherit their indefiniteness from claim 10 from which they depend.
Claims 11-16 and 18 will also be examined as best understood by the Examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5-9, 10-13, and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Han et al. WIPO Application WO 2023003349 A1 (hereinafter Han) in view of Naka et al. U.S. Patent 6512520 B1 (hereinafter Naka).
Regarding claim 1, Han teaches a method for transmitting dynamic 3-dimensional (3D) avatar data (Transmitting Point Cloud Data Containing Avatar Data, Page 12 Para. 5 and Fig. 9) , the method comprising:
generating (Creating an Avatar Page 28 Para. 4 or Generating a Point Cloud Page 12 Para. 5) a plurality of data elements (Point Cloud Data - Attributes, Geometry, Auxiliary Data, Mesh Data, Page 13 Para. 5 and Fig. 9) configuring the dynamic 3D avatar data (Point Cloud Data Containing Avatar Data, Page 12 Para. 5 and Fig. 9),
performing a first encoding and transmission for a first data element (Point Cloud Data Containing Attribute Stream, Page 12 Para. 5 and Fig. 9 and Geometry Stream, Page 12 Para. 5 and Fig. 9) of the plurality of data elements(Point Cloud Data - Attributes, Geometry, Auxiliary Data, Mesh Data, Page 13 Para. 5 and Fig. 9) wherein the first data element includes volumetric avatar data including avatar geometry data (Geometry Information – Voxel Information Page 32 Para. 6 and x, y, z position values Page 13 Para. 6) and avatar texture data(Texture, Color, Reflectance, Attributes 1-N, Page 13 Para. 7 to Page 14 Para. 1); The point cloud encoding unit can encode the point cloud data (Attributes, Geometry, Auxiliary Data, Mesh Data, Page 13 Para. 5 and Fig. 9) together into a single bitstream or into separate bitstreams (Page 14, Para. 2).
wherein the first data element(Point Cloud Data Containing Attribute Stream, Page 12 Para. 5 and Fig. 9 and Geometry Stream, Page 12 Para. 5 and Fig. 9) or the second data element is divided into a plurality of sub-data elements (Separating Point Cloud Data into Geometry and Attributes, Page 12 Para. 5 and Page 16 Para. 1) ,or each of the first data element and the second data element is divided into a plurality of sub-data elements,
Han fails to explicitly teach:
and performing a second encoding and transmission for a second data element of the plurality of data elements, wherein the second data element includes motion information of the dynamic 3D avatar data,
wherein the first data element is divided into the plurality of sub-data elements based on the motion information of the dynamic 3D avatar data included in the second data element, for the first encoding and transmission.
Han and Naka are analogous to the claimed invention because both of them are in the same field of encoding and transmitting avatar data.
Naka teaches:
and performing a second encoding (Col. 5 Lines 1-20) and transmission for a second data element (Motion Data Stream Sm, Col. 5 Lines 53-67 to Col. 6 Lines 1-21) of the plurality of data elements, wherein the second data element includes motion information(Motion Data Stream Sm, Col. 5 Lines 53-67 to Col. 6 Lines 1-21) of the dynamic 3D avatar data,
wherein the first data element (Skeletal Data Stream Ss, Col. 5 Lines 53-67 to Col. 6 Lines 1-21) is divided into the plurality of sub-data elements (Skeletal Data Stream Divided into an Arbitrary Data Length Col. 10 Lines 66-67 to Col. 11 Lines 1-9 and further divided into Joints/ Shape Corresponding to Skeleton, Col. 6 Lines 22-39, Fig. 6) based on the motion information (Duration of Motion, Col. 10 Lines 66-67 to Col. 11 Lines 1-9) of the dynamic 3D avatar data (Three data streams, Col. 5 Lines 53-67 to Col. 6 Lines 1-3) included in the second data element(Motion Data Stream Sm, Col. 5 Lines 53-67 to Col. 6 Lines 1-21), for the first encoding and transmission (Col. 4 Lines 54-67 to Col. 5 Lines 1-20 and Fig. 1 and Fig. 2). As stated above, Naka teaches encoding and transmitting avatar data (Col. 2 Lines 30-41). Naka teaches generating 3-DCG data streams representing 3-dimensional virtual spaces including avatar data. (Col. 4 Lines 54-67 to Col. 5 Lines 1-20). The 3-DCG is generated according to a stream authoring command inputted by the user (Col. 4 Lines 54-67). The stream authoring divides the data streams into arbitrary lengths for synchronization between three different data streams(Col. 10 Lines 66-67 to Col. 11 Lines 1-9). The three different data streams making up the avatar data are the skeletal data stream Ss, motion data stream Sm, and audio data stream Sa (Col. 5 Lines 53-67 to Col. 6 Lines 1-3). The conditions used to divide the data streams into arbitrary lengths are determined arbitrarily and can be a number of joints and degree of freedom, or by duration of motion and generation time of sound(Col. 10 Lines 66-67 to Col. 11 Lines 1-9).The skeletal data stream Ss includes data to define hierarchical relation of the skeletal structure, the initial states of the hierarchical data, states of joints, shape corresponding to the skeleton, and relations corresponding to the skeletal structure (Col. 6 Lines 22-39, Fig. 6). The motion data stream sm contains information about the skeletal structures initial values, and the motion values associated with the joints, (Col. 8 Lines 59-67 to Col. 9 Lines 1-10, Fig. 7 and 11). Both the motion data stream sm and skeletal data stream ss contain information about the skeletal structure. Different joints of the skeletal structure move differently and have specific restrictive conditions based on the skeleton (Col. 7 Lines 40-54). Once the streams have been generated they are transmitted to the 3-DCG data stream transmitting (Col. 11 Lines 15-41), where the PCS encodes a 3-DCG data stream DS (Col. 5 Lines 1-20). Thus, Naka teaches the first data element(Skeletal Data Stream Ss, Col. 5 Lines 53-67 to Col. 6 Lines 1-21) is divided into the plurality of sub-data elements(Joints and Shape Corresponding to Skeleton Col. 6 Lines 22-39 and the Stream Divided into an Arbitrary Data Length Col. 10 Lines 66-67 to Col. 11 Lines 1-9) based on the motion information (Duration of Motion) of the dynamic 3D avatar data included in the second data element(Motion Data Stream Sm, Col. 5 Lines 53-67 to Col. 6 Lines 1-21), for the first encoding and transmission.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Han’s Avatar Data Streams to incorporate Naka’s Encoding and Transmission for a Second data element, wherein the second data element includes motion information of the dynamic 3D avatar data, wherein the first data element is divided into the plurality of sub-data elements based on the motion information of the dynamic 3D avatar data included in the second data element, for the first encoding and transmission. Since doing so would provide the benefit of reproducing skeletal structures in frames naturally by defining motion amounts of joints of the skeleton (Naka et al. Col. 2 Lines 5-14) and would allow for animation recreation to fit the environment better. As information regarding skeletal motion is linked with the portions of skeleton being moved.
Regarding claim 2, Han teaches the method of Claim 1, wherein:
a period (Specific Time Period, Page 17 Para. 2) of the first encoding and transmission and (Point Cloud Data Containing Attribute Stream, Page 12 Para. 5 and Fig. 9 and Geometry Stream, Page 12 Para. 5 and Fig. 9) and The point cloud can be represented in frames/pictures which can represent a specific time period (Page 17 Para. 2) in frames. The point cloud is divided into several streams or combined into one with sub elements (Page 14 Para. 2). Han teaches a delivery unit that delivers the point cloud for transmission processing or data that has already been processed. This delivery unit follows any transmission protocol and delivers data in an on-demand manner. The delivery unit delivers the point cloud based on orientation, viewpoint, or any information selected by the user (Page 14, Para. 4). Thus, a user could choose to deliver portion of the point cloud based on the characteristics of the point cloud data, such as the first data element.
Han fails to explicitly teach a period of the second encoding and transmission are adaptively determined based on a characteristic of the first data element and the second data element, respectively.
Naka teaches:
a period (Real-Time Basis, Col. 2 Lines 42-67) of the second encoding(Col. 5 Lines 1-20) and transmission are adaptively determined based on a characteristic of the first data element(Skeletal Data Stream Ss, Col. 5 Lines 53-67 to Col. 6 Lines 1-21) and the second data element(Motion Data Stream Sm, Col. 5 Lines 53-67 to Col. 6 Lines 1-21), respectively.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Han’s Encoding and Transmitting to incorporate Naka’s period of the second encoding and transmission are adaptively determined based on a characteristic of the first data element and the second data element, respectively. Since doing so would provide the benefit of transmitting and receiving on a real-time basis synchronously the motions of the avatar in a 3-dimensional space (Naka et al. Col. 2 Lines 30-41).
Regarding claim 3, Han teaches the method of Claim 1, wherein:
when the first data element(Point Cloud Data Containing Attribute Stream, Page 12 Para. 5 and Fig. 9 and Geometry Stream, Page 12 Para. 5 and Fig. 9) is divided into the plurality of sub-data elements (Separating Point Cloud Data into Geometry and Attributes, Page 12 Para. 5 and Page 16 Para. 1), the first encoding and transmission for the first data element includes:
a first period-based(Specific Time Period, Page 17 Para. 2) encoding and transmission for a first sub-data element (Slice Data - Attr00 or Geom00) among the plurality of sub-data elements(Attributes 1-N or Slices, Page 22 Para. 10), A slice in encoding can be a portion of a video frame that has been divided into manageable segments. Slices are often encoded/decoded/transmitted separately, but in parallel. In Han’s the slices are portions of the attribute or geometry streams which are portions of the point cloud.
However, Han fails to teach:
and a second period-based encoding and transmission for a second sub-data element among the plurality of sub-data elements.
Naka teaches:
and a second period-based(Real-Time Basis, Col. 2 Lines 42-67) encoding(Col. 5 Lines 1-20) and transmission for a second sub-data element (Motion values or Skeletal Structure Initial Values, Col. 8 Lines 59-67 to Col. 9 Lines 1-10) among the plurality of sub-data elements.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Han’s Encoding and Transmitting to incorporate Naka’s second period-based encoding and transmission for a second sub-data element among the plurality of sub-data elements. Since doing so would provide the benefit of transmitting and receiving on a real-time basis synchronously the motions of the avatar in a 3-dimensional space (Naka et al. Col. 2 Lines 30-41).
Regarding claim 6, Han teaches the method of Claim 1, wherein:
an encoding and a transmission of the plurality of sub-data elements(Geometry – x, y, z position values Page 13 Para. 6, Attribute – Texture, Color, Reflectance, Attributes 1-N, Page 13 Para. 7 to Page 14 Para. 1) are performed independently or in parallel (Page 14 Para. 2).
Regarding claim 7, Han teaches the method of Claim 1, wherein:
based on a combination of the first data element(Point Cloud Data Containing Attribute Stream, Page 12 Para. 5 and Fig. 9 and Geometry Stream, Page 12 Para. 5 and Fig. 9), or at least one sub-data element(Separating Point Cloud Data into Geometry and Attributes, Page 12 Para. 5 and Page 16 Para. 1) of the first data element;
at least one sub-avatar data (Point Cloud or Portion of Point Cloud) is configured. The avatar data is represented as point cloud data. Combining the point cloud streams results in the full point cloud. Thus, combining sub elements of the attribute and geometry streams results in portions of the point cloud. Which would be sub avatar data, as the data would only consist of the portions of the point cloud. Resulting in portions of the avatar data as only portions of the point cloud were encoded.
However, Han fails to teach:
and the second data element, or at least one sub-data element of the second data element,
Naka teaches:
based on a combination of the first data element(Skeletal Data Stream Ss, Col. 5 Lines 53-67 to Col. 6 Lines 1-21), or at least one sub-data element(Skeletal Data Stream Divided into an Arbitrary Data Length Col. 10 Lines 66-67 to Col. 11 Lines 1-9 and further divided into Joints/ Shape Corresponding to Skeleton, Col. 6 Lines 22-39, Fig. 6) of the first data element;
and the second data element(Motion values or Skeletal Structure Initial Values, Col. 8 Lines 59-67 to Col. 9 Lines 1-10), or at least one sub-data element(Motion values or Skeletal Structure Initial Values, Col. 8 Lines 59-67 to Col. 9 Lines 1-10) of the second data element,
at least one sub-avatar data (Portion of the Avatar – Joint/limb) is configured.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Han’s Encoding and Transmitting to incorporate Naka’s Second Data Element, or at least one Sub-data Element of the Second Data Element, Since doing so would provide the benefit of reproducing skeletal structures in frames naturally by defining motion amounts of joints of the skeleton (Naka et al. Col. 2 Lines 5-14) and would allow for animation recreation to fit the environment better. As information regarding skeletal motion is linked with the portions of skeleton being moved.
Regarding claim 8, Han teaches the method of Claim 1, wherein:
the motion information of the dynamic 3D avatar data includes avatar skeleton motion data.
Naka teaches:
the motion information (Motion Data Stream Sm, Col. 5 Lines 53-67 to Col. 6 Lines 1-21) of the dynamic 3D avatar data includes avatar skeleton motion data. The motion data stream sm contains information about the skeletal structures initial values, and the motion values associated with the joints, (Col. 8 Lines 59-67 to Col. 9 Lines 1-10).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Han’s Avatar Data Streams to incorporate Naka’s motion information of the dynamic 3D avatar data includes avatar skeleton motion data. Since doing so would provide the benefit of reproducing skeletal structures in frames naturally by defining motion amounts of joints of the skeleton (Naka et al. Col. 2 Lines 5-14) and would allow for animation recreation to fit the environment better.
Regarding claim 9, Han teaches the method of Claim 8, wherein: the plurality of data elements further include at least one of avatar audio data, avatar haptic data, or avatar metadata (Auxiliary Data, Page 13 Para. 7 to Page 14 Para. 1). Auxiliary Data represents metadata about the point cloud.
Regarding claim 10, Han teaches a method for obtaining dynamic 3-dimensional (3D) avatar data (Obtaining Decoded Point Cloud Data Containing Avatar Data, Page 13 Para. 2, Fig. 10, Page 28 Para. 4) the method comprising:
obtaining a first data element (Point Cloud Data Containing Attribute Stream, Page 12 Para. 5 and Fig. 9 and Geometry Stream, Page 12 Para. 5 and Fig. 9) based on a first reception and decoding (Point Cloud Decoding, fig. 10) for an encoded first data element of a plurality of data elements (Point Cloud Data - Attributes, Geometry, Auxiliary Data, Mesh Data, Page 13 Para. 5 and Fig. 9),wherein the first data element includes volumetric avatar data including avatar geometry data (Geometry Information – Voxel Information Page 32 Para. 6 and x, y, z position values Page 13 Para. 6) and avatar texture data(Texture, Color, Reflectance, Attributes 1-N, Page 13 Para. 7 to Page 14 Para. 1); The point cloud encoding unit can encode the point cloud data (Attributes, Geometry, Auxiliary Data, Mesh Data, Page 13 Para. 5 and Fig. 9) together into a single bitstream or into separate bitstreams (Page 14, Para. 2).
and obtaining the dynamic 3D avatar data(Obtaining Decoded Point Cloud Data Containing Avatar Data, Page 13 Para. 2, Fig. 10) based on the plurality of data elements(Point Cloud Data - Attributes, Geometry, Auxiliary Data, Mesh Data, Page 13 Para. 5 and Fig. 9) including the first data element(Point Cloud Data Containing Attribute Stream, Page 12 Para. 5 and Fig. 9 and Geometry Stream, Page 12 Para. 5 and Fig. 9)
wherein the first data element(Point Cloud Data Containing Attribute Stream, Page 12 Para. 5 and Fig. 9 and Geometry Stream, Page 12 Para. 5 and Fig. 9) or the second data element is divided into a plurality of sub-data elements (Separating Point Cloud Data into Geometry and Attributes, Page 12 Para. 5 and Page 16 Para. 1) ,or each of the first data element and the second data element is divided into a plurality of sub-data elements,
Han fails to explicitly teach:
obtaining a second data element based on a second reception and decoding for an encoded second data element of the plurality of data elements, wherein the second data element includes motion information of the dynamic 3D avatar data;
wherein the first data element is divided into the plurality of sub-data elements based on the motion information of the dynamic 3D avatar data included in the second data element, for the first encoding and transmission.
Han and Naka are analogous to the claimed invention because both of them are in the same field of encoding and transmitting avatar data.
Nake teaches:
obtaining a second data element(Motion Data Stream Sm, Col. 5 Lines 53-67 to Col. 6 Lines 1-21) based on a second reception and decoding (Col. 6 Lines 65-67 to Col. 7 Lines 1-39) for an encoded second data element of the plurality of data elements(Motion values or Skeletal Structure Initial Values, Col. 8 Lines 59-67 to Col. 9 Lines 1-10), wherein the second data element includes motion information(Motion Data Stream Sm, Col. 5 Lines 53-67 to Col. 6 Lines 1-21) of the dynamic 3D avatar data;
and obtaining the dynamic 3D avatar data(Three data streams, Col. 5 Lines 53-67 to Col. 6 Lines 1-3) based on the plurality of data elements including the first data element(Skeletal Data Stream Ss, Col. 5 Lines 53-67 to Col. 6 Lines 1-21) and the second data element(Motion Data Stream Sm, Col. 5 Lines 53-67 to Col. 6 Lines 1-21),
wherein the first data element (Skeletal Data Stream Ss, Col. 5 Lines 53-67 to Col. 6 Lines 1-21) is divided into the plurality of sub-data elements (Skeletal Data Stream Divided into an Arbitrary Data Length Col. 10 Lines 66-67 to Col. 11 Lines 1-9 and further divided into Joints/ Shape Corresponding to Skeleton, Col. 6 Lines 22-39, Fig. 6) based on the motion information (Duration of Motion, Col. 10 Lines 66-67 to Col. 11 Lines 1-9) of the dynamic 3D avatar data (Three data streams, Col. 5 Lines 53-67 to Col. 6 Lines 1-3) included in the second data element(Motion Data Stream Sm, Col. 5 Lines 53-67 to Col. 6 Lines 1-21), for the first encoding and transmission (Col. 4 Lines 54-67 to Col. 5 Lines 1-20 and Fig. 1 and Fig. 2). As stated above, Naka teaches encoding and transmitting avatar data (Col. 2 Lines 30-41). Naka teaches generating 3-DCG data streams representing 3-dimensional virtual spaces including avatar data. (Col. 4 Lines 54-67 to Col. 5 Lines 1-20). The 3-DCG is generated according to a stream authoring command inputted by the user (Col. 4 Lines 54-67). The stream authoring divides the data streams into arbitrary lengths for synchronization between three different data streams(Col. 10 Lines 66-67 to Col. 11 Lines 1-9). The three different data streams making up the avatar data are the skeletal data stream Ss, motion data stream Sm, and audio data stream Sa (Col. 5 Lines 53-67 to Col. 6 Lines 1-3). The conditions used to divide the data streams into arbitrary lengths are determined arbitrarily and can be a number of joints and degree of freedom, or by duration of motion and generation time of sound(Col. 10 Lines 66-67 to Col. 11 Lines 1-9).The skeletal data stream Ss includes data to define hierarchical relation of the skeletal structure, the initial states of the hierarchical data, states of joints, shape corresponding to the skeleton, and relations corresponding to the skeletal structure (Col. 6 Lines 22-39, Fig. 6). The motion data stream sm contains information about the skeletal structures initial values, and the motion values associated with the joints, (Col. 8 Lines 59-67 to Col. 9 Lines 1-10, Fig. 7 and 11). Both the motion data stream sm and skeletal data stream ss contain information about the skeletal structure. Different joints of the skeletal structure move differently and have specific restrictive conditions based on the skeleton (Col. 7 Lines 40-54). Once the streams have been generated they are transmitted to the 3-DCG data stream transmitting (Col. 11 Lines 15-41), where the PCS encodes a 3-DCG data stream DS (Col. 5 Lines 1-20). Thus, Naka teaches the first data element(Skeletal Data Stream Ss, Col. 5 Lines 53-67 to Col. 6 Lines 1-21) is divided into the plurality of sub-data elements(Joints and Shape Corresponding to Skeleton Col. 6 Lines 22-39 and the Stream Divided into an Arbitrary Data Length Col. 10 Lines 66-67 to Col. 11 Lines 1-9) based on the motion information (Duration of Motion) of the dynamic 3D avatar data included in the second data element(Motion Data Stream Sm, Col. 5 Lines 53-67 to Col. 6 Lines 1-21), for the first encoding and transmission.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Han’s Avatar Data Streams to incorporate Naka’s obtaining a second data element, wherein the second data element includes motion information of the dynamic 3D avatar data, wherein the first data element is divided into the plurality of sub-data elements based on the motion information of the dynamic 3D avatar data included in the second data element, for the first encoding and transmission. Since doing so would provide the benefit of reproducing skeletal structures in frames naturally by defining motion amounts of joints of the skeleton (Naka et al. Col. 2 Lines 5-14) and would allow for animation recreation to fit the environment better. As information regarding skeletal motion is linked with the portions of skeleton being moved.
Regarding claim 11, Han teaches the method of Claim 10, wherein:
based on a period (Specific Time Period, Page 17 Para. 2) of at least one of the first reception and decoding(Point Cloud Decoding, Fig. 10) or corresponding to a plurality of frames(Page 17 Para. 2), The point cloud can be mapped to frames/pictures.
at least one of the first data element (Point Cloud Data Containing Attribute Stream, Page 12 Para. 5 and Fig. 9 and Geometry Stream, Page 12 Para. 5 and Fig. 9) or obtained from a first frame (Portion of Point Cloud) among the plurality of frames (Entire Point Cloud) is applied to at least one remaining frame (Portion of Point Cloud) among the plurality of frames (Entire Point Cloud). The point cloud can have multiple frames associated with it; thus, portions of a point cloud stream could belong to multiple frames (Page 13 Para. 6).
Regarding claim 12, Han teaches the method of Claim 10, wherein:
when the first data element(Point Cloud Data Containing Attribute Stream, Page 12 Para. 5 and Fig. 9 and Geometry Stream, Page 12 Para. 5 and Fig. 9) is divided into the plurality of sub-data elements(Separating Point Cloud Data into Geometry and Attributes, Page 12 Para. 5 and Page 16 Para. 1), the first reception and decoding(Point Cloud Decoding, Fig. 10 for the first data element includes:
a first period-based (Specific Time Period, Page 17 Para. 2) reception and decoding(Point Cloud Decoding, Fig. 10) for a first sub-data element (Slice Data - Attr00 or Geom00) among the plurality of sub-data elements(Attributes 1-N or Slices, Page 22 Para. 10), A slice in encoding can be a portion of a video frame that has been divided into manageable segments. Slices are often encoded/decoded/transmitted separately, but in parallel. In Han’s the slices are portions of the attribute or geometry streams which are portions of the point cloud.
However, Han fails to teach:
and a second period-based(Real-Time Basis, Col. 2 Lines 42-67) reception and decoding(Col. 6 Lines 65-67 to Col. 7 Lines 1-39) for a second sub-data element among(Motion values or Skeletal Structure Initial Values, Col. 8 Lines 59-67 to Col. 9 Lines 1-10) the plurality of sub-data elements.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Han’s Encoding and Transmitting to incorporate Naka’s second period-based reception and decoding for a second sub-data element among the plurality of sub-data elements. Since doing so would provide the benefit of transmitting and receiving on a real-time basis synchronously the motions of the avatar in a 3-dimensional space (Naka et al. Col. 2 Lines 30-41).
Regarding claim 13, Han teaches the method of Claim 12, wherein:
through merging first sub-avatar data(Portion of Point Cloud based on Geometry or Attribute Streams) based on the first sub-data element(Slice Data - Attr00 or Geom00) obtained based on the first period(Specific Time Period, Page 17 Para. 2), and second sub-avatar data (Portion of Point Cloud based on Another Stream, the dynamic 3D avatar data in a current frame (Point Cloud or Portion of Point Cloud) is obtained. The sub-data elements can be portions of the attribute or geometry streams. These portions of streams contain portions of the point cloud. Since the point cloud can be of an avatar the portion of streams would contain sub-avatar data. As well as, since the point cloud can be mapped to frames/pictures. The portion of point cloud retrieved/decoded could be any frame or the only frame.
However, Han fails to teach:
second sub-avatar data based on the second sub-data element obtained based on the second period,
Naka teaches:
through merging first sub-avatar(Portion of the Avatar – Joint/limb) databased on the first sub-data element(Skeletal Data Stream Divided into an Arbitrary Data Length Col. 10 Lines 66-67 to Col. 11 Lines 1-9 and further divided into Joints/ Shape Corresponding to Skeleton, Col. 6 Lines 22-39, Fig. 6) obtained based on the first period(Real-Time Basis, Col. 2 Lines 42-67), and second sub-avatar data(Portion of the Avatar – Joint/limb) based on the second sub-data element(Motion values or Skeletal Structure Initial Values, Col. 8 Lines 59-67 to Col. 9 Lines 1-10) obtained based on the second period(Real-Time Basis, Col. 2 Lines 42-67), the dynamic 3D avatar data in a current frame (Col. 8 Lines 66-67 to Col. 9 Lines 1-10) is obtained.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Han’s Encoding and Transmitting to incorporate Naka’s second sub-avatar data based on the second sub-data element obtained based on the second period, Since doing so would provide the benefit of reproducing skeletal structures in frames naturally by defining motion amounts of joints of the skeleton (Naka et al. Col. 2 Lines 5-14) and would allow for animation recreation to fit the environment better. As information regarding skeletal motion is linked with the portions of skeleton being moved.
Regarding claim 16, Han teaches the method of Claim 10, wherein:
a post-processing process (Combining Point Cloud Streams, Fig. 10 and Fig. 12)related to a removal or an alleviation of a discontinuity between the plurality of sub-data elements (Geometry – x, y, z position values Page 13 Para. 6, Attribute – Texture, Color, Reflectance, Attributes 1-N, Page 13 Para. 7 to Page 14 Para. 1) is performed. The point cloud streams are combined to render the decoded point cloud data and provide content to a user, like displaying an avatar.
Regarding claim 17, has similar limitations as of claim 8, therefore it is rejected under the same rationale as claim 8.
Regarding claim 18, has similar limitations as of claim 9, therefore it is rejected under the same rationale as claim 9.
Regarding claim 19, Han teaches a device (First or Second Communication Device) for transmitting dynamic 3-dimensional (3D) avatar data, the device comprising: at least one processor (Processors 911, 921, 913, 923, Page 4 Para. 5 to Page 5 Para. 1); and at least one memory (Memory 914 and 924, Page 4 Para. 5 to Page 5 Para. 1) operably connected to the at least one processor, and storing an instruction for making the device to perform an operation when being executed by the at least one processor, wherein the operation includes the transmitting method of claim 1, therefore it is rejected under the same rationale as claim 1.
Regarding claim 20, Han teaches a device(First or Second Communication Device) for obtaining dynamic 3-dimensional (3D) avatar data, the device comprising: at least one processor(Processors 911, 921, 913, 923, Page 4 Para. 5 to Page 5 Para. 1); and at least one memory (Memory 914 and 924, Page 4 Para. 5 to Page 5 Para. 1) operably connected to the at least one processor, and storing an instruction for making the device to perform an operation when executed by the at least one processor, wherein the operation includes the obtaining method of claim 10, therefore it is rejected under the same rationale as claim 10.
Claim(s) 4 are rejected under 35 U.S.C. 103 as being unpatentable over Han et al. WIPO Application WO 2023003349 A1 (hereinafter Han) in view of Naka et al. U.S. Patent 6512520 B1 (hereinafter Naka) in further view of NPL “Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence” by Yonggan Fu, Yuecheng Li, Chenghui Li, Jason Saragih, Peizhao Zhang, Xiaoliang Dai, Yingyan (Celine) Lin, Georgia Institute of Technology, and Meta (hereinafter Fu).
Regarding claim 4, Han and fails to teach the method of Claim 3, wherein:
the first period and the second period are adaptively determined based on a degree of a change in an attribute of the first sub-data element and the second sub-data element over time, respectively.
Naka teaches:
the first period(Real-Time Basis, Col. 2 Lines 42-67) and the second period(Real-Time Basis, Col. 2 Lines 42-67) are adaptively determined (Synchronized with Each Other, Col. 8 Lines 11-23 and Are Transmitted Simultaneously, Col. 2 Lines 42-67 to Col. 3 Lines 1-11) based on a degree of a change in an attribute of the first sub-data element(Skeletal Data Stream Divided into an Arbitrary Data Length Col. 10 Lines 66-67 to Col. 11 Lines 1-9 and further divided into Joints/ Shape Corresponding to Skeleton, Col. 6 Lines 22-39, Fig. 6) and the second sub-data element(Motion values or Skeletal Structure Initial Values, Col. 8 Lines 59-67 to Col. 9 Lines 1-10) over time, respectively.
Han, Naka, and Fu are analogous to the claimed invention because all of them are in the same field of encoding avatar data.
However, Han and Naka fail to teach:
the first period and the second period are adaptively determined based on a degree of a change in an attribute
Fu teaches the method of Claim 3, wherein:
the first period (Number of Frames) and the second period (Number of Frames) are adaptively determined based on a degree of a change in an attribute (Difference in Left/Right Eye or Mouth) of the first sub-data element (Mouth, Fig. 1 (b)) and the second sub-data element (Eyes, Fig. 1 (b)) over time, respectively. (Section: 4.3 LATEX: Leveraging Temporal Redundancy) Fu teaches reducing computations for frames deemed redundant. The frames of the avatar face are divided into portions of frames containing eyes and mouth (Fig. 1 (b)). By utilizing LATEX which compares the differences between frames (Differences Left/Right Eye or Mouth). If the differences are deemed substantial the frame is encoded, but if it is not the frame is extrapolated (Section: 4.3.2 Proposed Adaptive Latent Extrapolation).
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Han’s Point Cloud Frame altered by Naka to incorporate Fu’s first period and second period are adaptively determined based on a degree of a change in an attribute of the first sub-data element and the second sub-data element over time, respectively Since doing so would provide the benefit of reducing computation by reducing encoding redundant frames.
Claim(s) 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Han et al. WIPO Application WO 2023003349 A1 (hereinafter Han) ) in view of Naka et al. U.S. Patent 6512520 B1 (hereinafter Naka) in further view of NPL “Retargeting Human-Object Interaction to Virtual Avatars” by Yeonjoon Kim, Hangil Park, Seungbae Bang, and Sung-Hee Lee (hereinafter Kim).
Regarding claim 14, Han teaches the method of Claim 10, wherein:
(Point Cloud Data Containing Attribute Stream, Page 12 Para. 5 and Fig. 9 and Geometry Stream, Page 12 Para. 5 and Fig. 9), the dynamic 3D avatar data(Portion of Point Cloud based on Geometry or Attribute Streams) in a current frame(Point Cloud or Potion of Point Cloud) is obtained. The point cloud can be mapped to frames/pictures. The portion of point cloud retrieved/decoded could be any frame or the only frame. The points are based on the position information in the geometry stream. Thus, the attribute stream refers to the geometry stream to determine which attributes belong to specific points in the point cloud. (Page 22 Para. 10) Similarly, the geometry would refer to the attribute stream for the specific point attributes.
However, Han and Naka fails to teach through a retargeting with applying information of the second data element for the first data element, the dynamic 3D avatar data in a current frame is obtained.
Han, Naka, and Kim are analogous to the claimed invention because all of them are in the same field of processing/generating avatars/meshes.
Kim teaches through a retargeting (Retargeting Motion) based on information of the second data element (Input Motion) for the first data element (Input Object Geometries), the dynamic 3D avatar data in a current frame is obtained. (Fig. 2 and Section: 3 Motion Retargeting Method) Kim teaches retargeting the motion of an avatar based on the surroundings meshes. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Han’s Data Elements altered by Peters to incorporate Kim’s Retargeting. Since doing so would provide the benefit of retargeting avatar motion to better fit the 3D spaces surrounding them, (Section: 1 Introduction) as Han’s already incorporates meshes of the environment and avatar.
Regarding claim 15, Han teaches the method of Claim 10, wherein:
based on a combination of the first data element(Point Cloud Data Containing Attribute Stream, Page 12 Para. 5 and Fig. 9 and Geometry Stream, Page 12 Para. 5 and Fig. 9), or at least one sub-data element(Geometry – x, y, z position values Page 13 Para. 6, Attribute – Texture, Color, Reflectance, Attributes 1-N, Page 13 Para. 7 to Page 14 Para. 1) of the first data element;
at least one sub-avatar data(Portion of Point Cloud based on Geometry or Attribute Streams)
However, Han fails to teach:
and the second data element, or at least one sub-data element of the second data element,
at least one sub-avatar data is independently retargeted.
Naka teaches:
based on a combination of the first data element(Skeletal Data Stream Ss, Col. 5 Lines 53-67 to Col. 6 Lines 1-21), or at least one sub-data element(Skeletal Data Stream Divided into an Arbitrary Data Length Col. 10 Lines 66-67 to Col. 11 Lines 1-9 and further divided into Joints/ Shape Corresponding to Skeleton, Col. 6 Lines 22-39, Fig. 6) of the first data element;
and the second data element(Motion values or Skeletal Structure Initial Values, Col. 8 Lines 59-67 to Col. 9 Lines 1-10), or at least one sub-data element(Motion values or Skeletal Structure Initial Values, Col. 8 Lines 59-67 to Col. 9 Lines 1-10) of the second data element,
at least one sub-avatar data(Portion of the Avatar – Joint/limb) is
However, Han and Naka fail to teach:
at least one sub-avatar data is independently retargeted.
Kim teaches:
based on a combination of the first data element (Input Object Geometries), or at least one sub-data element of the first data element;
and the second data element(Input Motion), or at least one sub-data element of the second data element,
at least one sub-avatar data is independently retargeted(Retargeting Motion). (Fig. 2 and Section: 3 Motion Retargeting Method) Kim teaches retargeting the motion of an avatar based on the surroundings meshes.
Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Han’s Data Elements altered by Peters to incorporate Kim’s Retargeting. Since doing so would provide the benefit of retargeting avatar motion to better fit the 3D spaces surrounding them, (Section: 1 Introduction) as Han’s already incorporates meshes of the environment and avatar.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIANNA R COCHRAN whose telephone number is (571)272-4671. The examiner can normally be reached Mon-Fri. 7:30am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRIANNA RENAE COCHRAN/ Examiner, Art Unit 2615
/MICHAEL J COBB/ Primary Examiner, Art Unit 2615
