Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 3-4, 6, 8, 11-12, and 23-26 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 7, 9, 21, and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Smolic (NPL Doc, “A Survey on Coding of Static and Dynamic 3D Meshes”) in view of Poularakis (Pub No. US 2024/0397091 A1 ) in further view of Zhang et al. (WIPO Publication WO 2021000241 A1, published on Jan 7, 2021 using US Publication 20220122296 as the English translation), Ilola (Pub No. US 20230050860 A1), and Nadro (US 2022/0292777 A1).
As per claim 1, Smolic teaches the claimed:
A computer-implemented method for encoding three-dimensional (3D) content represented by a dynamic mesh (Please see Smolic in the middle of the 1st page “The following two sections give an overview on single rate and progressive coding techniques for static and dynamic meshes, explaining them in more detail pointing out the main ideas of each encoding approach”), the method comprising:
integerizing the coordinates of each vertex of the plurality of vertices (Please see in Smolic at the bottom of page 257 to the top of page 258 where they refer to: “… Firstly, the coordinates of the vertices are quantized up to a user-given number of bits which results in integer values for each coordinate”);
segmenting the integerized coordinates for the plurality of vertices into one or more (Smolic in figure 8.15 shows that the integerized coordinates for the plurality of vertices into a grid of sub-blocks), and local coordinates of vertices in each (Please see in Smolic at the bottom of page 257 to the top of page 258 where they refer to: “… Firstly, the coordinates of the vertices are quantized up to a user-given number of bits which results in integer values for each coordinate.”
In this passage, the user-given number of bits corresponds to the claimed “video bit depth”.
Smolic makes reference to using local coordinates at the bottom of page 271 to the top of page 272 where they refer to: “It is encoded as an offset from the predicted value relative to the local frame.”);
Smolic alone does not explicitly teach the remaining claim limitations.
It is noted above that Smolic teaches of segmenting in figure 8.15 into a grid, however, Smolic does not show a clear example of 3D sub-blocks per se. However, Smolic in combination with Poularakis teaches the claimed of 3D sub-blocks and each 3D sub-block containing at least one vertex of the plurality of vertices (Please see in Poularakis in [0118] “The fifth node indicates that the third point in the point ordering is at an offset of (1,1,1) from the start (2,2,2), i.e. the third point is at (3,3,3). In other words, this example uses local coordinates within the volume represented by each leaf node of the octree. In other examples, the 3D position may be a global position in the overall volume of the 3D scene. [0119] 6. {1, (1,1,1)}”
In this passage, each leaf node of the octree corresponds to an occupied 3D sub-block. Further, each 3D sub-block uses local coordinates to represent the mesh data (e.g. Poularakis at the end of [0053] “… For example, the described techniques could be applied to volumetric video recorded as mesh data rather than point cloud data”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the 3-D blocks as taught by Poularakis with the system of Smolic in order to better organize the 3D spatial geometry into a 3D related data structure. This may help with storage, retrieval, and tracking of the 3D geometry as it appears in 3D space when encoding 3D geometry represented by a mesh.
for each 3D sub-block, converting coordinates of a vertex inside the 3D sub-block to a local coordinate system of the 3D sub-block (Please see in Poularakis in [0118] “The fifth node indicates that the third point in the point ordering is at an offset of (1,1,1) from the start (2,2,2), i.e. the third point is at (3,3,3). In other words, this example uses local coordinates within the volume represented by each leaf node of the octree. In other examples, the 3D position may be a global position in the overall volume of the 3D scene. [0119] 6. {1, (1,1,1)}” and Poularakis in [0020] “Optionally, in the first particular type of embodiment, the ordering information comprises the 3D position of each point of the current frame. The 3D position may be defined within an overall space containing the entire 3D scene, or may be a position within a smaller local space, such as a node of an octree dividing the overall space.”
In this instance, going from the 3D position defined within the overall space to then a position in a local space such as an octree node (3D sub-block) requires converting the coordinates);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the local coordinate values as taught by Poularakis with the system of Smolic in order to more efficiently store this data. For instance, a local coordinate system often has lower magnitude values as compared to a global coordinate system and thus can store position values more efficiently.
However, Smolic in combination with Poularakis and Zhang teaches the claimed:
and mapping each vertex inside the 3D sub-block to a corresponding 2D patch in a geometry component image of the dynamic mesh that represents the mesh frame (As mentioned above, Smolic teaches of encoding a dynamic mesh. Smolic is silent about the mapping per se. However, Zhang teaches that this was known in the art. Please see Zhang in the English translation where there is a mapping of the vertices in figure 13 in each sub-block (cube blocks) onto planes of each 3D sub-block in figures 16-19).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to encode a dynamic mesh as taught by Zhang with the system of Smolic and Poularakis. This encoding of a dynamic mesh is useful for encoding mesh structures that change often over time such as modeling a human face that is moving or talking over time in a 3D space.
compressing the geometry component image and other geometry component images of the dynamic mesh using a video encoder to generate a geometry component bitstream (Ilola in [0156] “3D representation of at least one object as an input for an encoder; means for projecting the 3D representation onto at least one 2D patch; means for generating at least a geometry image and a texture image from the 2D patch; means for generating, based on the geometry image, a mesh comprising a number of vertices; means for mapping the number of vertices to two-dimensional (2D) coordinates of the texture image; and means for signaling said 2D coordinates of the texture image to be applied to the number of vertices of the mesh in or along a bitstream.”);
and generating a coded mesh bitstream for the dynamic mesh by including at least the geometry component bitstream ( Ilola in [0117 - 0118] “The encoded coordinate + attributes data and vertex connectivity are multiplexed 416 to create the final compressed output bitstream 418. In the decoder, shown in FIG. 4b, the input bitstream 420 is demultiplexed 422 to generate the compressed bitstreams for vertex coordinates + attributes data and vertex connectivity. The vertex coordinate s+ attributes data is decompressed using MPEG-I V-PCC decoder 424. Vertex reordering 428 is carried out on the reconstructed vertex coordinates 426 at the output of MPEG-I V-PCC decoder 424 to match the vertex order at the encoder.”). In this passage, the “coordinate + attributes data and vertex connectivity” corresponds to the claimed “geometry component”.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to generate a coded mesh bitstream as taught by Ilola with the system of Smolic, Poularakis, and Zhang in order to transmit the mesh data in real-time to another device that is located remotely from the source device. Thus, it aids in communicating 3D mesh structures or geometry over distances.
Smolic alone does not explicitly teach the remaining claim limitation relating to “normalizing”.
However, Nadro teaches the claimed:
normalizing coordinates of each vertex of a plurality of vertices in a mesh frame of the dynamic mesh (Towards the end of [0011] “… In an aspect, the method produces the compact heterogeneous mesh sequence with mesh vertices characterized by the normalized position values, which may be four-byte or other quantized values”. In this passage, the mesh sequence corresponds to the claimed “dynamic mesh”. In order to get normalized position values, the vertices would have to have had their coordinates normalized at some point in time).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to normalize coordinates as taught by Nadro with the system of Smolic, Poularakis, Zhang, and Ilola because it increases the efficiency of compression algorithms. For example, it may help ensures that mesh data regardless of its original size, position, or orientation fits within a uniform, original framework or format.
As per claim 7, Smolic alone does not explicitly teach the claimed limitations.
However, Smolic in combination with Zhang teaches the claimed:
The computer-implemented method of claim 1, wherein each vertex of the plurality of vertices belongs to one and only one 3D sub-block ( Please see Zhang in the English translation in figure 7 where each 3D sub-block is divided into smaller 3D sub-blocks and each of these shows only 1 vertex belong to each one).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to assign each vertex to one 3D sub-block as taught by Zhang with the system of Smolic and Poularakis. Assigning only one vertex (or a minimal, optimized set of vertex data) for each 3D sub-block is crucial for optimizing rendering performance, minimizing memory usage, and ensuring efficient, smooth geometric manipulation.
As per claim 9, this claim is similar in scope to limitations recited in claim 1, and thus is rejected under the same rationale. The prior art reference of Smolic would have to have some type of computer-readable medium present in order to function and run on a computer-based system as mentioned in the reference.
As per claim 21, the reasons and rationale for the rejection of claim 1 is incorporated herein. It is noted that this claim appears to be a decoder version of the steps of independent claim 1. Smolic teaches that it was known the decoding is the reverse steps of the encoder, e.g. please see the end of the caption in figure 8.6 where they state “… The decoder knows how the encoder works and can replay the encoding process in reverse”. Thus, it would have been obvious to reverse all the encoding steps taught by the prior art in independent claim 1 to achieve the claimed decoding. Smolic teaches that a reconstructed dynamic mesh may be rendered for display, e.g. please see the middle of page 279 in Smolic where they state: “… The figure shows rendered views with texture mapping at 4 different times from 4 different virtual viewpoints”).
Smolic does not teach the claimed “by converting color information from color planes of the corresponding 2D patch”.
… by converting color information from color planes of the corresponding 2D patch (Please see Ilola in [0116] “FIGS. 4a and 4b show extensions to the V-PCC encoder and decoder, respectively, to support mesh encoding and mesh decoding, respectively. The extensions to the V-PCC encoder and decoder are disclosed more in detail in the document MPEG M47608.” And in Ilola [0119] “In the mesh coding according to the extensions for V-PCC, as shown in FIGS. 4a and 4b, the vertices are directly mapped to color values in image textures, i.e. each geometry value will have one occupancy value and one color attribute value. The shading (i.e. applying color values to faces of a mesh) is carried out by interpolating color values from the vertices of each triangle”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize color information from color planes as taught by Ilola with the system of Smolic taught in Claim 1 of connecting a 2D patch in a geometry component image with coordinate vertices in the local coordinate system of the 3D sub-block. Color values are one of various attributes of geometry values that are being converted.
As per claim 27, Smolic teaches the claimed:
The computer-implemented method of claim 21, wherein decoding the geometry component bitstream is performed via lossy video encoding (Smolic in the middle of page 250 “Additionally, encoders (both single rate and progressive) can be classified into lossy and lossless encoders …But because the mesh itself is intended to approximate a surface in R3, it is often beneficial to choose a different mesh that also approximates the surface but can be compressed much better. The encoder first transforms the mesh into another mesh – a process called remeshing – and compresses the resulting mesh afterwards. Those encoders are lossy because the decoder reconstructs the modified mesh and not the original mesh”).
Claims 2 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Smolic in view of Poularakis, Zhang, Ilola, Nadro, and in further view of Graziosi et al. (Pub No. US 20210295566 A1)
As per claim 2, Smolic alone does not explicitly teach the remaining claim limitations.
However, Smolic in combination with Graziosi teaches the claimed:
The computer-implemented method of claim 1, wherein normalizing the coordinates of a vertex in the mesh frame of the dynamic mesh comprises:
converting the coordinates of the vertex into a positive range; and integerizing the converted coordinates to fit into a geometry bit depth (As mentioned above, Smolic teaches of encoding a dynamic mesh and integerizing the coordinates. Graziosi teaches of converting the the coordinates of the vertex into a positive range, e.g. please see Graziosi in their claim 3 where they recite: “wherein mesh voxelization includes finding a lowest vertex value below zero and shifting the mesh values so the lowest vertex value is above zero”).
The claimed integerizing step is taught when the vertex coordinates are converted using Graziosi before they are integerized using Smolic (as taught by Smolic at the bottom of page 257 to the top of page 258). In this case, when Smolic at the bottom of page 257 to the top of page 258 recites: “… Firstly, the coordinates of the vertices are quantized up to a user-given number of bits which results in integer values for each coordinate.” The user-given number of bits corresponds to the claimed “coordinates to fit into a geometry bit depth”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to convert the coordinates into a positive range as taught by Graziosi with the system of Smolic because this may simplify the numerical values for later bit depth encoding. For example, if all of the vertex values are positive, this may be simpler to encode than if the vertex values were both positive and negative.
As per claim 10, this claim is similar in scope to limitations recited in claim 2, and thus is rejected under the same rationale.
Claims 5 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Smolic in view of Poularakis, Zhang, Ilola, Nadro, and in further view of Ramasubramonian et al. (Pub No. US 2021/0327097 A1).
As per claim 5, Smolic alone does not explicitly teach the claimed limitations.
However, Smolic in combination with Ramasubramonian teaches the claimed:
The computer-implemented method of claim 1, wherein the video bit depth is specified in a sequence parameter set or a geometry sequence parameter set of the geometry component bitstream (Please see Ramasubramonian in [0204] “In some examples, the G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) may store the output of the scaling operation at a higher bit depth than the input. This higher bit depth may be pre-determined, or it may be derived using the global scale parameters. For example, if the bit depth of the coordinates before global scaling is N, and the precision of the global scale refinement value is M, the internal bit depth may be set as M+N”. In this passage, the “bit depth” corresponds to the claimed “video bit depth” and the “global scale parameters” corresponds to the claimed “sequence parameter set”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the video bit depth to be specified in a sequence parameter set as taught by Ramasubramonian with the system of Smolic in order to better scale the encoder to more closely match the global scale of the data being represented.
As per claim 13, this claim is similar in scope to limitations recited in claim 5, and thus is rejected under the same rationale.
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over Smolic in view of Poularakis, Zhang, Ilola, Nadro, and in further view of Zakharchenko et al. (Pub No. US 20220232234 A1).
As per claim 22, Smolic alone does not explicitly teach the claimed limitations.
However, Smolic in combination with Zakharchenko teaches the claimed:
The computer-implemented method of claim 21, wherein the integerization parameter is decoded from a bitstream header of the geometry component bitstream (Zakharchenko in [0132] “… Wavelet transforms, integer transforms, sub-band transforms or other types of transforms could also be used … The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the transform scaling and quantization component 213 may then perform a scan of the matrix including the quantized transform coefficients. The quantized transform coefficients are forwarded to the header formatting and CABAC component 231 to be encoded in the bitstream.”).
Zakharchenko further teaches of decoding the bitstream, e.g. please see the abstract).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the integerization parameter from a bitstream header for decoding as taught by Poularakis with the system of Smolic. This helps to reduce bit depth amounts for the encoded information (Zakharchenko in [0132]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SARAH Y. LEE whose telephone number is (571)272-8374. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F. Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
SARAH Y. LEE
Examiner
Art Unit 2616
/DANIEL F HAJNIK/Supervisory Patent Examiner, Art Unit 2616