Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 6-9, 11, 18-21, 23-26, 28 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schwarz US 20220159297 “Schwarz”, in view of FLYNN D ET AL: "G-PCC: Signalling of default attribute values", 130. MPEG MEETING; 20200420 - 20200424; ALPBACH; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11 ), no. m53681 15 April 2020 (2020-04-15), XP030287361, “FLYNN” (IDS)
Regarding claim 1, SCHWARZ discloses a method comprising encoding (SCHWARZ, abstract), in a data stream (SCHWARZ, ¶ 26), a representation (SCHWARZ, ¶ 35) of a volumetric scene ("volumetric video coding":, i.e. SCHWARZ, ¶ SCHWARZ, ¶ 33, i.e. In dense point clouds or voxel arrays, the reconstructed 3D scene may contain tens or even hundreds of millions of points. If such representations are to be stored or interchanged between entities, then efficient compression of the presentations becomes fundamental. Standard volumetric video representation formats, such as point clouds, meshes, voxel, suffer from poor temporal compression performance. Identifying correspondences for motion-compensation in 3D-space is an ill-defined problem, as both, geometry and respective attributes may change. For example, temporal successive “frames” do not necessarily have the same number of meshes, points or voxel. Therefore, compression of dynamic 3D scenes is inefficient. 2D-video based approaches for compressing volumetric data, i.e. multiview with depth, have much better compression efficiency, but rarely cover the full scene. Therefore, they provide only limited 6DOF capabilities. See, ¶ 48, fig. 2a, )) and metadata (SCHWARZ, ¶ 141, i.e. Further aspects relate to the operation of a decoder. FIG. 6 shows an example of a decoding method comprising receiving (600) a bitstream in a decoder, said bitstream comprising an encoded geometry image, texture image, occupancy map and auxiliary patch information from a 2D patch, wherein the auxiliary patch information comprises metadata relating to properties of the patch and one or more indicators for configuring reconstruction of a 3D representation of at least one object; decoding (602) the geometry image, the texture image, the occupancy map and the auxiliary patch information; and reconstructing (604) a 3D representation of said at least one object based on the decoded geometry image, texture image, occupancy map and at least one indicator for configuring reconstruction of the 3D representation of the object.) comprising a color value indicating to a renderer to set missing color values (SCHWARZ, ¶ 56, i.e. the padding process aims at filling the empty space between patches in order to generate a piecewise smooth image suited for video compression. TMC2v0 uses a simple padding strategy, which proceeds as follows: Each block of T×T (e.g., 16×16) pixels is processed independently. If the block is empty (i.e., all its pixels belong to empty space), then the pixels of the block are filled by copying either the last row or column of the previous T×T block in raster order. If the block is full (i.e., no empty pixels), nothing is done. If the block has both empty and filled pixels (i.e. a so-called edge block), then the empty pixels are iteratively filled with the average value of their non-empty neighbors.) to the color value when rendering a viewport image of the volumetric scene (SCHWARZ, ¶ 121, i.e. Table 2 discloses an example, where the Sequence parameter set syntax is enabled for a simple signalling of volume plus rendering information, wherein if sps_sequence_volume_indicator is signalled, a single RGB color value is signalled on how to fill the inside volume).
It is noted that SCHWARZ is silent about meta data as claimed.
However FLYNN discloses metadata indicating a color value to be used when color values are missing (FLYNN, sect. Proposal, "The G-PCC specification does not describe the default value of attributes in the case where attribute data is not present. Instead of requiring a decoder to decode the attribute data, it may be desirable to omit the attribute data and reconstruct using a default attribute value. This contribution proposes an SPS attribute description default attribute value ": abs; "add a default attribute value to the SPS attribute description")
Both SCHWARZ and FLYNN teach systems with video processing, and those systems are comparable to that of the instant application. Because the two cited references are analogous to the instant application, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains, to include in the SCHWARZ disclosure, to signal missing data using meta information, as taught by FLYNN. Such inclusion would have increased the usefulness of the system by avoiding the need to coding when all the attribute values are identical therefore improving the efficiency, and would have been consistent with the rationale of combining prior art elements according to known methods to yield predictable results to show a prima facie case of obviousness (MPEP 2143(I)(A)) under KSR International Co. v. Teleflex Inc., 127 S. Ct. 1727, 82 USPQ2d 1385, 1395-97 (2007).
Regarding claim 2, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the method of claim 1, wherein the color value is associated with a quality level indicating to the renderer a visibility level of visual artifacts when setting missing color values to the color value (SCHWARZ, ¶ 49. i.e. The patch generation process aims at decomposing the point cloud into a minimum number of patches with smooth boundaries, while also minimizing the reconstruction error", see SCHWARZ, ¶ 38 for minimizing patches.).
Regarding claim 3, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the method of claim 1, wherein the color value is determined as a function of color values of pixels of a multi-views image used to generate the representation of the volumetric scene (SCHWARZ, ¶ 60, i.e., … If the block has both empty and filled pixels, then the empty pixels are iteratively filled with the average value of their non-empty neighbors.).
Regarding claim 4, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the method of claim 1, wherein the metadata comprises at least two color values, each given color value being associated with a region of the volumetric scene (see SCHWARZ, i.e. ¶ 112, "Default color: Default colour to render inside volume. Such a color could either be hard coded or signalled per scene, sequence, object, frame, GOP, etc.") indicating to the renderer to use the given color value for parts of the viewport image representing the region of the volumetric scene (SCHWARZ, ¶ 139)
Regarding claim 6, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the method of claim 1 wherein the metadata further comprise a second value for a rendering (SCHWARZ, ¶ 114) attribute other than color, indicating to the renderer to set missing values (SCHWARZ, ¶ 130) for the attribute to the second value when rendering a viewport image of the volumetric scene ( SCHWARZ, ¶113 "Default transparency: Default transparency to render points inside volume.).
Regarding claim 7, SCHWARZ/FLYNN, for the same motivation of combination, discloses a method comprising: obtaining, from a data stream, a representation of a volumetric scene and metadata comprising a color value (see rejection of claim 1). ; and rendering a viewport image of the volumetric scene and setting missing color values to the color value (see rejection of claim 1).
Regarding claim 8, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the method of claim 7, wherein the color value is associated with a quality level and wherein a method to set the missing color values is selected according to the quality level. (SCHWARZ, ¶ 49. i.e. The patch generation process aims at decomposing the point cloud into a minimum number of patches with smooth boundaries, while also minimizing the reconstruction error", see SCHWARZ, ¶ 38 for minimizing patches.).
Regarding claim 9, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the method of claim 7, wherein the metadata comprises at least two color values, each given color value being associated with a region of the volumetric scene and wherein the given color value is used for parts of the viewport image representing the region of the volumetric scene (see SCHWARZ, i.e. ¶ 112, "Default color: Default colour to render inside volume. Such a color could either be hard coded or signalled per scene, sequence, object, frame, GOP, etc.", see SCHWARZ, ¶ 139 for levels of signaling)
Regarding claim 11, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the method one of claim 7, wherein the metadata further comprise a second value (SCHWARZ, ¶ 114) for a rendering attribute other than color, and wherein missing values for the attribute of the viewport image are set to the second value ( SCHWARZ, ¶113 "Default transparency: Default transparency to render points inside volume.).
Regarding claim 18, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the device comprising a memory associated to a processor configured to encode, in a data stream, a representation of a volumetric scene and metadata comprising a color value indicating to a renderer to set missing color values to the color value when rendering a viewport image of the volumetric scene (see rejection of claim 1). .
Regarding claim 19, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the device of claim 18, wherein the color value is associated with a quality level indicating to the renderer a visibility level of visual artifacts when setting missing color values to the color value (SCHWARZ, ¶ 49. i.e. The patch generation process aims at decomposing the point cloud into a minimum number of patches with smooth boundaries, while also minimizing the reconstruction error", see SCHWARZ, ¶ 38 for minimizing patches.).
Regarding claim 20, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the device of claim 18, wherein the color value is determined as a function of color values of pixels of a multi-views image used to generate the representation of the volumetric scene (SCHWARZ, ¶ 60, i.e., … If the block has both empty and filled pixels, then the empty pixels are iteratively filled with the average value of their non-empty neighbors.).
Regarding claim 21, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the device of claim 18,wherein the metadata comprises at least two color values, each given color value being associated with a region of the volumetric scene indicating to the renderer to use the given color value for parts of the viewport image representing the region of the volumetric scene (see SCHWARZ, i.e. ¶ 112, "Default color: Default colour to render inside volume. Such a color could either be hard coded or signalled per scene, sequence, object, frame, GOP, etc.", see SCHWARZ, ¶ 139 for levels of signaling).
Regarding claim 23, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the device of claim 18,wherein the metadata further comprise a second value for a rendering (SCHWARZ, ¶ 114) attribute other than color, indicating to the renderer to set missing values (SCHWARZ, ¶ 130) for the attribute to the second value when rendering a viewport image of the volumetric scene ( SCHWARZ, ¶113 "Default transparency: Default transparency to render points inside volume.).
Regarding claim 24, SCHWARZ/FLYNN, for the same motivation of combination, discloses a device comprising a memory associated with a processor configured to: obtain, from a data stream, a representation of a volumetric scene and metadata comprising a color value; and render a viewport image of the volumetric scene and set missing color values to the color value (see rejection of claim 1).
Regarding claim 25, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the device of claim 24, wherein the color value is associated with a quality level and wherein a method to set the missing color values is selected according to the quality level. (SCHWARZ, ¶ 49. i.e. The patch generation process aims at decomposing the point cloud into a minimum number of patches with smooth boundaries, while also minimizing the reconstruction error", see SCHWARZ, ¶ 38 for minimizing patches.).
Regarding claim 26, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the device of claim 24,wherein the metadata comprises at least two color values, each given color value being associated with a region of the volumetric scene and wherein the given color value is used for parts of the viewport image representing the region of the volumetric scene (see SCHWARZ, i.e. ¶ 112, "Default color: Default colour to render inside volume. Such a color could either be hard coded or signaled per scene, sequence, object, frame, GOP, etc.", see SCHWARZ, ¶ 139 for levels of signaling).
Regarding claim 28, SCHWARZ/FLYNN, for the same motivation of combination, further discloses the device of claim 24,wherein the metadata further comprise a second value for a rendering (SCHWARZ, ¶ 114) attribute other than color, and wherein missing values (SCHWARZ, ¶ 130) for the attribute of the viewport image are set to the second value ( SCHWARZ, ¶113 "Default transparency: Default transparency to render points inside volume.).
Claim(s) 5, 10, 22, and 27 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schwarz US 20220159297 “Schwarz”, in view of FLYNN D ET AL: "G-PCC: Signalling of default attribute values", 130. MPEG MEETING; 20200420 - 20200424; ALPBACH; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11 ), no. m53681 15 April 2020 (2020-04-15), XP030287361, “FLYNN” (IDS), further in view of MPEG "Text of ISO/IEC DIS 23090-5 Visual Volumetric Video-based Coding and Video-based Point Cloud Compression 2nd Edition", 135. MPEG MEETING; 20210712 - 20210716; ONLINE; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11 ), no. n20761 23 July 2021 (2021-07-23), XP030296513, “MPEG”.
Regarding claim 5, SCHWARZ /FLYNN discloses the method of claim 1.
It is noted that SCHWARZ /FLYNN is silent about SEI as claimed.
However, MPEG discloses wherein the metadata are encoded in a supplemental enhancement information message (see MPEG section F.3.12, pg. 225, i.e. use SEI to convey metadata.).
Both SCHWARZ /FLYNN and MPEG teach systems with video compression, and those systems are comparable to that of the instant application. Because the two cited references are analogous to the instant application, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains, to include in the SCHWARZ /FLYNN disclosure, encoding the meta data with the SEI, as taught by MPEG. Such inclusion would have increased the usefulness of the system by coding efficiency according with the standard, and would have been consistent with the rationale of combining prior art elements according to known methods to yield predictable results to show a prima facie case of obviousness (MPEP 2143(I)(A)) under KSR International Co. v. Teleflex Inc., 127 S. Ct. 1727, 82 USPQ2d 1385, 1395-97 (2007).
Regarding claim 10, SCHWARZ /FLYNN/MPEG, for the same motivation of combination, further discloses the method of claim 7, wherein the metadata are decoded from a Supplemental Enhancement Information message (see citation of claim 5).
Regarding claim 22, SCHWARZ /FLYNN/MPEG, for the same motivation of combination, further discloses the device of claim 18,wherein the metadata are encoded in a supplemental enhancement information message (see citation of the rejection to claim 5).
Regarding claim 27, SCHWARZ /FLYNN/MPEG, for the same motivation of combination, further discloses the device of claim 24,wherein the metadata are decoded from a Supplemental Enhancement Information message (see citation of the rejection to claim 5).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 9786077 B2 Unified image processing for combined images based on spatially co-located zones
US 20170195615 A1 MOBILE TERMINAL AND OPERATING METHOD THEREOF
US 20160337706 A1 METHOD AND APPARATUS FOR TRANSRECEIVING BROADCAST SIGNAL FOR PANORAMA SERVICE
US 20160314610 A1 IMAGE PROCESSING METHOD AND APPARATUS WITH ADAPTIVE SAMPLING
US 20150341552 A1 Developing a Panoramic Image
US 20140300693 A1 IMAGE GENERATION APPARATUS AND IMAGE GENERATION METHOD
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FRANK F HUANG whose telephone number is (571)272-0701. The examiner can normally be reached Monday-Friday, 8:30 am - 6:00 pm (Eastern Time), Federal Alternative First Friday Off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jay Patel can be reached at (571)272-2988.. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FRANK F HUANG/Primary Examiner, Art Unit 2485
Prosecution Insights