Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-18 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Schwarz et al. (US 20230068178 A1).
Regarding claim 1, Schwarz et al. discloses a calculation method comprising:
obtaining three-dimensional points that represent an object in a space on a computer, each of the three-dimensional points indicating a position on the object (In 3D point clouds, each point of each 3D surface is described as a 3D point with color and/or other attribute information such as surface normal or material reflectance. Point cloud is a set of data points (i.e. locations) in a coordinate system, for example in a three-dimensional coordinate system being defined by X, Y, and Z coordinates. The points may represent an external surface of an object in the screen space, e.g. in a 3D space. A point may be associated with a vector of attributes. A point cloud can be used to reconstruct an object or a scene as a composition of the points, para. 0042);
classifying each three-dimensional point in the three-dimensional points into one of groups based on a normal direction of the each three-dimensional point (initial clustering may then be refined by iteratively updating the cluster index associated with each point based on its normal and the cluster indices of its nearest neighbors, para. 0072); and
calculating a first accuracy of each group in the groups, the first accuracy increasing with an increase of a second accuracy of at least one three-dimensional point belonging to the each group (a 3D mesh that may be compared 711 against the original input mesh to generate a mesh edge residual map 712, para. 0160, The inputs to the mesh edge evaluation is a reconstructed 3D mesh frame and its original (uncompressed) counterpart. The output of this process is a 2D frame representing the difference between the original and the reconstruction (prediction), i.e. a mesh residual map 712, para. 0162),
wherein the three-dimensional points are generated by a sensor detecting light from the object from different positions and in different directions (video may be captured using one or more three-dimensional (3D) cameras. When multiple cameras are in use, the captured footage is synchronized so that the cameras provide different viewpoints to the same world. In contrast to traditional two-dimensional/tree-dimensional (2D/3D) video, volumetric video describes a 3D model of the world where the viewer is free to move and observe different parts of the world, para. 0037), and
the normal direction of each three-dimensional point in the three-dimensional points is determined based on the different directions used to generate the each three-dimensional point (At the initial stage of the patch generation 102, a normal per each point is estimated. The tangent plane and its corresponding normal are defined per each point, based on the point's nearest neighbors m within a predefined search distance. A k-dimensional tree may be used to separate the data and find neighbors in a vicinity of a point p.sub.i and a barycenter c=p of that set of points is used to define the normal, para. 0061).
Regarding claim 2, Schwarz et al. discloses the calculation method according to claim 1, wherein the classifying includes: dividing the space into subspaces; and
classifying each three-dimensional point in the three-dimensional points into the one of the groups based on the normal direction of the each three-dimensional point and a subspace in the subspaces that includes the each three-dimensional point (initial clustering may then be refined by iteratively updating the cluster index associated with each point based on its normal and the cluster indices of its nearest neighbors, para. 0072).
Regarding claim 3, Schwarz et al. discloses the calculation method according to claim 2, wherein a first subspace in the subspaces includes a first three-dimensional point and a second three-dimensional point included in the three-dimensional points, a line extending in a normal direction of the first three-dimensional point intersects a first face among faces defining the first subspace, and the classifying includes classifying the first three-dimensional point and the second three-dimensional point into a same group when a line extending in a normal direction of the second three-dimensional point intersects the first face (Based on this information, each point is associated with a corresponding plane of a point cloud bounding box. Each plane is defined by a corresponding normal. More precisely, each point may be associated with the plane that has the closest normal (i.e. maximizes the dot product of the point normal), see paras. 0063-0072.
Regarding claim 4, Schwarz et al. discloses the calculation method according to claim 1, wherein
the sensor is any one or combination of a LiDAR sensor, a depth sensor, and an image sensor (Volumetric video can be rendered from synthetic 3D animations, reconstructed from multi-view video using 3D reconstructing techniques such as structure from motion, or captured with a combination of cameras and depth sensors such as LiDAR, for example, para. 0038).
Regarding claim 5, Schwarz et al. discloses the calculation method according to claim 1, wherein
for each three-dimensional point in the three-dimensional points, a composite direction of the different directions used to generate the each three-dimensional point is opposite to the normal direction of the each three-dimensional point (Examiner articulates that there is always an occurrence wherein a composite direction of the different directions used to generate the each three-dimensional point is opposite to the normal direction of the each three-dimensional point.).
Regarding claim 6, Schwarz et al. discloses the calculation method according to claim 2, wherein each of the groups corresponds to any one of faces defining the subspaces,
the classifying includes classifying each three-dimensional point in the three-dimensional points into the one of the groups that corresponds to, among the faces defining the subspace including the each three-dimensional point, an intersecting face that intersects a line extending in the normal direction of the each three-dimensional point, and the calculation method further comprises displaying the intersecting face in a color according to the first accuracy of the group corresponding to the intersecting face (volumetric video data represents a three-dimensional scene or object, and can be used as input for augmented reality (AR), virtual reality (VR) and mixed reality (MR) applications. Such data describes geometry (shape, size, position in 3D space) and respective attributes (e.g., color, opacity, reflectance, . . . ). In addition, the volumetric video data can define any possible temporal changes of the geometry and attributes at given time instances (such as frames in 2D video), para. 0039).
Regarding claim 7, Schwarz et al. discloses the calculation method according to claim 6, wherein the displaying includes displaying the intersecting face in one color when the first accuracy of the group corresponding to the intersecting face is greater than or equal to a predetermined accuracy and in a different color when the first accuracy of the group corresponding to the intersecting face is less than the predetermined accuracy (volumetric video data represents a three-dimensional scene or object, and can be used as input for augmented reality (AR), virtual reality (VR) and mixed reality (MR) applications. Such data describes geometry (shape, size, position in 3D space) and respective attributes (e.g., color, opacity, reflectance, . . . ). In addition, the volumetric video data can define any possible temporal changes of the geometry and attributes at given time instances (such as frames in 2D video), para. 0039).
Regarding claim 8, Schwarz et al. discloses the calculation method according to claim 2, wherein each of the groups corresponds to any one of the subspaces, and the calculation method further comprises displaying each subspace in the subspaces in a color according to the first accuracy of the group corresponding to the each subspace (volumetric video data represents a three-dimensional scene or object, and can be used as input for augmented reality (AR), virtual reality (VR) and mixed reality (MR) applications. Such data describes geometry (shape, size, position in 3D space) and respective attributes (e.g., color, opacity, reflectance, . . . ). In addition, the volumetric video data can define any possible temporal changes of the geometry and attributes at given time instances (such as frames in 2D video), para. 0039).
Regarding claim 9, Schwarz et al. discloses the calculation method according to claim 8, wherein the displaying includes displaying, among the subspaces, only a subspace corresponding to a group with the first accuracy less than a predetermined accuracy (volumetric video data represents a three-dimensional scene or object, and can be used as input for augmented reality (AR), virtual reality (VR) and mixed reality (MR) applications. Such data describes geometry (shape, size, position in 3D space) and respective attributes (e.g., color, opacity, reflectance, . . . ). In addition, the volumetric video data can define any possible temporal changes of the geometry and attributes at given time instances (such as frames in 2D video), para. 0039).
Regarding claim 10, Schwarz et al. discloses the calculation method according to claim 1, wherein the calculating includes, for each group in the groups, extracting a predetermined number of three-dimensional points from one or more three-dimensional points belonging to the each group, and calculating the first accuracy of the each group based on the second accuracy of each of the predetermined number of three-dimensional points extracted (The inputs to the mesh edge evaluation is a reconstructed 3D mesh frame and its original (uncompressed) counterpart. The output of this process is a 2D frame representing the difference between the original and the reconstruction (prediction), i.e. a mesh residual map 712, para. 0162).
Regarding claim 11, Schwarz et al. discloses the calculation method according to claim 1, wherein the calculating includes calculating the first accuracy of each group in the groups based on a reprojection error indicating the second accuracy of each of one or more three-dimensional points belonging to the each group (The inputs to the mesh edge evaluation is a reconstructed 3D mesh frame and its original (uncompressed) counterpart. The output of this process is a 2D frame representing the difference between the original and the reconstruction (prediction), i.e. a mesh residual map 712, para. 0162).
Regarding claim 12, Schwarz et al. discloses the calculation method according to claim 1, further comprising:
displaying the first accuracy of each group in the groups superimposed on a second three-dimensional model that is formed of at least a portion of the three-dimensional points and has a lower resolution than a first three-dimensional model formed of the three-dimensional points (volumetric video data represents a three-dimensional scene or object, and can be used as input for augmented reality (AR), virtual reality (VR) and mixed reality (MR) applications. Such data describes geometry (shape, size, position in 3D space) and respective attributes (e.g., color, opacity, reflectance, . . . ). In addition, the volumetric video data can define any possible temporal changes of the geometry and attributes at given time instances (such as frames in 2D video), para. 0039).
Regarding claim 13, Schwarz et al. discloses the calculation method according to claim 12, wherein the displaying includes displaying the first accuracy of each group in the groups superimposed on a bird's-eye view of the second three-dimensional model (volumetric video data represents a three-dimensional scene or object, and can be used as input for augmented reality (AR), virtual reality (VR) and mixed reality (MR) applications. Such data describes geometry (shape, size, position in 3D space) and respective attributes (e.g., color, opacity, reflectance, . . . ). In addition, the volumetric video data can define any possible temporal changes of the geometry and attributes at given time instances (such as frames in 2D video), para. 0039).
Regarding claim 14, Schwarz et al. discloses the calculation method according to claim 1, wherein the sensor is an image sensor, and the calculation method further comprises displaying the first accuracy of each group in the groups superimposed on an image captured by the image sensor and used to generate the three-dimensional points (volumetric video data represents a three-dimensional scene or object, and can be used as input for augmented reality (AR), virtual reality (VR) and mixed reality (MR) applications. Such data describes geometry (shape, size, position in 3D space) and respective attributes (e.g., color, opacity, reflectance, . . . ). In addition, the volumetric video data can define any possible temporal changes of the geometry and attributes at given time instances (such as frames in 2D video), para. 0039).
Regarding claim 15, Schwarz et al. discloses the calculation method according to claim 1, further comprising: displaying the first accuracy of each group in the groups superimposed on a map of a target space in which the object is located (volumetric video data represents a three-dimensional scene or object, and can be used as input for augmented reality (AR), virtual reality (VR) and mixed reality (MR) applications. Such data describes geometry (shape, size, position in 3D space) and respective attributes (e.g., color, opacity, reflectance, . . . ). In addition, the volumetric video data can define any possible temporal changes of the geometry and attributes at given time instances (such as frames in 2D video), para. 0039).
Regarding claim 16, Schwarz et al. discloses the calculation method according to claim 1, further comprising: displaying the first accuracy of each group in the groups while the sensor is detecting light from the object (volumetric video data represents a three-dimensional scene or object, and can be used as input for augmented reality (AR), virtual reality (VR) and mixed reality (MR) applications. Such data describes geometry (shape, size, position in 3D space) and respective attributes (e.g., color, opacity, reflectance, . . . ). In addition, the volumetric video data can define any possible temporal changes of the geometry and attributes at given time instances (such as frames in 2D video), para. 0039).
Regarding claim 17, Schwarz et al. discloses the calculation method according to claim 1, further comprising:
calculating a direction opposite to a composite direction of two or more normal directions of, among the three-dimensional points, two or more three-dimensional points belonging to a first group included in the groups (Examiner articulates that there is always an occurrence wherein a composite direction of the different directions used to generate the each three-dimensional point is opposite to the normal direction of the each three-dimensional point.).
Regarding claim 18, Schwarz et al. discloses acalculation device comprising:
a processor (para. 0008); and memory (para. 0008), wherein using the memory, the processor executes:
obtaining three-dimensional points that represent an object in a space on a computer, each of the three-dimensional points indicating a position on the object (In 3D point clouds, each point of each 3D surface is described as a 3D point with color and/or other attribute information such as surface normal or material reflectance. Point cloud is a set of data points (i.e. locations) in a coordinate system, for example in a three-dimensional coordinate system being defined by X, Y, and Z coordinates. The points may represent an external surface of an object in the screen space, e.g. in a 3D space. A point may be associated with a vector of attributes. A point cloud can be used to reconstruct an object or a scene as a composition of the points, para. 0042);
classifying each three-dimensional point in the three-dimensional points into one of groups based on a normal direction of the each three-dimensional point (initial clustering may then be refined by iteratively updating the cluster index associated with each point based on its normal and the cluster indices of its nearest neighbors, para. 0072); and
calculating a first accuracy of each group in the groups, the first accuracy increasing with an increase of a second accuracy of at least one three-dimensional point belonging to the each group (a 3D mesh that may be compared 711 against the original input mesh to generate a mesh edge residual map 712, para. 0160, The inputs to the mesh edge evaluation is a reconstructed 3D mesh frame and its original (uncompressed) counterpart. The output of this process is a 2D frame representing the difference between the original and the reconstruction (prediction), i.e. a mesh residual map 712, para. 0162),
wherein the three-dimensional points are generated by a sensor detecting light from the object from different positions and in different directions (video may be captured using one or more three-dimensional (3D) cameras. When multiple cameras are in use, the captured footage is synchronized so that the cameras provide different viewpoints to the same world. In contrast to traditional two-dimensional/tree-dimensional (2D/3D) video, volumetric video describes a 3D model of the world where the viewer is free to move and observe different parts of the world, para. 0037), and
the normal direction of each three-dimensional point in the three-dimensional points is determined based on the different directions used to generate the each three-dimensional point (At the initial stage of the patch generation 102, a normal per each point is estimated. The tangent plane and its corresponding normal are defined per each point, based on the point's nearest neighbors m within a predefined search distance. A k-dimensional tree may be used to separate the data and find neighbors in a vicinity of a point p.sub.i and a barycenter c=p of that set of points is used to define the normal, para. 0061).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS J LETT whose telephone number is (571)272-7464. The examiner can normally be reached Mon-Fri 9-6 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571) 272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS J LETT/Primary Examiner, Art Unit 2611
Prosecution Insights