Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-4, 6-7, 11-17, 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ren et al. (US 20220300681) in view of Qian et al. (“Deep Magnification-Flexible Upsampling over 3D Point Clouds”).
Re claim 1, Ren teaches method comprising: receiving, at a device, a plurality of different surfaces that collectively form a three-dimensional (3D) asset ([0021] In the present disclosure, a “complete point cloud object scan” refers to a point cloud corresponding to an object scanned from more than one location such that multiple surfaces of the object are represented in the point cloud. A “dense” point cloud refers to a point cloud corresponding to one or more surfaces of an object in which the number of points per area unit of the surface is relatively high. A “surface model” refers to a three-dimensional model of one or more surfaces of an object; the surface(s) may be represented as polygons, points, texture maps, and/or any other means of representing three-dimensional surfaces.
generating a different plurality of 3D primitives at the dynamic resolution across each surface of the plurality of different surfaces ([0059] The points of the point cloud frame 100 are clustered in space where light emitted by the lasers of the LIDAR sensor are reflected by objects in the environment, thereby resulting in clusters of points corresponding to the surface of the object visible to the LIDAR sensor. A first cluster of points 112 corresponds to reflections from a car. In the example point cloud frame 100, the first cluster of points 112 is enclosed by a bounding box 122 and associated with an object class label, in this case the label “car” 132. A second cluster of points 114 is enclosed by a bounding box 122 and associated with the object class label “bicyclist” 134, and a third cluster of points 116 is enclosed by a bounding box 122 and associated with the object class label “pedestrian” 136. Each point cluster 112, 114, 116 thus corresponds to an object instance: an instance of object class “car”, “bicyclist”, and “pedestrian” respectively. The entire point cloud frame 100 is associated with a scene type label 140 “intersection” indicating that the point cloud frame 100 as a whole corresponds to the environment near a road intersection (hence the presence of a car, a pedestrian, and a bicyclist in close proximity to each other).
Ren does not explicitly teach determining a dynamic resolution at which to render the 3D asset based on available resources of the device and presenting a visualization of the 3D asset at the dynamic resolution by rendering the different plurality of 3D primitives generated across each surface of the plurality of different surfaces in place of the plurality of different surfaces.
However, Qian teaches determining a dynamic resolution at which to render the 3D asset based on available resources of the device (see p. 8355: “Upsampling raw point clouds with various upsampling factors is common in point cloud processing. For example, the input point clouds captured by different sensors may have different resolutions. Therefore, the user may have to upsample each of them with different factors to obtain the desired resolution. The user may also determine the upsampling factor based on resource constraints, such as display, computing power, and transmission bandwidth. Besides, the desired point cloud resolution varies with application scenarios. For example, a high-resolution point cloud is highly expected for surface reconstruction, while a moderately sparse one is tolerated for object detection.”
and presenting a visualization of the 3D asset at the dynamic resolution by rendering the different plurality of 3D primitives generated across each surface of the plurality of different surfaces in place of the plurality of different surfaces (see p. 8355, Fig. 1, wherein a sparse input point cloud of different surfaces is rendered into a 3d asset with reconstructed surface at different density point cloud resolutions).
Ren in view of Qian teaches claim 1. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ren’s 3d reconstruction system with captured 3d scene data including point clouds to explicitly include generating the 3D asset at the dynamic resolution by rendering different 3d primitives, as taught by Qian, as the references are in the analogous art of capturing and processing point cloud 3d data. An advantage of the modification is that it achieves the result of allowing for a scalable resolution of a 3d object at varying resolutions, as well as improving the resolution/upscaling based on different resource factors.
Re claim 2, Ren in view of Qian teaches claim 1. Furthermore, Qian teaches receiving a point cloud representation of the 3D asset at a first resolution, wherein the point cloud representation comprises a first plurality of points; replacing different sets of the first plurality of points that collectively form a parametrized shape with a different surface from the plurality of different surfaces that recreates the parameterized shape; and streaming the plurality of different surfaces in place of the first plurality of points in response to the plurality of different surfaces representing the 3D asset with less data than the first plurality of points (see p. 8354, in reference to Fig. 1, wherein a plurality of 3d point clouds at a first resolution such as 8x and a replacement resolution such as 4x point cloud having less data then the first plurality of points). For motivation, see claim 1.
Re claim 3, Ren in view of Qian teaches claim 2. Furthermore, Qian teaches wherein generating the different plurality of 3D primitives comprises:
generating a second plurality of points across the plurality of different surfaces at the dynamic resolution that is greater than the first resolution, wherein the second plurality of points comprises more points than the first plurality of points (see p. 8354, in reference to Fig. 1, wherein a plurality of 3d point clouds at a first resolution such as 8x and a replacement resolution such as 12x point cloud having more point data then the first plurality of points). For motivation, see claim 1.
Re claim 4, Ren in view of Qian teaches claim 1. Furthermore, Qian teaches wherein the dynamic resolution is greater than a first resolution at which the 3D asset is originally defined, and wherein the different plurality of 3D primitives at the dynamic resolution comprises more 3D primitives than the first resolution at which the 3D asset is originally defined (see p. 8354, in reference to Fig. 1, wherein a plurality of 3d point clouds at a first resolution such as 8x and a replacement resolution such as 12x point cloud having more point data then the first plurality of points. The resolutions are more than the original sparse input point cloud). For motivation, see claim 1.
Re claim 6, Ren in view of Qian teaches claim 1. Furthermore, Qian teaches detecting a change in the available resources; increasing from the dynamic resolution to a second resolution in response to detecting the change in the available resources; and generating additional 3D primitives amongst each of the different plurality of 3D primitives that are generated at the dynamic resolution across each surface of the plurality of different surfaces until the 3D primitives across each surface of the plurality of different surfaces reaches the second resolution (see p. 8355: “Upsampling raw point clouds with various upsampling factors is common in point cloud processing. For example, the input point clouds captured by different sensors may have different resolutions. Therefore, the user may have to upsample each of them with different factors to obtain the desired resolution. The user may also determine the upsampling factor based on resource constraints, such as display, computing power, and transmission bandwidth. Besides, the desired point cloud resolution varies with application scenarios. For example, a high-resolution point cloud is highly expected for surface reconstruction, while a moderately sparse one is tolerated for object detection”) and (see p. 8355, Fig. 1, wherein a sparse input point cloud of different surfaces is rendered into a 3d asset with reconstructed surface at different density point cloud resolutions, such that based on the resource constraints, a higher resolution rendering can be performed from a more dense point cloud). For motivation, see claim 1.
Re claim 7, Ren in view of Qian teaches claim 1. Furthermore, Qian teaches wherein generating the different plurality of 3D primitives at the dynamic resolution comprises:
generating a first set of 3D primitives across a particular surface of the plurality of different surfaces at first resolution; and adding a second set of 3D primitives in between the first set of 3D primitives and across the particular surface until the dynamic resolution is reached (see p. 8355, in reference to Fig. 1-2, 8 wherein new points of a 3d point cloud are generated to allow for upsampling point clouds with local neighborhood information, as shown in Fig. 1-2, 8). For motivation, see claim 1.
Re claim 11, Ren and Qian teaches claim 1. Furthermore, Qian teaches replacing each surface of the plurality of different surfaces with the different plurality of 3D primitives at the dynamic resolution that is generated across that surface (see p. 8355, Fig. 1, wherein a sparse input point cloud of different surfaces is rendered into a 3d asset with reconstructed surface at different density point cloud resolutions across the surfaces). For motivation, see claim 1.
Re claim 12, Ren and Qian teaches claim 1. Furthermore, Qian teaches wherein each surface of the plurality of different surfaces is defined as an equation, and wherein each 3D primitive of the different plurality of 3D primitives that is generated for a particular surface of the plurality of different surfaces is a defined with a discrete position in a 3D space and color values that are presented at that discrete position (see 8364-8365, Fig. 15 shows more visual results of real world data upsampled by the proposed method. Here we also displayedthe associated colors of the point clouds for better visualization purposes. Particularly, the color attributes of newly upsampled points are kept identical to the closest points in the sparse
input. From Fig. 15, it can be seen that the quality of upsampled point clouds gradually improves with the upsampling factor increasing, i.e., more geometry details exhibit) and (see abstract, and p. 8356-8359, equations for defining surfaces and generation of reconstructed surfaces). For motivation, see claim 1.
Re claim 13, Ren and Qian teaches claim 1. Furthermorre, Qian teaches changing from the dynamic resolution to a second resolution; generating a different number of 3D primitives across each surface of the plurality of different surfaces for the second resolution than for the dynamic resolution; and presenting a different visualization of the 3D asset at the second resolution by rendering the different number of 3D primitives generated across each surface of the plurality of different surfaces (see p. 8354, in reference to Fig. 1, wherein a plurality of 3d point clouds at a first resolution such as 8x and a replacement resolution such as 12x point cloud having more point data then the first plurality of points. The resolutions are more than the original sparse input point cloud). For motivation, see claim 1.
Claims 14 claim limitations in scope to claim 1 and is rejected for at least the reasons above.
Claims 15-17 claims limitations in scope to claims 2-4 and is rejected for at least the reasons above.
Claim 19 claims limitations in scope to claim 6 and is rejected for at least the reasons above.
Claims 20 claim limitations in scope to claim 1 and is rejected for at least the reasons above.
Claim(s) 5, 9-10, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ren et al. (US 20220300681) in view of Qian et al. (“Deep Magnification-Flexible Upsampling over 3D Point Clouds”) and Park et al. (“Template-based Reconstruction of Surface Mesh Animation from Point Cloud Animation”).
Re claim 5, Ren and Qian teaches claim 1. Ren and Qian do not explicitly teach
receiving an animation that is defined for a particular surface of the plurality of different surfaces; determining the different plurality of 3D primitives that are generated across the particular surface; and animating the different plurality of 3D primitives to maintain a position about the particular surface as the particular surface is moved throughout the animation.
However, Park teaches receiving an animation that is defined for a particular surface of the plurality of different surfaces; determining the different plurality of 3D primitives that are generated across the particular surface; and animating the different plurality of 3D primitives to maintain a position about the particular surface as the particular surface is moved throughout the animation (see p. 1010-1011, in reference to Fig. 1 and 4, wherein an input point cloud with 3d points is used in the reconstruction of an animation about the surface, as the reconstructed mesh animation moves through the animation of a shuffle dance).
Ren, Qian, and Park teaches claim 5. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ren and Qian’s point based rendering system to explicitly receive animation defining surfaces for animation, as taught by Park, as the references are in the analogous art of 3d point cloud primitives used for rendering. An advantage of the modification is that it achieves the result of using point cloud primitives to aid in animating a plurality of surfaces that make up an object, such as a human model animation while maintaining certain primitive shapes during animation for consistency, as taught by Park.
Re claim 9, Ren and Qian teaches claim 1. Ren and Qian do not explicitly teach receiving an animation framework comprising a plurality of animation elements and different animations specified for the plurality of animation elements at different times; determining the different plurality of 3D primitives that are generated across a particular surface of the plurality of different surfaces; determining that the particular surface is linked to a particular animation element of the plurality of animation elements; and animating the different plurality of 3D primitives that are generated across the particular surface based on a different animation that is specified for the particular animation element
However, Park teaches receiving an animation framework comprising a plurality of animation elements and different animations specified for the plurality of animation elements at different times; determining the different plurality of 3D primitives that are generated across a particular surface of the plurality of different surfaces; determining that the particular surface is linked to a particular animation element of the plurality of animation elements; and animating the different plurality of 3D primitives that are generated across the particular surface based on a different animation that is specified for the particular animation element (see p. 1010-1013, in reference to Fig. 1 and 4, wherein an input point cloud with 3d points is animated about the surface and a reconstructed mesh animation is generated based on 3d primitive data, as the surface moves through the animation of a shuffle dance at different times). Ren and Qian and Park teach claim 9. For motivation, see claim 5.
Re claim 10, Ren, Qian, and Park teach claim 9. Furthermore, Park teaches wherein animating the different plurality of 3D primitives comprises: maintaining a position of each 3D primitive from the different plurality of 3D primitives relative to the particular animation element as the particular animation element moves according to the different animation that is specified for the particular animation element (see p. 1010-1013, in reference to Fig. 1 and 4, wherein an input point cloud with 3d points is animated about the surface and a reconstructed mesh animation is generated based on 3d primitive data, as the surface moves through the animation of a shuffle dance at different times, and maintains positions relative to the animation). For motivation, see claim 5.
Claim 18 claims limitations in scope to claim 5 and is rejected for at least the reasons above.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ren et al. (US 20220300681) in view of Qian et al. (“Deep Magnification-Flexible Upsampling over 3D Point Clouds”) and Bonatto et al. (“Explorations for Real-Time Point Cloud Rendering of Natural Scenes in Virtual Reality”).
Re claim 8, Ren in view of Qian teaches claim 1. Furthermore, Qian is not relied upon to explicitly teach generating the visualization from a first distance by rendering the different plurality of 3D primitives generated across each surface of the plurality of different surfaces; and generating the visualization from a second distance by rendering the different plurality of 3D primitives with additional 3D primitives across each surface of the plurality of different surfaces in response to gaps appearing between the different plurality of 3D primitives from the second distance.
However, Bonatto teaches generating the visualization from a first distance by rendering the different plurality of 3D primitives generated across each surface of the plurality of different surfaces; and generating the visualization from a second distance by rendering the different plurality of 3D primitives with additional 3D primitives across each surface of the plurality of different surfaces in response to gaps appearing between the different plurality of 3D primitives from the second distance (see P. 6, Fig. 9, wherein Splats are used to fill up holes for better visual rendering), (see p. 2, in reference to Fig. 1-2, wherein Splatting with surfels primitives aids in response to gaps), (see p. 5, 6.2. Level of Detail: This technique is a powerful tool to modulate the workload of a graphics system in real-time [21]. The principle is to create
different versions of the same object, each one with a different resolution. In the case of point cloud rendering, the different versions- or levels- represent the same object with different point densities… At run-time, the most appropriate LOD is chosen for each leaf [17]. This choice can be made depending on the distance to the view point, the size of the object, the eccentricity within the display device or even the relative velocity between the object and the viewer), and (see pp. 5-6, in reference to Fig. 7-8, wherein in Fig. 7, the left close distance image is a denser image than the middle far away image).
Ren in view of Qian and Bonatto teaches claim 8. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ren and Qian’s point cloud-based rendering system to explicitly include rendering based on a first distance and a second different distance having additional primitives in response to gaps appearing, as taught by Bonatto, as the references are in the analogous art of cloud point-based rendering. An advantage of the modification is that it achieves the result of taking into account distance/depth of a user’s view to render surfaces, providing more/less detail based on the needs of the system, thus better utilizing the resources of the system.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Peter Hoang whose telephone number is (571)270-1346. The examiner can normally be reached Monday-Friday 8:00 am - 5:00 pm PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hajnik F. Daniel can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PETER HOANG/ Primary Examiner, Art Unit 2616
Prosecution Insights