DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 18 objected to because of the following informalities:
The subject “the simulation device” lacks antecedent basis. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 6, 10, and 14-18 are rejected under 35 U.S.C. 103 as being unpatentable over Davis et al. (US 20080247636 A1, hereinafter “Davis”) in view of Kim (Pope Kim. 2012. Screen space decals in Warhammer 40,000: Space Marine. In ACM SIGGRAPH 2012 Talks (SIGGRAPH '12). Association for Computing Machinery, New York, NY, USA, Article 6, 1. https://doi.org/10.1145/2343045.2343053, hereinafter “Kim”) and Proenca et al. (Proença, Pedro F., and Yang Gao. “Probabilistic Combination of Noisy Points and Planes for RGB-D Odometry.” Towards Autonomous Robotic Systems, edited by Yang Gao et al., Springer International Publishing, 2017, pp. 340–50, https://doi.org/10.1007/978-3-319-64107-2_27., hereinafter “Proenca”)
Regarding claim 18,
Davis teaches:
A computing device for performing a three-dimensional (3D) simulation (Davis: Claim 15, “A system for interactive virtual inspection of modeled objects. . .”; ¶17, “. . . a three-dimensional (3D) model of an object of interest also referred to herein as a virtual object, is utilized for inspection and analysis. An operator interacts with the virtual object. . .”),
the simulation device comprising: a memory; and a processor, wherein the processor is configured to (Davis: Claim 15, “. . . comprising: a processor; a display device coupled to said processor; a memory device; and program code resident in said memory device, said program code executable by said processor to run. . .”):
generate, based on viewpoint information about a viewpoint from which a 3D object is viewed, first depth information in a pixel unit corresponding to the 3D object (Davis: ¶6, “. . . interactive virtual inspection of modeled objects comprising acquiring a three-dimensional model of a modeled object; displaying a first view of the modeled object; identifying by a user, a location of interest on a surface of the modeled object that is visible within the first view. . .”; ¶27, “. . . a range map 126 can be used to map points back and forth between the 3D model 120 and a corresponding image 122. For example, as shown, a first range map 126A provides pixel-by-pixel range data that maps image points in the first image 122A to corresponding locations on the 3D model. . .”);
and generate a marker based on a user input (Davis: Abstract, “. . . a user to identify locations of interest on a surface of the modeled object visible within the first view. The user enters information to create a markup tag that annotates the location of interest, and the markup tag is automatically associated with the location of interest on the modeled object. . .”; NOTE: The generated marker is the created markup tag from the user interaction with the 3d model).
Although Davis generates a marker on the 3D model surface, Davis fails to teach: generate, based on the reference plane, a marker curved surface covering the 3D object; and generate a marker on the marker curved surface. (NOTE: Davis’s marker (markup tag) is generated directly to the 3d model object, and not on a surface covering the 3D object.)
The analogous art Kim teaches:
generate, based on the reference plane, a marker curved surface covering the 3D object; and generate a marker on the marker curved surface (Kim: Section 1, “. . . projects a convex volume mesh onto the underlying geometry, stored in a depth buffer, and applies a volume texture on the pixels rasterized onto the screen. . .”; Section 2, “. . . Once the local positions are calculated, we run two rejection passes: out-of-box rejection and orientation-based rejection. . . reject any pixels which are facing away more than a certain angle. . . allows different decals to have different stretching tolerance”; NOTE: Kim teaches generating markers, such as decals, over a convex volume mesh covering the underlying geometry of a 3D model. The convex volume mesh constituting a curved surface. Kim further teaches the boundaries of a volume mesh as reference planes to reject pixels which are outside the boundaries, or reject pixels facing away more than a certain angle. Therefore the volume mesh is generated based on the planes boundaries rejecting pixels outside or facing more than a certain angle which leaves non-rejected pixels)
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Davis, and Kim to include: generate, based on the reference plane, a marker curved surface covering the 3D object; and generate a marker on the marker curved surface.
The reason for doing so is to improve the speed of collision detection pass since “there was no need for high-resolution collision meshes to support the duplication of fine mesh patches”.
However, the combination of Davis and Kim fails to teach: generate a reference plane based on a predetermined statistical value by assigning a weight to the first depth information.
The analogous art Proenca teaches:
generate a reference plane based on a predetermined statistical value by assigning a weight to the first depth information (Proenca: Abstract: “. . . Depth measurement uncertainty is modelled and propagated through the extraction of geometric primitives to the frame-to-frame motion estimation, where pose is optimized by weighting the residuals of 3D point and planes matches. . .”; Section 3: “The proposed system, outlined in Fig. 1, starts by detecting points and planes from an RGB-D frame. Samples from the noisy depth map are used to obtain 3D points, through back-projection, and the 3D points corresponding to detected planes are used in turn to estimate the plane parameters through a weighted least squares framework, which takes into account the depth measurement uncertainties. Once, point and plane matches are found between two adjacent frames, pose is estimated in iteratively reweighted least squares by minimizing both the point and plane residuals, according to their uncertainties. For this purpose, uncertainty is propagated throughout this process. . . .”; Section 5, “. . . Planes are first detected by using the method of [3], which processes efficiently organized point clouds in real-time, yielding a segmentation output, . . .”; Section 5.1, “. . . It is useful to express planes as infinite planes in the hessian normal form: θ={Nx,Ny,Nz,d}. . . . Similarly to [16], we use a minimal plane representation: θm=[Nx,Ny,Nz]/d, as an intermediate parameterization. Since, a plane with d=0 implies detecting a plane . . . estimated by minimizing the point-to-plane distances through the following weighted least-squares problem: where the scaling weights were chosen to be the inverse of the point depth uncertainties” NOTE: Proenca’s method generates a reference plane based on predetermined statistical value (depth measurement uncertainty from sensor) by assigning weights to the depth information (3D points) using a weighted least squares framework.
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Davis, Kim, and Proenca, and include: generate a reference plane based on a predetermined statistical value by assigning a weight to the first depth information.
The reason for doing so is to model “the uncertainty of the 3D points and planes in order to estimate optimally the camera pose”, “due to the systematic noise of these depth sensors” (Proenca Section 1) and is beneficial, “when few image feature points are detected either due to non-textured planar surfaces or blur caused by sudden motion” (Proenca: Section 9).
Regarding method claim 1, method claim 1 is drawn to the method corresponding to the instructions of using same as claimed in apparatus claim 18. Therefore, method claim 1 corresponds to the instructions in the apparatus of claim 18 and is rejected for the same reasons of obviousness as used above.
Regarding CRM claim 17, CRM claim 17 is drawn to the CRM corresponding to the instructions of using same to perform the method of claim 1. Therefore, CRM claim 17 corresponds to the instructions to perform the method of claim 1 and is rejected for the same reasons of obviousness as used above.
Regarding claim 2, depending on 1,
The combination of Davis, Kim, and Proenca teaches:
The method of claim 1,
Davis further teaches:
wherein the first depth information in a pixel unit is information determined based on a distance between a viewpoint in the viewpoint information for each pixel and the 3D object (Davis: ¶27m “. . . a range map 126 can be used to map points back and forth between the 3D model 120 and a corresponding image 122. For example, as shown, a first range map 126A provides pixel-by-pixel range data that maps image points in the first image 122A to corresponding locations on the 3D model. The resolution of the range map 126A is the same as the first image 122A. As such, range data is provided in the range map 126A for each pixel in the first image 122A. Moreover, the unique range data corresponding to each pixel of the first image 122A provides surface location and depth dimension information required to establish a one-to-one mapping between its corresponding image pixel and an associated location on the 3D model. . .”; ¶36, “. . . orientation of the 3D model to identify a 2D coordinate taken from the point of view of the camera, and from which the ray is projected,. . .”).
Regarding claim 3, depending on 1,
The combination of Davis, Kim, and Proenca teaches:
The method of claim 1,
Davis further teaches:
wherein the generating of the first depth information in a pixel unit comprises
generating first depth information comprising a depth value in a pixel unit of a first view space (Davis: ¶27, “. . . range map 126A provides pixel-by-pixel range data . . . the unique range data corresponding to each pixel of the first image 122A provides surface location and depth dimension information required to establish a one-to-one mapping between its corresponding image pixel and an associated location on the 3D model. . .”; NOTE: The first view is the image 122A taken by the camera to include the object of interest. The depth value generated is the depth dimension of each pixel mapped to the corresponding image pixel and location on the 3D model. The first view space is the view with the 3D model, such as shown in Fig. 5)
generated through view transform in a depth texture (NOTE: In paragraph 38, rays are projected from the point of view to identify the intersection of the ray and a surface of the 3D model, transforming the per-pixel information in a depth texture of the 3d object, which is Davis’s range map associated with the location of the 3D model. The range map 126A generated is the depth texture associated to the 3D model because it contains pixel by pixel range data from the viewpoint defining the depth texture of the 3D model. In applicant’s paragraph 55, “The first depth texture may have a depth value for each pixel in an image including the 3D object”. Therefore the claimed depth texture is the range map associated to the 3D model of an object.),
wherein the first view space is space in which a viewpoint comprised in the viewpoint information is an origin (Davis: ¶38, “. . . to determining the point of intersection between a projected ray and a surface of the 3D model. . . projected rays originating from the point of view of the camera. . .”; NOTE: The point of view of the camera is the origin because that is the original viewpoint where an image is captured.).
Regarding claim 6, depending on 1,
The combination of Davis, Kim, and Proenca teaches:
The method of claim 1,
However Davis fails to teach: wherein the generating of the marker curved surface comprises generating, based on the first depth information, a marker curved surface based on pixels closer to a viewpoint than the reference plane.
Kim teaches:
wherein the generating of the marker curved surface comprises generating, based on the first depth information, a marker curved surface based on pixels closer to a viewpoint than the reference plane (NOTE: See the rejection of claim 1, 18. Kim generates a marker curved surface constituting the convex volume mesh covering the 3d object. In Section 2, Kim discloses that the projection mesh is based on depth buffer information to find the location of each pixel. It then rejects pixels that are out of the boundary planes of the projection mesh, and also rejects pixels that are facing away at a certain angle. Because the decal marker is drawn in the front, the pixels corresponding to the marker curved surface where a decal marker is drawn are closer to a viewpoint than the reference plane where the other pixels are rejected such as pixels that are behind the model, not visible to the viewpoint.).
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Davis, and Kim to include: wherein the generating of the marker curved surface comprises generating, based on the first depth information, a marker curved surface based on pixels closer to a viewpoint than the reference plane.
The reason for doing so is to solve Z-fighting between mesh planes when the underlying mesh has sparse vertices (Kim, Introduction Section 1).
Regarding claim 10, depending on 1,
The combination of Davis, Kim, and Proenca teaches:
The method of claim 1,
Although Davis teaches using smoothing process of surface points corresponding to the surface of a 3D object, which then a marker (markup tag) is placed on the surface. However, Davis marker is applied directly on the 3D surface of the object, and not a marker curved surface covering the 3D object (Davis: ¶57, “. . . Various averaging or smoothing processes may be used to accomplish such stitching, with appropriate hue, saturation and/or luminance adjustments being made to the adjoined data as appropriate. The combined maps may be associated with geometrically corrected surface points rendered as a solid model from 3D dimensional measurements. . .”).
Kim teaches drawing a decal, constituting a marker on a second layer mesh surface, which is the convex volume mesh covering the underlying geometry of a 3d object as discussed in the rejection of claim 1.
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Davis, Kim, and Proenca, applying a smoothing process on Kim’s generated marker curved surface (convex volume mesh) and include: wherein the generating of the marker curved surface comprises smoothing the marker curved surface.
The reason for doing so is “so that the resulting data file is harmonized to 3D dimensions and 2D surface mapping, and the graphical presentation provides a virtual 3D solid model of the actual appearance of the tested object” (Davis: ¶57).
Regarding claim 14, depending on 1,
The combination of Davis, Kim, and Proenca teaches:
The method of claim 1, further comprising:
Davis further teaches:
changing, based on a user input, at least one of a position of a marker or a marker identifier (Davis: ¶62, “As shown in the main window 208, the user has designated a location 218 on the view that shows a feature of interest by inserting onto the view, a plurality of control points 220 that outline the feature of interest. The location 218 may be designated by a markup geometry defined, for example, by a line, area or volume so as to encompass a feature of interest. Predefined markup region shapes and/or sizes may be selectable, e.g., via a menu and/or the operator may have the flexibility to draw the markup location 218 by freehand drawing the control points 220 directly onto the 2D or 3D image, such as via a mouse command. In this regard, tools may be provided to allow the operator to move, insert or remove control points, such as by clicking and dragging control points with a mouse or other input device.”; ¶66, “. . . the operator can set the properties of the markups, such as by providing general information such as a scan identification, markup identification, inspector, measurement types”).
Regarding claim 15, depending on 1,
The combination of Davis, Kim, and Proenca teaches:
The method of claim 1, further comprising:
when the generated marker is occluded by the 3D object because a viewpoint is changed (Davis: ¶32, “. . . when operating on a 3D view of the 3D model, a feature of interest may not be entirely visible in a given orientation of the 3D view. The entire feature may be identified by rotating the 3D view into different orientations to enable markup of the feature in a piecewise fashion as different segments of the feature are visible in different orientations. . .”; NOTE Davis’s system allows interaction with the 3d model such as rotating the 3d model. This rotation makes the marker occluded from view because the object is rotated and the marker rotated along with the object and is now positioned behind.),
displaying the generated marker based on the changed viewpoint and allowing a user input for the generated marker (Davis: ¶68-69, “These markups are then directly converted and coalesced into the 3D model and can thus be visualized on corresponding 2D and 3D views of the virtual object. Moreover, the markups may be displayed in 3D views of the virtual object regardless of whether textural features of the 3D model are also displayed. As noted more fully herein, as the operator changes views, markups are projected to those views where the corresponding location of interest is captured within the field of view of the corresponding image. As such, the location 218 and corresponding markup 222 created by the operator with respect to a representation of the virtual object in a first view, e.g., the active view in the main window 208, is automatically conveyed to each view having at least a portion of the identified image visible. That is, a markup region defined in any view is automatically configured into the 3D virtual object model and is conveyed (and thus displayable) in any other 2D or 3D view. For example, as seen in the thumbnail window 210, the identified location 218 in the view in the main window 208 is automatically conveyed to each thumbnail view in the thumbnail window 210 that includes a view of at least a portion of the identified location 218. Moreover, the associated markup 222 is automatically conveyed to those views. If the operator selects a thumbnail having the location 218 for viewing, the operator can inspect the associated markup 222 without having to re-enter the markup inspection results. . . Moreover, the location of interest 218 on the 3D model may not be completely viewable from a single 2D image. For example, contours, corners and other geometries may obscure the entire feature of interest from the 2D view in the main window 208. As such, the location 218 on the 3D model may be edited, revised and/or otherwise modified by interaction with multiple views,. . .”),
Regarding claim 16, depending on 1,
The combination of Davis, Kim, and Proenca teaches:
The method of claim 1, further comprising:
Davis further teaches:
displaying an annotation on the generated marker (Davis: ¶17, “. . . the operator may identify a location of interest in a particular 2D or 3D view, which may correspond to a specific feature, region, area or other aspect of the virtual object and associate a markup, e.g., annotation, tag, metadata, etc.. . . “; ¶21, “. . . a markup tag of user-defined information is created at 108 that annotates the location of interest. . .”; ¶49, “. . . The operator creates a markup tag at 190 having information that annotates the location of interest selected by the user in the current view and the markup tag created by the operator is associated with the designated global coordinate points of the 3D model. . .”; ¶65, “. . . The operator may then annotate the location 218 with a markup that describes the location.. . .” )
Claim 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Davis in view of Kim and Proenca further in view of Tang et al. (US 20160048726 A1, hereinafter “Tang”).
Regarding claim 4, depending on 1,
The combination of Davis, Kim, and Proenca teaches:
The method of claim 1,
Although Proenca teaches generating a reference plane using weighted least squares, the combination of Davis, Kim, and Proenca fails to disclose: wherein the generating of the reference plane comprises: calculating an average value by assigning a weight to the first depth information in a pixel unit; and generating the reference plane based on the average value.
The analogous art Tang teaches:
calculating an average value by assigning a weight to the first depth information in a pixel unit (Tang: ¶13, “. . . a weighted average of the depth values of the pixels. The weights are decided based on how far a pixel is to the origin, . .”; ¶57, “. . . As may be seen, Eqn. 4 computes a weighted center of mass based on the various weights, and then moves the candidate hand tracking box to the newly-computed center of mass location. In some embodiments, an iterative process is used, wherein, at the current reference pixel location, a depth value average of surrounding pixels is taken. . . “; NOTE: The assigned weights to the pixel unit is the distance on how far a pixel is to the origin.);
and generating the reference plane based on the average value (Tang: ¶76, “. . . FIG. 8 illustrates an exemplary located hand candidate 820 inside bounding box 810 that is at an xy-plane extrema of the depth map 800. The extrema in FIG. 8 was found by tracking along path 840 towards the user's hand. . .”; NOTE: The generated reference plane is the xy-plane that is positioned at the hand.).
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Davis, Kim, Proenca, and Tang, and include: wherein the generating of the reference plane comprises: calculating an average value by assigning a weight to the first depth information in a pixel unit; and generating the reference plane based on the average value.
The reason for doing so is because “many applications, e.g., touchless drawing, user interaction control, and sign language interaction, may benefit from the introduction of a robust hand tracking application that is able to find and track the three-dimensional location of hands throughout images in a video stream, as well as retain the identification (ID) of particular hands within the scene if there are multiple hands. More detailed applications of hand trackers, such as pose and gesture detections (e.g., by analyzing the positions of individual fingers on a hand) are also possible” (Tang: ¶38).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Davis in view of Kim and Proenca further in view of Tang further in view of Tegzes et al. (US 9979863 B2, hereinafter “Tegzes”)
Regarding claim 5, depending on 1,
The combination of Davis, Kim, and Proenca teaches:
The method of claim 1,
Although Proenca teaches generating a reference plane using weighted least squares, the combination of Davis, Kim, and Proenca fails to disclose: wherein the generating of the reference plane comprises calculating an average value by assigning weights.
The analogous art Tang teaches:
wherein the generating of the reference plane comprises calculating an average value by assigning weights (NOTE: See the rejection of claim 4.)
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Davis, Kim, Proenca, and Tang, and include: wherein the generating of the reference plane comprises calculating an average value by assigning weights.
The reason for doing so is because “many applications, e.g., touchless drawing, user interaction control, and sign language interaction, may benefit from the introduction of a robust hand tracking application that is able to find and track the three-dimensional location of hands throughout images in a video stream, as well as retain the identification (ID) of particular hands within the scene if there are multiple hands. More detailed applications of hand trackers, such as pose and gesture detections (e.g., by analyzing the positions of individual fingers on a hand) are also possible” (Tang: ¶38).
Although Davis teaches the viewpoint information including the image taken generating a 3D model of an object for viewing, and although Tang generates a reference plane based on weighted average of pixel depth values, and although Tang discloses in paragraph 13 that the weights are decided based on how far the pixel is to the origin, both fails to disclose: assigning weights that gradually decrease from a central region to an edge region in an image obtained.
The analogous art Tegzes teaches:
assigning weights that gradually decrease from a central region to an edge region in an image obtained (Tegzes: col 2 lines 35-40, “. . . spatial filtering with weights modulated according to a distance from the center of a region of interest. . . “; col 4 lines 65-67 to col 5 lines 1-5, “. . . parameter weighting can be that the center of interest falls in the center of the image, and parameter weighting depends only on the geometrical distance (R) of the processed pixel from this center. For example, if R is smaller than a given radius R1, a weak filtering is applied; and if R is larger than another radius R2, a strong filtering is applied. . .”, Claim 1: “. . . weighted averaging of individual pixels over consecutive frames of the image with weights determined by weight parameters that decrease as a distance from a center of the area of interest increases and threshold parameters that increase as the distance from the center of the area of interest increases.. . . “; NOTE: The weight given to a pixel with a distance R from the center is decreases as R increases, weight decreases as the pixel gets closer to the edge of the image, lowest weight is the pixels on the edge of the image. Weight increases as R of a pixel as R gets close to the central region. Therefore the weights given to a pixel is higher the closer it is from the center and gradually decreases the farther the pixel is from the center.)
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Davis, Kim, Proenca, Tang, and Tegzes and include: assigning weights that gradually decrease from a central region to an edge region in an image obtained based on the viewpoint information.
The reason for doing so is “to selectively modulate the intensity of noise filtering to ensure a stronger noise reduction around the image edges than in the area of interest” (Tegzes: col 3 lines 62-65).
Claims 7-9, and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Davis in view of Kim and Proenca further in view of Guo (US 20210225067 A1, hereinafter “Guo”).
Regarding claim 7, depending on 1,
The combination of Davis, Kim, and Proenca teaches:
The method of claim 1,
Although Kim teaches generating a marker curved surface based on the reference plane as discussed in the rejection of claim 1, Kim fails to teach: generating a second depth texture using a projection matrix redefined based on the reference plane.
The analogous art Guo teaches generating multiple depth textures based on clip planes, particularly depth texture information of a near clip plane, and a far clip plane, corresponding to a first and second depth textures by vertex transformation from a view space to a clip space using a projection matrix (Guo ¶75, “. . . transform vertex coordinates from a 3D scene space to a 2D screen space through coordinate transformation. The transformation process may be implemented by a using clip matrix. The clip matrix may be also referred to as a projection matrix. . . A goal of the clip space is to be able to clip a primitive conveniently. A primitive located inside the space is retained, a primitive located outside the space is removed, . . . Among 6 clip planes of the view frustum, a clip plane closest to the camera is referred to as a near clip plane, and a clip plane farthest from the camera is referred to as a far clip plane. The near clip plane and the far clip plane determine a depth range that the camera can see. . .”; :NOTE: Guo creates a clipping space to generate multiple depth textures such as the depth map corresponding to the near clip plane, and the depth map of a second depth texture corresponding to the far clip plane. The clip space is based on vertex transformation using clip matrix, or projection matrix. The planes corresponding to the clip space are the clip planes are the reference planes where primitives outside the clip space are clipped, and primitives inside the clip space are retained. The projection matrix is redefined when primitives are clipped outside the reference planes boundaries, particularly the primitives behind the far clip plane.)
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Davis, Kim, Proenca, and Guo and include: wherein the generating of the marker curved surface comprises generating a second depth texture using a projection matrix redefined based on the reference plane.
The reason for doing so is “to reduce the calculation and processing amounts required for the rendering as much as possible, and improve the rendering efficiency” (Guo: ¶75).
Regarding claim 8, depending on 7,
The combination of Davis, Kim, Proenca, and Guo teaches:
The method of claim 7,
Guo further teaches:
generating second depth information comprising a depth value in a pixel unit of a second view space generated through view transform in the second depth texture (Guo: ¶76, “. . . a coordinate transformation . . . the vertex coordinates are converted . . . a principle of geometric transformation. . . . the terminal may use a principle of similar triangles to first calculate position coordinates, in a far clip plane, corresponding to the screen space pixel, and then calculate the position coordinates, in the scene space, corresponding to the screen space pixel according to the position coordinates in the far clip plane and a scene depth. As shown in FIG. 13, an example of calculating position coordinates PosB, in a scene space, corresponding to a screen space pixel A is taken, PosB=PosA*DepthB, where PosA is position coordinates of the screen space pixel A in the far clip plane, and DepthB is the scene depth. . .”; NOTE: The scene depth, which is the second depth texture associated to the far clip plane comprise depth values (pixel. . . position coordinates).)
Regarding claim 9, depending on 8,
The combination of Davis, Kim, Proenca, and Guo teaches:
The method of claim 8,
Guo further teaches:
wherein the second depth information comprises a depth value determined based on the reference plane and a viewpoint (Guo: ¶75, “. . . The clip space is determined by a view frustum of a camera. . . Among 6 clip planes of the view frustum, a clip plane closest to the camera is referred to as a near clip plane, and a clip plane farthest from the camera is referred to as a far clip plane. The near clip plane and the far clip plane determine a depth range that the camera can see. NOTE: The far clip plane corresponds to the reference plane clipping primitives outside that plane boundary and is used as a reference to determine the depth range values relative to the camera. The clip space defined by the clip planes is based on the viewpoint of the camera, which is the view frustum of a camera).
Regarding claim 12, depending on 1,
The combination of Davis, Kim, and Proenca teaches:
The method of claim 10,
However, Davis fails to disclose that the smoothing process is a Gaussian blur, and therefore, the combination of Davis, Kim, and Proenca fails to teach: wherein the smoothing of the marker curved surface comprises processing Gaussian blur on the marker curved surface.
The analogous art Guo further teaches a DrawMesh command applying a Gaussian Blur as illustrated in Fig. 12.
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Davis, Kim, Proenca, and Guo and include: wherein the smoothing of the marker curved surface comprises processing Gaussian blur on the marker curved surface.
The reason for doing so is to predictably smooth the convex volume mesh, constituting to the marker curved surface via Gaussian blurring; and to “improve processing efficiency” (Guo: ¶65).
Claims 11, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Davis in view of Kim and Proenca further in view of Wang et al. (US 20200082554 A1, hereinafter “Wang”).
Regarding claim 11, depending on 10,
The combination of Davis, Kim, and Proenca teaches:
The method of claim 10,
As discussed in the rejection of claim 1, Kim generates a convex volume mesh covering the underlying geometry of a 3d object. This is discussed in section 1 of Kim that directly addresses the problem of Z-fighting that causes flickering between the underlying geometry mesh and a marker mesh, such as a decal mesh. The solution is to create a volume mesh covering the 3D object so when the decal marker is applied to the boundaries of the convex volume covering the geometry, the decal marker can be displayed solving Z-fighting.
However, Kim’s convex volume mesh is positioned at the exact pixel depth of the 3D model and rejects the pixel that is “out-of-the box” or pixels facing away at a certain angle discussed in section 2.
Therefore, the combination Davis, Kim, and Proenca fails to teach: increasing second depth information in a pixel unit at a predetermined rate.
The analogous art Wang teaches:
increasing second depth information in a pixel unit at a predetermined rate (Wang: ¶6, “. . . processing three-dimensional (3D) data,. . .”; ¶7, “. . . the first depth map subject to the expansion processing to obtain a second depth map matched to the three-dimensional point cloud data. . .”; ¶78, “. . . the expansion processing module 303 may be configured to construct a subrounded structure element with a predetermined scale parameter to perform the expansion processing on the sparse depth map. . .”; ¶28, “The expansion processing may be implemented as follows. . . . consisted of pixels . . . , so as to enlarge the set of points containing the depth value and corresponding to the target object O in the sparse depth map”; NOTE: Paragraph 56-58 discloses implementation of a Gaussian filtering process for smoothing. ¶56-58, “. . . smooth each edge of the image subject to the expansion processing . . . bilateral filtering is implemented by optimizing Gaussian filtering . . .”).
It would have been obvious to try to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Davis, Kim, and Proenca, and Wang to include: wherein the smoothing of the marker curved surface comprises increasing second depth information in a pixel unit at a predetermined rate.
The reason for doing so “to optimize a conversional method for processing 3D data, such that an accuracy of 3D point cloud data is improved “ (Wang:,¶6).
Regarding claim 13, depending on 10,
As discussed in the rejection of claim 1, Kim generates a convex volume mesh covering the underlying geometry of a 3d object. This is discussed in section 1 of Kim that directly addresses the problem of Z-fighting that causes flickering between the underlying geometry mesh and a marker mesh, such as a decal mesh. The solution is to create a volume mesh covering the 3D object so when the decal marker is applied to the boundaries of the convex volume covering the geometry, the decal marker can be displayed solving Z-fighting.
However, Kim’s convex volume mesh is positioned at the exact pixel depth of the 3D model and rejects the pixel that is “out-of-the box” or pixels facing away at a certain angle discussed in section 2.
Therefore, the combination Davis, Kim, and Proenca fails to teach: processing Gaussian blur on a result obtained by increasing second depth information in a pixel unit at a predetermined rate.
The analogous art Wang teaches:
processing Gaussian blur on a result obtained by increasing second depth information in a pixel unit at a predetermined rate. (Wang: ¶6, “. . . processing three-dimensional (3D) data,. . .”; ¶7, “. . . the first depth map subject to the expansion processing to obtain a second depth map matched to the three-dimensional point cloud data. . .”; ¶78, “. . . the expansion processing module 303 may be configured to construct a subrounded structure element with a predetermined scale parameter to perform the expansion processing on the sparse depth map. . .”; ¶28, “The expansion processing may be implemented as follows. . . . consisted of pixels . . . , so as to enlarge the set of points containing the depth value and corresponding to the target object O in the sparse depth map”; ¶56-58, “. . . smooth each edge of the image subject to the expansion processing . . . bilateral filtering is implemented by optimizing Gaussian filtering . . .”NOTE: Paragraph 56-58 discloses implementation of a Gaussian filtering process for smoothing. The expanded or increased pixel points are processed with a Gaussian filtering, which is Gaussian blurring,).
It would have been obvious to try to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Davis, Kim, and Proenca, and Wang to include: wherein the smoothing of the marker curved surface comprises processing Gaussian blur on a result obtained by increasing second depth information in a pixel unit at a predetermined rate.
The reason for doing so “to optimize a conversional method for processing 3D data, such that an accuracy of 3D point cloud data is improved “ (Wang:,¶6).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PATRICK GALERA whose telephone number is (571)272-5070. The examiner can normally be reached Mon-Fri 0800-1700 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at 571-270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PATRICK P GALERA/Examiner, Art Unit 2617 /KING Y POON/Supervisory Patent Examiner, Art Unit 2617
Prosecution Insights