DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Status of Claims
The amendment filed 01/27/2026 has been entered. Claims 1, 12, and 20 have been amended. Claim 9 has been canceled. Claims 1-8 and 10-20 are now pending.
Joint Inventors
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Response to Arguments
Applicant’s arguments with respect to 35 USC 101 have been fully considered and are persuasive. As such, the previously presented 35 USC 101 rejections are withdrawn.
Applicant’s arguments with respect to 35 USC 102 and 35 USC 103 rejections have been fully considered, but are moot because amendments to the claim language have necessitated new grounds of rejection set forth below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-8, 12-17 and are rejected under 35 U.S.C. 103 as being unpatentable over Kaus et al. (US 20080218513 A1), hereinafter Kaus in view of Hinterstoisser (US-9868212-B1), hereinafter Hinterstoisser.
Regarding claim 1, Kaus discloses:
A method for spatially modeling a three-dimensional object within a three-dimensional workspace, the method comprising:
Obtaining… an object representative polygon mesh including a set of polygons in three dimensions, wherein the object representative polygon mesh represents a surface of the three- dimensional object (see at least [0044]: “In the following, referring to FIG. 1, a method of producing a triangulation of a surface of a physical object according to an exemplary embodiment of the present invention comprises the step of labelling voxels 101.”)
converting, with the processor, the object representative polygon mesh into an object representative triangle mesh including a set of first triangles (see at least [0046]: “Preferably, multiple connected T-junctions, e.g. T-junctions connected to multiple organs, are associated with an correct subset of neighbouring surface labels, e.g. each T-junction is associated with the correct subset of neighbouring organs. Preferably, this associating is done before decomposing the T-junctions. By using such an associating step it may be ensured to obtain afterwards a correct Delaunay-Triangulation from the Voronoi-graph.”)
subdividing, with the processor, the object representative triangle mesh into a subdivided object representative triangle mesh including a set of second triangles (see at least [0047]: “After decomposing in a next step 105 vertices from the micro mesh may be selected according to a so called thinning process. Thus, the number of vertices used for a subsequent triangulation is reduced saving processing power… From this discrete approximation of the Voronoi-graph its dual the surface Delaunay-Triangulation can be generated. By using a local curvature-adaptive thinning process the local shape of the physical object is incorporated in an early stage of the algorithm yielding an elegant method for obtaining shape-adaptive triangular meshes.”)
wherein the subdivided object representative triangle mesh is overlaid with a voxel grid including a set of voxels (see at least [0048]: “The thinning corresponds to a growing of the surface regions which are represented by the voxels. One surface region is represented by substantially all adjacent surface voxels having substantially the same outer surface orientation, i.e. can be approximated by a single triangle, without loosing a great amount of information.”);
generating, with the processor, a point collection including a plurality of points each corresponding to a voxel in the voxel grid, wherein each point is generated based on vertices of the subdivided object representative triangle mesh located in the voxels of the voxel grid (see at least [0048]: “The thinning corresponds to a growing of the surface regions which are represented by the voxels. One surface region is represented by substantially all adjacent surface voxels having substantially the same outer surface orientation, i.e. can be approximated by a single triangle, without loosing a great amount of information. The loosing of information is thereby dependent on the rate of the thinning. Preferably, the region growing is modified so that no vertex that is labelled with a first label is shifted into a region in which vertices having a different second label. Furthermore, the region growing preferably is adapted so that no vertex that is part of a triangle is shifted to a boundary edge of the boundary edge, or vice versa, i.e. no vertex that is part of a boundary edge is shifted to a triangle.”)
and generating, with the processor, based on the point collection and the voxel grid, at least one of: a surface point cloud representation of the three-dimensional object, a surface voxel representation of the three-dimensional object, or a volume voxel representation of the three-dimensional object (see at least [0060]: “After cleaning the source image a micro mesh is created. The micro mesh is the union of all surfaces that separate two voxels of different label in the source image and comprises a plurality of polygons each having a face. This micro mesh is the intermediate representation for the algorithm of the triangulation process to operate on.”)
Kaus does not explicitly disclose, but Hinterstoisser teaches:
obtaining, from one or more sensors associated with a grid of square pixels for recording images of at least a portion of the three-dimensional workspace, scan data of the three- dimensional object within the three-dimensional workspace, from which a processor, connected to the one or more sensors, generates an object representative polygon mesh including a set of polygons in three dimensions (see at least col. 17, lines 42-65: ‘As another example, in some implementations object detection and pose determination engine 140A or 140B utilizes a voxel based approach to determine whether a distance between a given scene reference point and a given additional scene point satisfies a bounding measure of an object. For example, the engine 140A or 140B may generate a voxel grid in the 3D space of the scene point cloud data, where the bound of each voxel is based on the bounding measure of the object under consideration. For example, the bound of each voxel may be the true maximum diameter (i.e., the distance between the maximum extents of the object) and/or based on the true maximum diameter (e.g., 105% of the maximum diameter). The engine 140A or 140B may store, for each voxel, an indication of all of the scene points that “fit into” the voxel and/or, for each scene point, an indication of the voxel into which it fits. For example, the stored data may indicate: a first voxel and a first group of scene points that fit into that first voxel, a second voxel and a second group of scene points that fit into that second voxel, and so forth. Also, for example, the stored data may indicate: a first scene point and the voxel into which it fits, a second scene point and the voxel into which it fits, and so forth. As used herein, a scene point fits into a voxel if the position of the scene point is within the 3D area defined by the voxel.”)
wherein each voxel in the voxel grid corresponds with a square pixel in the grid of square pixels (see at least col. 17, lines 42-65: ‘As another example, in some implementations object detection and pose determination engine 140A or 140B utilizes a voxel based approach to determine whether a distance between a given scene reference point and a given additional scene point satisfies a bounding measure of an object. For example, the engine 140A or 140B may generate a voxel grid in the 3D space of the scene point cloud data, where the bound of each voxel is based on the bounding measure of the object under consideration. For example, the bound of each voxel may be the true maximum diameter (i.e., the distance between the maximum extents of the object) and/or based on the true maximum diameter (e.g., 105% of the maximum diameter). The engine 140A or 140B may store, for each voxel, an indication of all of the scene points that “fit into” the voxel and/or, for each scene point, an indication of the voxel into which it fits. For example, the stored data may indicate: a first voxel and a first group of scene points that fit into that first voxel, a second voxel and a second group of scene points that fit into that second voxel, and so forth. Also, for example, the stored data may indicate: a first scene point and the voxel into which it fits, a second scene point and the voxel into which it fits, and so forth. As used herein, a scene point fits into a voxel if the position of the scene point is within the 3D area defined by the voxel.”)
where the at least one of the surface point cloud representation of the three- dimensional object, the surface voxel representation of the three-dimensional object, and the volume voxel representation of the three-dimensional object effects operation of machinery within the three-dimensional workspace (see at least col. 16, lines 19-30: “ In some implementations where the system is incorporated in a robot and/or is in communication with a robot, the system adjusts the robot based on determining the object is present and/or based on the determined pose of the object. For example, the system may provide one or more control commands to one or more actuators of the robot to position the robot and/or one or more operational components of the robot based on a determined pose. For instance, a pallet jack robot may drive wheels thereof autonomously and/or adjust the height of forks thereof autonomously based on a determined pose for a pallet to enable the robot to position the forks in fork holes of the pallet.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Kaus with the specific methods of object representation as taught by Hinterstoisser. This is because as stated by Hinterstoisser’s Summary: “The present disclosure is generally directed to on one or more techniques that are applicable to 3D object detection and pose determination and that may optionally increase the robustness and/or efficiency of the 3D object recognition and pose determination. As used herein, robustness refers to the ability to detect an object and/or determine the pose of an object and/or refers to the accuracy of such detection and/or determination. For example, robustness may reference the ability to detect an object and/or determine the pose of an object in challenging environments such as environments with a relatively large amount of background clutter, a relatively large amount of occlusion, etc. As used herein, efficiency refers to time and/or computer resources used in detecting an object and/or determining the pose of an object.”
Regarding claim 2, the combination of Kaus and Hinterstoisser teaches:
The method of claim 1, wherein:
the object representative polygon mesh is converted into the object representative triangle mesh by dividing each polygon in the object representative polygon mesh into a subset of first triangles of the set of first triangles (see at least [0047]: “From this discrete approximation of the Voronoi-graph its dual the surface Delaunay-Triangulation can be generated.”)
Regarding claim 3, the combination of Kaus and Hinterstoisser teaches:
The method of claim 1, wherein the subdivided object representative triangle mesh is generated based on:
subdividing, for each first triangle in the set of first triangles, the first triangle into N smaller triangles, where N is greater than 1, if the first triangle contains an edge longer than a predetermined threshold, to generate an updated triangle mesh and repeating the subdividing step for each triangle in the updated triangle mesh until each edge of every triangle is shorter than or equal to the predetermined threshold, to generate the subdivided object representative triangle mesh (see at least [0070]: “After the calculation of the local curvature the triangulation can be generated. For achieving the triangulation in a first step an appropriate gamma is determined, wherein gamma is the so-called detail balance parameter. In a descriptive way gamma can be understand as a parameter which determines how many details can be seen in the output mesh or final triangulation. For determining an appropriate gamma the triangulation is bisected till a gamma is found so that an output mesh created by utilisation of that gamma matches a predetermined target polygon count (TPC).” See further [0072-0073]: “Wherein the parameter AverageRange is defined by four times the square root of DecimationRatio, i.e. AverageRange= {square root over (DecimationRatio)}. Wherein the parameter DecimationRatio is defined by the number of the micro mesh faces divided by TPC and multiplied by the average micro mesh edge length…”)
Regarding claim 4, the combination of Kaus and Hinterstoisser teaches:
The method of claim 3, further comprising:
overlaying the subdivided object representative triangle mesh with the voxel grid; and determining a target point in each voxel of the voxel grid (see at least [0060]: “After cleaning the source image a micro mesh is created. The micro mesh is the union of all surfaces that separate two voxels of different label in the source image and comprises a plurality of polygons each having a face. This micro mesh is the intermediate representation for the algorithm of the triangulation process to operate on.”)
Regarding claim 5, the combination of Kaus and Hinterstoisser teaches:
The method of claim 4, wherein determining the target point in each voxel comprises:
randomly selecting a point in the voxel as the target point, wherein the randomly selecting is based on a uniform distribution of points over an entire volume of the voxel (see at least [0075]: “After determining gamma, R.sub.max and R.sub.min a selection and thinning can be performed as a first phase of the triangulation. In a first step all micro mesh faces are added to a so-called candidate queue (LocalFaceQueue). Then all one-point-polygons are selected from the queue. In a next step the micro mesh face which has the highest local curvature and which is left in the candidate queue is extracted and added to the selection.”)
Regarding claim 6, the combination of Kaus and Hinterstoisser teaches:
The method of claim 4, wherein generating the point collection comprises:
for each vertex in the subdivided object representative triangle mesh, identifying, from the voxel grid, a voxel encompassing the vertex (see at least [0019]: “After the decomposing of the at least one T-junction into at least one triangle and at least one two-point-polygon it can occur that a second type of T-junction occurs when more than two 2-point-polygons meet at a common voxel/vertex.”)
if the voxel is associated with a chosen point, determining whether the vertex is closer to the target point in the voxel compared to the chosen point, and in response to a determination that the vertex is closer to the target point in the voxel compared to the chosen point, setting the vertex as a new chosen point to replace the chosen point for the voxel (See at least [0019]: “After the decomposing of the at least one T-junction into at least one triangle and at least one two-point-polygon it can occur that a second type of T-junction occurs when more than two 2-point-polygons meet at a common voxel/vertex. These occurrences of T-junctions may be decomposed by detaching the 2-point-polygons from each other, inserting a 1-point-polygon at the vertex and connecting the 2-point-polygons to the newly created 1-point-polygon.”)
and if the voxel is not associated with a chosen point, setting the vertex as a chosen point for the voxel (see at least [0062]-[0063]: “This way a unique label is assigned to each combination of voxel labels, enabling to derive the information which voxel labels are being separated by the respective polygon from the surface label alone. For creation of the micro mesh only the faces A>B, i.e. polygons having a face A>B, are used. After creating the micro mesh the following steps are performed on the micro mesh. In a first step the connectivity of the polygons of the micro mesh is calculated, wherein vertices of the micro mesh know all faces that are connected to them and faces know about their neighbouring faces as well as the vertices that makes up the polygon. Thus the connectivity can be determined.”)
Regarding claim 7, the combination of Kaus and Hinterstoisser teaches:
The method of claim 6, wherein:
the surface point cloud representation of the three-dimensional object is generated based on extracting chosen points from the voxels of the voxel grid, wherein the chosen points form the surface point cloud representation of the three-dimensional object (see at least [0082]: “After finishing the selection and the thinning an adaptive refinement can be performed as the second phase of the triangulation. In the adaptive refinement phase all selected faces become region seed points from which simultaneous region growing is performed until the entire surface is covered. The result of this region growing is the Voronoi-graph approximation. From this Voronoi-graph its dual the Delaunay-Triangulation can be created which represent the output mesh.”)
the surface voxel representation of the three-dimensional object is generated based on marking, in the point collection, each voxel that has a chosen point, wherein the marked voxels form the surface voxel representation of the three-dimensional object (see at least [0082]: “After finishing the selection and the thinning an adaptive refinement can be performed as the second phase of the triangulation. In the adaptive refinement phase all selected faces become region seed points from which simultaneous region growing is performed until the entire surface is covered. The result of this region growing is the Voronoi-graph approximation. From this Voronoi-graph its dual the Delaunay-Triangulation can be created which represent the output mesh.”)
and the volume voxel representation of the three-dimensional object is generated based on expanding, starting from a corner of the voxel grid, the voxel grid along axis directions to generate an expanded voxel grid that forms the volume voxel representation of the three- dimensional object (see at least [0082]: “After finishing the selection and the thinning an adaptive refinement can be performed as the second phase of the triangulation. In the adaptive refinement phase all selected faces become region seed points from which simultaneous region growing is performed until the entire surface is covered. The result of this region growing is the Voronoi-graph approximation. From this Voronoi-graph its dual the Delaunay-Triangulation can be created which represent the output mesh.”)
Regarding claim 8, the combination of Kaus and Hinterstoisser teaches:
The method of claim 1, further comprising:
for each first triangle in the set of first triangles, subdividing the first triangle into N smaller triangles, where N is greater than 1, if the first triangle contains an edge longer than a predetermined threshold, repeating the subdividing step for each of the N smaller triangles until each edge of every triangle generated from the first triangle is shorter than or equal to the predetermined threshold; (see at least [0070]: “After the calculation of the local curvature the triangulation can be generated. For achieving the triangulation in a first step an appropriate gamma is determined, wherein gamma is the so-called detail balance parameter. In a descriptive way gamma can be understand as a parameter which determines how many details can be seen in the output mesh or final triangulation. For determining an appropriate gamma the triangulation is bisected till a gamma is found so that an output mesh created by utilisation of that gamma matches a predetermined target polygon count (TPC).” See further [0072-0073]: “Wherein the parameter AverageRange is defined by four times the square root of DecimationRatio, i.e. AverageRange= {square root over (DecimationRatio)}. Wherein the parameter DecimationRatio is defined by the number of the micro mesh faces divided by TPC and multiplied by the average micro mesh edge length…”)
overlaying each triangle generated from the first triangle with one or more voxels in the voxel grid (see at least [0019]: “After the decomposing of the at least one T-junction into at least one triangle and at least one two-point-polygon it can occur that a second type of T-junction occurs when more than two 2-point-polygons meet at a common voxel/vertex.”)
and for each voxel overlaying a triangle generated from the first triangle, randomly selecting a target point in the voxel, and determining a chosen point in the voxel based on a vertex of the triangle that is closer to the target point compared to any other vertex in the voxel (see at least [0075]: “After determining gamma, R.sub.max and R.sub.min a selection and thinning can be performed as a first phase of the triangulation. In a first step all micro mesh faces are added to a so-called candidate queue (LocalFaceQueue). Then all one-point-polygons are selected from the queue. In a next step the micro mesh face which has the highest local curvature and which is left in the candidate queue is extracted and added to the selection.”)
Regarding claim 12, Kaus discloses:
A system for spatially modeling a three-dimensional object within a three-dimensional workspace, the system comprising:
a non-transitory memory having instructions stored thereon and at least one processor operatively coupled to the non-transitory memory and the one or more sensors, and configured to read the instructions (see at least [0010]: “Further, an exemplary embodiment relates to a computer readable medium in which a program for producing a triangulation of a surface of a physical object is stored which program comprises the following method steps, if executed by a processor.”) to:
obtain… an object representative polygon mesh including a set of polygons in three dimensions, wherein the object representative polygon mesh represents a surface of the three-dimensional object (see at least [0044]: “In the following, referring to FIG. 1, a method of producing a triangulation of a surface of a physical object according to an exemplary embodiment of the present invention comprises the step of labelling voxels 101.”)
convert the object representative polygon mesh into an object representative triangle mesh including a set of first triangles (see at least [0046]: “Preferably, multiple connected T-junctions, e.g. T-junctions connected to multiple organs, are associated with an correct subset of neighbouring surface labels, e.g. each T-junction is associated with the correct subset of neighbouring organs. Preferably, this associating is done before decomposing the T-junctions. By using such an associating step it may be ensured to obtain afterwards a correct Delaunay-Triangulation from the Voronoi-graph.”)
subdivide the object representative triangle mesh into a subdivided object representative triangle mesh including a set of second triangles (see at least [0047]: “After decomposing in a next step 105 vertices from the micro mesh may be selected according to a so called thinning process. Thus, the number of vertices used for a subsequent triangulation is reduced saving processing power… From this discrete approximation of the Voronoi-graph its dual the surface Delaunay-Triangulation can be generated. By using a local curvature-adaptive thinning process the local shape of the physical object is incorporated in an early stage of the algorithm yielding an elegant method for obtaining shape-adaptive triangular meshes.”)
wherein the subdivided object representative triangle mesh is overlaid with a voxel grid including a set of voxels (see at least [0048]: “The thinning corresponds to a growing of the surface regions which are represented by the voxels. One surface region is represented by substantially all adjacent surface voxels having substantially the same outer surface orientation, i.e. can be approximated by a single triangle, without loosing a great amount of information.”)
generate a point collection including a plurality of points each corresponding to a voxel in the voxel grid, wherein each point is generated based on vertices of the subdivided object representative triangle mesh located in the voxels of the voxel grid (see at least [0048]: “The thinning corresponds to a growing of the surface regions which are represented by the voxels. One surface region is represented by substantially all adjacent surface voxels having substantially the same outer surface orientation, i.e. can be approximated by a single triangle, without loosing a great amount of information. The loosing of information is thereby dependent on the rate of the thinning. Preferably, the region growing is modified so that no vertex that is labelled with a first label is shifted into a region in which vertices having a different second label. Furthermore, the region growing preferably is adapted so that no vertex that is part of a triangle is shifted to a boundary edge of the boundary edge, or vice versa, i.e. no vertex that is part of a boundary edge is shifted to a triangle.”)
and generate, based on the point collection and the voxel grid, at least one of:
a surface point cloud representation of the three-dimensional object, a surface voxel representation of the three-dimensional object, or a volume voxel representation of the three-dimensional object (see at least [0060]: “After cleaning the source image a micro mesh is created. The micro mesh is the union of all surfaces that separate two voxels of different label in the source image and comprises a plurality of polygons each having a face. This micro mesh is the intermediate representation for the algorithm of the triangulation process to operate on.”)
Kaus does not explicitly disclose, but Hinterstoisser teaches:
one or more sensors associated with a grid of square pixels for recording images of at least a portion of the three-dimensional workspace (see at least col. 17, lines 42-65: ‘As another example, in some implementations object detection and pose determination engine 140A or 140B utilizes a voxel based approach to determine whether a distance between a given scene reference point and a given additional scene point satisfies a bounding measure of an object. For example, the engine 140A or 140B may generate a voxel grid in the 3D space of the scene point cloud data, where the bound of each voxel is based on the bounding measure of the object under consideration. For example, the bound of each voxel may be the true maximum diameter (i.e., the distance between the maximum extents of the object) and/or based on the true maximum diameter (e.g., 105% of the maximum diameter). The engine 140A or 140B may store, for each voxel, an indication of all of the scene points that “fit into” the voxel and/or, for each scene point, an indication of the voxel into which it fits. For example, the stored data may indicate: a first voxel and a first group of scene points that fit into that first voxel, a second voxel and a second group of scene points that fit into that second voxel, and so forth. Also, for example, the stored data may indicate: a first scene point and the voxel into which it fits, a second scene point and the voxel into which it fits, and so forth. As used herein, a scene point fits into a voxel if the position of the scene point is within the 3D area defined by the voxel.”)
obtain, from the one or more sensors, scan data of the three-dimensional object within the three-dimensional workspace and generate from the scan data an object representative polygon mesh including a set of polygons in three dimensions (see at least col. 17, lines 42-65: ‘As another example, in some implementations object detection and pose determination engine 140A or 140B utilizes a voxel based approach to determine whether a distance between a given scene reference point and a given additional scene point satisfies a bounding measure of an object. For example, the engine 140A or 140B may generate a voxel grid in the 3D space of the scene point cloud data, where the bound of each voxel is based on the bounding measure of the object under consideration. For example, the bound of each voxel may be the true maximum diameter (i.e., the distance between the maximum extents of the object) and/or based on the true maximum diameter (e.g., 105% of the maximum diameter). The engine 140A or 140B may store, for each voxel, an indication of all of the scene points that “fit into” the voxel and/or, for each scene point, an indication of the voxel into which it fits. For example, the stored data may indicate: a first voxel and a first group of scene points that fit into that first voxel, a second voxel and a second group of scene points that fit into that second voxel, and so forth. Also, for example, the stored data may indicate: a first scene point and the voxel into which it fits, a second scene point and the voxel into which it fits, and so forth. As used herein, a scene point fits into a voxel if the position of the scene point is within the 3D area defined by the voxel.”)
wherein each voxel in the voxel grid corresponds with a square pixel in the grid of square pixels (see at least col. 17, lines 42-65: ‘As another example, in some implementations object detection and pose determination engine 140A or 140B utilizes a voxel based approach to determine whether a distance between a given scene reference point and a given additional scene point satisfies a bounding measure of an object. For example, the engine 140A or 140B may generate a voxel grid in the 3D space of the scene point cloud data, where the bound of each voxel is based on the bounding measure of the object under consideration. For example, the bound of each voxel may be the true maximum diameter (i.e., the distance between the maximum extents of the object) and/or based on the true maximum diameter (e.g., 105% of the maximum diameter). The engine 140A or 140B may store, for each voxel, an indication of all of the scene points that “fit into” the voxel and/or, for each scene point, an indication of the voxel into which it fits. For example, the stored data may indicate: a first voxel and a first group of scene points that fit into that first voxel, a second voxel and a second group of scene points that fit into that second voxel, and so forth. Also, for example, the stored data may indicate: a first scene point and the voxel into which it fits, a second scene point and the voxel into which it fits, and so forth. As used herein, a scene point fits into a voxel if the position of the scene point is within the 3D area defined by the voxel.”)
and a volume voxel representation of the three- dimensional object, where the at least one of the surface point cloud representation of the three-dimensional object, the surface voxel representation of the three-dimensional object, and the volume voxel representation of the three-dimensional object effects operation of machinery within the three-dimensional workspace (see at least col. 16, lines 19-30: “ In some implementations where the system is incorporated in a robot and/or is in communication with a robot, the system adjusts the robot based on determining the object is present and/or based on the determined pose of the object. For example, the system may provide one or more control commands to one or more actuators of the robot to position the robot and/or one or more operational components of the robot based on a determined pose. For instance, a pallet jack robot may drive wheels thereof autonomously and/or adjust the height of forks thereof autonomously based on a determined pose for a pallet to enable the robot to position the forks in fork holes of the pallet.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Kaus with the specific methods of object representation as taught by Hinterstoisser. This is because as stated by Hinterstoisser’s Summary: “The present disclosure is generally directed to on one or more techniques that are applicable to 3D object detection and pose determination and that may optionally increase the robustness and/or efficiency of the 3D object recognition and pose determination. As used herein, robustness refers to the ability to detect an object and/or determine the pose of an object and/or refers to the accuracy of such detection and/or determination. For example, robustness may reference the ability to detect an object and/or determine the pose of an object in challenging environments such as environments with a relatively large amount of background clutter, a relatively large amount of occlusion, etc. As used herein, efficiency refers to time and/or computer resources used in detecting an object and/or determining the pose of an object.”
Regarding claim 13, the combination of Kaus and Hinterstoisser teaches:
The system of claim 12, wherein the subdivided object representative triangle mesh is generated based on:
subdividing, for each first triangle in the set of first triangles, the first triangle into N smaller triangles, where N is greater than 1, if the first triangle contains an edge longer than a predetermined threshold, to generate an updated triangle mesh; and repeating the subdividing step for each triangle in the updated triangle mesh until each edge of every triangle is shorter than or equal to the predetermined threshold, to generate the subdivided object representative triangle mesh (see at least [0070]: “After the calculation of the local curvature the triangulation can be generated. For achieving the triangulation in a first step an appropriate gamma is determined, wherein gamma is the so-called detail balance parameter. In a descriptive way gamma can be understand as a parameter which determines how many details can be seen in the output mesh or final triangulation. For determining an appropriate gamma the triangulation is bisected till a gamma is found so that an output mesh created by utilisation of that gamma matches a predetermined target polygon count (TPC).” See further [0072-0073]: “Wherein the parameter AverageRange is defined by four times the square root of DecimationRatio, i.e. AverageRange= {square root over (DecimationRatio)}. Wherein the parameter DecimationRatio is defined by the number of the micro mesh faces divided by TPC and multiplied by the average micro mesh edge length…”)
Regarding claim 14, the combination of Kaus and Hinterstoisser teaches:
The system of claim 13, wherein the at least one processor is further configured to read the instructions to:
overlay the subdivided object representative triangle mesh with the voxel grid (see at least [0060]: “After cleaning the source image a micro mesh is created. The micro mesh is the union of all surfaces that separate two voxels of different label in the source image and comprises a plurality of polygons each having a face. This micro mesh is the intermediate representation for the algorithm of the triangulation process to operate on.”)
and determine a target point in each voxel of the voxel grid, by randomly selecting a point in the voxel as the target point based on a uniform distribution of points over an entire volume of the voxel (see at least [0075]: “After determining gamma, R.sub.max and R.sub.min a selection and thinning can be performed as a first phase of the triangulation. In a first step all micro mesh faces are added to a so-called candidate queue (LocalFaceQueue). Then all one-point-polygons are selected from the queue. In a next step the micro mesh face which has the highest local curvature and which is left in the candidate queue is extracted and added to the selection.”)
Regarding claim 15, the combination of Kaus and Hinterstoisser teaches:
The system of claim 13, wherein the point collection is generated based on:
for each vertex in the subdivided object representative triangle mesh, identifying, from the voxel grid, a voxel encompassing the vertex (see at least [0019]: “After the decomposing of the at least one T-junction into at least one triangle and at least one two-point-polygon it can occur that a second type of T-junction occurs when more than two 2-point-polygons meet at a common voxel/vertex.”)
if the voxel is associated with a chosen point, determining whether the vertex is closer to the target point in the voxel compared to the chosen point, and in response to a determination that the vertex is closer to the target point in the voxel compared to the chosen point, setting the vertex as a new chosen point to replace the chosen point for the voxel (See at least [0019]: “After the decomposing of the at least one T-junction into at least one triangle and at least one two-point-polygon it can occur that a second type of T-junction occurs when more than two 2-point-polygons meet at a common voxel/vertex. These occurrences of T-junctions may be decomposed by detaching the 2-point-polygons from each other, inserting a 1-point-polygon at the vertex and connecting the 2-point-polygons to the newly created 1-point-polygon.”)
and if the voxel is not associated with a chosen point, setting the vertex as a chosen point for the voxel (see at least [0062]-[0063]: “This way a unique label is assigned to each combination of voxel labels, enabling to derive the information which voxel labels are being separated by the respective polygon from the surface label alone. For creation of the micro mesh only the faces A>B, i.e. polygons having a face A>B, are used. After creating the micro mesh the following steps are performed on the micro mesh. In a first step the connectivity of the polygons of the micro mesh is calculated, wherein vertices of the micro mesh know all faces that are connected to them and faces know about their neighbouring faces as well as the vertices that makes up the polygon. Thus the connectivity can be determined.”)
Regarding claim 16, the combination of Kaus and Hinterstoisser teaches:
The system of claim 12, wherein:
the surface point cloud representation of the three-dimensional object is generated based on extracting chosen points from the voxels of the voxel grid, wherein the chosen points form the surface point cloud representation of the three-dimensional object (see at least [0082]: “After finishing the selection and the thinning an adaptive refinement can be performed as the second phase of the triangulation. In the adaptive refinement phase all selected faces become region seed points from which simultaneous region growing is performed until the entire surface is covered. The result of this region growing is the Voronoi-graph approximation. From this Voronoi-graph its dual the Delaunay-Triangulation can be created which represent the output mesh.”)
the surface voxel representation of the three-dimensional object is generated based on marking, in the point collection, each voxel that has a chosen point, wherein the marked voxels form the surface voxel representation of the three-dimensional object (see at least [0082]: “After finishing the selection and the thinning an adaptive refinement can be performed as the second phase of the triangulation. In the adaptive refinement phase all selected faces become region seed points from which simultaneous region growing is performed until the entire surface is covered. The result of this region growing is the Voronoi-graph approximation. From this Voronoi-graph its dual the Delaunay-Triangulation can be created which represent the output mesh.”)
and the volume voxel representation of the three-dimensional object is generated based on expanding, starting from a corner of the voxel grid, the voxel grid along axis directions to generate an expanded voxel grid that forms the volume voxel representation of the three- dimensional object (see at least [0082]: “After finishing the selection and the thinning an adaptive refinement can be performed as the second phase of the triangulation. In the adaptive refinement phase all selected faces become region seed points from which simultaneous region growing is performed until the entire surface is covered. The result of this region growing is the Voronoi-graph approximation. From this Voronoi-graph its dual the Delaunay-Triangulation can be created which represent the output mesh.”)
Regarding claim 17, the combination of Kaus and Hinterstoisser teaches:
The system of claim 12, wherein the at least one processor is further configured to read the instructions to:
for each first triangle in the set of first triangles, subdivide the first triangle into N smaller triangles, where N is greater than 1, if the first triangle contains an edge longer than a predetermined threshold; repeat the subdividing step for each of the N smaller triangles until each edge of every triangle generated from the first triangle is shorter than or equal to the predetermined threshold; (see at least [0070]: “After the calculation of the local curvature the triangulation can be generated. For achieving the triangulation in a first step an appropriate gamma is determined, wherein gamma is the so-called detail balance parameter. In a descriptive way gamma can be understand as a parameter which determines how many details can be seen in the output mesh or final triangulation. For determining an appropriate gamma the triangulation is bisected till a gamma is found so that an output mesh created by utilisation of that gamma matches a predetermined target polygon count (TPC).” See further [0072-0073]: “Wherein the parameter AverageRange is defined by four times the square root of DecimationRatio, i.e. AverageRange= {square root over (DecimationRatio)}. Wherein the parameter DecimationRatio is defined by the number of the micro mesh faces divided by TPC and multiplied by the average micro mesh edge length…”)
overlay each triangle generated from the first triangle with one or more voxels in the voxel grid; (see at least [0019]: “After the decomposing of the at least one T-junction into at least one triangle and at least one two-point-polygon it can occur that a second type of T-junction occurs when more than two 2-point-polygons meet at a common voxel/vertex.”)
and for each voxel overlaying a triangle generated from the first triangle, randomly select a target point in the voxel, and determine a chosen point in the voxel based on a vertex of the triangle that is closer to the target point compared to any other vertex in the voxel (see at least [0075]: “After determining gamma, R.sub.max and R.sub.min a selection and thinning can be performed as a first phase of the triangulation. In a first step all micro mesh faces are added to a so-called candidate queue (LocalFaceQueue). Then all one-point-polygons are selected from the queue. In a next step the micro mesh face which has the highest local curvature and which is left in the candidate queue is extracted and added to the selection.”)
Regarding claim 20, Kaus discloses:
A non-transitory computer readable medium having instructions stored thereon for spatially modeling a three-dimensional object within a three-dimensional workspace, wherein the instructions, when executed by at least one processor, cause at least one device to perform operations comprising:
obtaining an object representative polygon mesh including a set of polygons in three dimensions, wherein the object representative polygon mesh represents a surface of the three- dimensional object; (see at least [0044]: “In the following, referring to FIG. 1, a method of producing a triangulation of a surface of a physical object according to an exemplary embodiment of the present invention comprises the step of labelling voxels 101.”)
converting the object representative polygon mesh into an object representative triangle mesh including a set of first triangles; (see at least [0046]: “Preferably, multiple connected T-junctions, e.g. T-junctions connected to multiple organs, are associated with an correct subset of neighbouring surface labels, e.g. each T-junction is associated with the correct subset of neighbouring organs. Preferably, this associating is done before decomposing the T-junctions. By using such an associating step it may be ensured to obtain afterwards a correct Delaunay-Triangulation from the Voronoi-graph.”)
subdividing the object representative triangle mesh into a subdivided object representative triangle mesh including a set of second triangles, (see at least [0047]: “After decomposing in a next step 105 vertices from the micro mesh may be selected according to a so called thinning process. Thus, the number of vertices used for a subsequent triangulation is reduced saving processing power… From this discrete approximation of the Voronoi-graph its dual the surface Delaunay-Triangulation can be generated. By using a local curvature-adaptive thinning process the local shape of the physical object is incorporated in an early stage of the algorithm yielding an elegant method for obtaining shape-adaptive triangular meshes.”)
wherein the subdivided object representative triangle mesh is overlaid with a voxel grid including a set of voxels; (see at least [0048]: “The thinning corresponds to a growing of the surface regions which are represented by the voxels. One surface region is represented by substantially all adjacent surface voxels having substantially the same outer surface orientation, i.e. can be approximated by a single triangle, without loosing a great amount of information.”);
generating a point collection including a plurality of points each corresponding to a voxel in the voxel grid, wherein each point is generated based on vertices of the subdivided object representative triangle mesh located in the voxels of the voxel grid; (see at least [0048]: “The thinning corresponds to a growing of the surface regions which are represented by the voxels. One surface region is represented by substantially all adjacent surface voxels having substantially the same outer surface orientation, i.e. can be approximated by a single triangle, without loosing a great amount of information. The loosing of information is thereby dependent on the rate of the thinning. Preferably, the region growing is modified so that no vertex that is labelled with a first label is shifted into a region in which vertices having a different second label. Furthermore, the region growing preferably is adapted so that no vertex that is part of a triangle is shifted to a boundary edge of the boundary edge, or vice versa, i.e. no vertex that is part of a boundary edge is shifted to a triangle.”)
and generating, based on the point collection and the voxel grid, at least one of:
a surface point cloud representation of the three-dimensional object, a surface voxel representation of the three-dimensional object, or a volume voxel representation of the three-dimensional object (see at least [0060]: “After cleaning the source image a micro mesh is created. The micro mesh is the union of all surfaces that separate two voxels of different label in the source image and comprises a plurality of polygons each having a face. This micro mesh is the intermediate representation for the algorithm of the triangulation process to operate on.”)
Kaus does not explicitly disclose, but Hinterstoisser teaches:
obtaining, from one or more sensors associated with a grid of square pixels for recording images of at least a portion of the three-dimensional workspace, scan data of the three- dimensional object within the three-dimensional workspace (see at least col. 17, lines 42-65: ‘As another example, in some implementations object detection and pose determination engine 140A or 140B utilizes a voxel based approach to determine whether a distance between a given scene reference point and a given additional scene point satisfies a bounding measure of an object. For example, the engine 140A or 140B may generate a voxel grid in the 3D space of the scene point cloud data, where the bound of each voxel is based on the bounding measure of the object under consideration. For example, the bound of each voxel may be the true maximum diameter (i.e., the distance between the maximum extents of the object) and/or based on the true maximum diameter (e.g., 105% of the maximum diameter). The engine 140A or 140B may store, for each voxel, an indication of all of the scene points that “fit into” the voxel and/or, for each scene point, an indication of the voxel into which it fits. For example, the stored data may indicate: a first voxel and a first group of scene points that fit into that first voxel, a second voxel and a second group of scene points that fit into that second voxel, and so forth. Also, for example, the stored data may indicate: a first scene point and the voxel into which it fits, a second scene point and the voxel into which it fits, and so forth. As used herein, a scene point fits into a voxel if the position of the scene point is within the 3D area defined by the voxel.”)
wherein each voxel in the voxel grid corresponds with a square pixel in the grid of square pixels (see at least col. 17, lines 42-65: ‘As another example, in some implementations object detection and pose determination engine 140A or 140B utilizes a voxel based approach to determine whether a distance between a given scene reference point and a given additional scene point satisfies a bounding measure of an object. For example, the engine 140A or 140B may generate a voxel grid in the 3D space of the scene point cloud data, where the bound of each voxel is based on the bounding measure of the object under consideration. For example, the bound of each voxel may be the true maximum diameter (i.e., the distance between the maximum extents of the object) and/or based on the true maximum diameter (e.g., 105% of the maximum diameter). The engine 140A or 140B may store, for each voxel, an indication of all of the scene points that “fit into” the voxel and/or, for each scene point, an indication of the voxel into which it fits. For example, the stored data may indicate: a first voxel and a first group of scene points that fit into that first voxel, a second voxel and a second group of scene points that fit into that second voxel, and so forth. Also, for example, the stored data may indicate: a first scene point and the voxel into which it fits, a second scene point and the voxel into which it fits, and so forth. As used herein, a scene point fits into a voxel if the position of the scene point is within the 3D area defined by the voxel.”)
where the at least one of the surface point cloud representation of the three-dimensional object, the surface voxel representation of the three-dimensional object, and the volume voxel representation of the three-dimensional object effects operation of machinery within the three-dimensional workspace (see at least col. 16, lines 19-30: “ In some implementations where the system is incorporated in a robot and/or is in communication with a robot, the system adjusts the robot based on determining the object is present and/or based on the determined pose of the object. For example, the system may provide one or more control commands to one or more actuators of the robot to position the robot and/or one or more operational components of the robot based on a determined pose. For instance, a pallet jack robot may drive wheels thereof autonomously and/or adjust the height of forks thereof autonomously based on a determined pose for a pallet to enable the robot to position the forks in fork holes of the pallet.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Kaus with the specific methods of object representation as taught by Hinterstoisser. This is because as stated by Hinterstoisser’s Summary: “The present disclosure is generally directed to on one or more techniques that are applicable to 3D object detection and pose determination and that may optionally increase the robustness and/or efficiency of the 3D object recognition and pose determination. As used herein, robustness refers to the ability to detect an object and/or determine the pose of an object and/or refers to the accuracy of such detection and/or determination. For example, robustness may reference the ability to detect an object and/or determine the pose of an object in challenging environments such as environments with a relatively large amount of background clutter, a relatively large amount of occlusion, etc. As used herein, efficiency refers to time and/or computer resources used in detecting an object and/or determining the pose of an object.”
Claim(s) 10, 11, 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Kaus and Hinterstoisser further in view of Vu et al. (US 20180222051 A1), hereinafter Vu.
Regarding claim 10, the combination of Kaus and Hinterstoisser teaches:
The method of claim 1.
Kaus does not explicitly disclose, but Vu teaches:
wherein the three-dimensional object is a robot or a human operator in a workspace (see at least [0010]: “In various embodiments, the system includes a plurality of sensors distributed about the workspace. Each of the sensors includes or is associated with a grid of pixels for recording representations of a portion of the workspace within a sensor field of view; the workspace portions collectively cover the entire workspace. A computer memory stores (i) a series of images from the sensors, (ii) a model of the robot and its permitted movements, and (iii) a safety protocol specifying speed restrictions of a robot in proximity to a human and a minimum separation distance between a robot and a human. A processor is configured to generate, from the stored images, a spatial representation of the workspace (e.g., as volumes, which may correspond to voxels, i.e., 3D pixels). The processor identifies and monitors, over time, a representation of space occupied by the robot within the workspace as a robot region in the volume. The processor generates, around the robot region, an envelope region spanning the permitted movements of the robot in accordance with the stored model.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Kaus with the application of Vu because, as stated by Vu’s background: “Because these robots are large, strong, and fast, they are capable of doing severe harm to a human over a wide “envelope” of possible movement trajectories. When industrial robots first came into widespread use, no sufficiently reliable sensing technologies were available to sense humans and other obstacles... What is needed, instead, is a system that provides semantic understanding of the various elements of the workcell; in particular, the robot, the workpiece and human workers.”
Regarding claim 11, the combination of Kaus, Hinterstoisser, and Vu teaches:
The metho of claim 10.
Kaus does not explicitly disclose, but Vu teaches:
computationally generating a first potential occupancy envelope for the robot or a second potential occupancy envelope for the human operator when performing a task in the workspace, based on at least one of: the surface point cloud representation, the surface voxel representation, or the volume voxel representation, wherein the first and second potential occupancy envelopes spatially encompass movements performable by the robot and the human operator, respectively, during performance of the task (see at least [0010]: “ A processor is configured to generate, from the stored images, a spatial representation of the workspace (e.g., as volumes, which may correspond to voxels, i.e., 3D pixels). The processor identifies and monitors, over time, a representation of space occupied by the robot within the workspace as a robot region in the volume. The processor generates, around the robot region, an envelope region spanning the permitted movements of the robot in accordance with the stored model.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Kaus with the application of Vu because, as stated by Vu’s background: “Because these robots are large, strong, and fast, they are capable of doing severe harm to a human over a wide “envelope” of possible movement trajectories. When industrial robots first came into widespread use, no sufficiently reliable sensing technologies were available to sense humans and other obstacles... What is needed, instead, is a system that provides semantic understanding of the various elements of the workcell; in particular, the robot, the workpiece and human workers.”
Regarding claim 18, the combination of Kaus and Hinterstoisser teaches:
The system of claim 12.
Kaus does not explicitly disclose, but Vu teaches:
wherein the three-dimensional object is a robot or a human operator in a workspace (see at least [0010]: “In various embodiments, the system includes a plurality of sensors distributed about the workspace. Each of the sensors includes or is associated with a grid of pixels for recording representations of a portion of the workspace within a sensor field of view; the workspace portions collectively cover the entire workspace. A computer memory stores (i) a series of images from the sensors, (ii) a model of the robot and its permitted movements, and (iii) a safety protocol specifying speed restrictions of a robot in proximity to a human and a minimum separation distance between a robot and a human. A processor is configured to generate, from the stored images, a spatial representation of the workspace (e.g., as volumes, which may correspond to voxels, i.e., 3D pixels). The processor identifies and monitors, over time, a representation of space occupied by the robot within the workspace as a robot region in the volume. The processor generates, around the robot region, an envelope region spanning the permitted movements of the robot in accordance with the stored model.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Kaus with the application of Vu because, as stated by Vu’s background: “Because these robots are large, strong, and fast, they are capable of doing severe harm to a human over a wide “envelope” of possible movement trajectories. When industrial robots first came into widespread use, no sufficiently reliable sensing technologies were available to sense humans and other obstacles... What is needed, instead, is a system that provides semantic understanding of the various elements of the workcell; in particular, the robot, the workpiece and human workers.”
Regarding claim 19, the combination of Kaus, Hinterstoisser and Vu teaches:
The system of claim 18.
Kaus does not explicitly disclose, but Vu teaches:
wherein the at least one processor is further configured to read the instructions to:
computationally generate a first potential occupancy envelope for the robot or a second potential occupancy envelope for the human operator when performing a task in the workspace, based on at least one of: the surface point cloud representation, the surface voxel representation, or the volume voxel representation, wherein the first and second potential occupancy envelopes spatially encompass movements performable by the robot and the human operator, respectively, during performance of the task. (see at least [0010]: “ A processor is configured to generate, from the stored images, a spatial representation of the workspace (e.g., as volumes, which may correspond to voxels, i.e., 3D pixels). The processor identifies and monitors, over time, a representation of space occupied by the robot within the workspace as a robot region in the volume. The processor generates, around the robot region, an envelope region spanning the permitted movements of the robot in accordance with the stored model.”)
It would have been prima facie obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, with a reasonable expectation for success, to combine the invention of Kaus with the application of Vu because, as stated by Vu’s background: “Because these robots are large, strong, and fast, they are capable of doing severe harm to a human over a wide “envelope” of possible movement trajectories. When industrial robots first came into widespread use, no sufficiently reliable sensing technologies were available to sense humans and other obstacles... What is needed, instead, is a system that provides semantic understanding of the various elements of the workcell; in particular, the robot, the workpiece and human workers.”
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ELIZABETH NELESKI whose telephone number is (571)272-6064. The examiner can normally be reached 10 - 6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THOMAS WORDEN can be reached at (571) 272-4876. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.R.N./Examiner, Art Unit 3658
/JASON HOLLOWAY/Primary Examiner, Art Unit 3658
Prosecution Insights