DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 -6, 16, 17, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al. (CN 116958489) in view of Reistmayr et al. (US 20220122326).
Regarding claim 1.
Gao teaches:
A method for detecting planes in an augmented reality environment, the method comprising:
generating a plurality of depth estimates from one or more of: a series of posed camera images and a dataset of visual inertial odometry (VIO) points (Gao [PG 21 Par 7] in an AR scene, depth estimation requires a coordinate system converted to a VIO (visual inertial odometer) to enable object identification. VIO can realize SLAM algorithm by fusing camera and IMU data, VIO itself can output sparse depth point (that is, VIO generates sparse depth, sparse depth can represent depth information of partial pixel point in target coordinate system). Thus, the relative depth of the network output can be mapped to the VIO coordinate system by linear transformation.);
determining a plurality of distance values by applying a signed distance function to the plurality of depth estimates (Gao [Pg 5 Par 5] the relative depth, is the depth obtained by quantizing the depth of the pixel point in the image with a certain size as unit, the relative depth retains the relative relationship (e.g., proportional relationship) between the absolute depths of the pixel points, and eliminates the actual size (i.e., depth unit) on which the absolute depth is based, The value of the relative depth may be between 0 and 1, or between 0 and a designated value (e.g., 10), or from a smaller designated value (e.g., 0.5) to a larger designated value (e.g., 10), the larger the value is, the farther the distance from the origin is.);
Gao fails to teach:
configuring a voxel representation of the plurality of distance values (Reistmayr [0054] Depth information 306 can include any measurement or value that indicates and/or corresponds to a distance between a surface of a real world object and a point in physical space (e.g., a voxel). … In a non-limiting example, the depth map system 300 can represent the distances using a signed distance function, such as a truncated signed distance function.);
fitting a plurality of local planes to blocks of voxels in the voxel representation, wherein each block comprises multiple voxels (Reistmayr [0063] For instance, the geometry estimator 406 can identify 3D coordinates corresponding to each voxel within a group of blocks that have been merged to generate a merged plane equation.);
generating a first larger plane from merging a first subset of the plurality of local planes based on predefined criteria (Reistmayr [0071] For instance, the individual points within the projected coordinates 506 can each correspond to a coordinate (e.g., a voxel coordinate) projected onto a plane (e.g., a merged plane corresponding to one or the planar surfaces 504).);
and generating a second larger plane from merging a second subset of the plurality of local planes based on the predefined criteria, wherein the first subset is distinct from the second subset, and wherein a first normal vector of the first larger plane is different from a second normal vector of the second larger plane (Reistmayr [0063] In some cases, the geometry estimator 406 can determine geometric shapes corresponding to one or more merged plane equations. The determined geometric shapes can include planar regions corresponding to (or approximately corresponding to) object surfaces within the scene. The geometry estimator 406 can determine the planar regions in various ways and/or using various techniques. In one example, the geometry estimator 406 can identify each distance measurement corresponding to a merged plane equation. For instance, the geometry estimator 406 can identify 3D coordinates corresponding to each voxel within a group of blocks that have been merged to generate a merged plane equation.).
Reistmayr teaches:
configuring a voxel representation of the plurality of distance values (Reistmayr [0054] Depth information 306 can include any measurement or value that indicates and/or corresponds to a distance between a surface of a real world object and a point in physical space (e.g., a voxel). … In a non-limiting example, the depth map system 300 can represent the distances using a signed distance function, such as a truncated signed distance function.);
fitting a plurality of local planes to blocks of voxels in the voxel representation, wherein each block comprises multiple voxels (Reistmayr [0063] For instance, the geometry estimator 406 can identify 3D coordinates corresponding to each voxel within a group of blocks that have been merged to generate a merged plane equation.);
generating a first larger plane from merging a first subset of the plurality of local planes based on predefined criteria (Reistmayr [0071] For instance, the individual points within the projected coordinates 506 can each correspond to a coordinate (e.g., a voxel coordinate) projected onto a plane (e.g., a merged plane corresponding to one or the planar surfaces 504).);
and generating a second larger plane from merging a second subset of the plurality of local planes based on the predefined criteria, wherein the first subset is distinct from the second subset, and wherein a first normal vector of the first larger plane is different from a second normal vector of the second larger plane (Reistmayr [0063] In some cases, the geometry estimator 406 can determine geometric shapes corresponding to one or more merged plane equations. The determined geometric shapes can include planar regions corresponding to (or approximately corresponding to) object surfaces within the scene. The geometry estimator 406 can determine the planar regions in various ways and/or using various techniques. In one example, the geometry estimator 406 can identify each distance measurement corresponding to a merged plane equation. For instance, the geometry estimator 406 can identify 3D coordinates corresponding to each voxel within a group of blocks that have been merged to generate a merged plane equation.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao with Reistmayr. Generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, as in Reistmayr, would benefit the Gao teachings by allowing for a way to have voxels which correspond to distances to be on a plane. Additionally, this is the application of a known technique, generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, to yield predictable results.
Regarding claim 2.
Gao and Reistmayr teach:
The method of claim 1,
wherein the method further comprises: generating a plurality of updated depth estimates based on an update to one or more of: the series of posed camera images and the dataset of VIO points (Reistmayr [0051] The scene representation engine 122 can perform various operations to generate and/or update representations of scenes in the real world environment around the user.);
determining a plurality of updated distance values by applying the signed distance function to the plurality of updated depth estimates (Reistmayr [0054] Depth information 306 can include any measurement or value that indicates and/or corresponds to a distance between a surface of a real world object and a point in physical space (e.g., a voxel). … In a non-limiting example, the depth map system 300 can represent the distances using a signed distance function, such as a truncated signed distance function.);
updating the voxel representation with the plurality of updated distance values (Reistmayr [0054] Depth information 306 can include any measurement or value that indicates and/or corresponds to a distance between a surface of a real world object and a point in physical space (e.g., a voxel). … In a non-limiting example, the depth map system 300 can represent the distances using a signed distance function, such as a truncated signed distance function.);
and removing at least one of the first larger plane and the second larger plane based at least on updates to the voxel representation (Reistmayr [0066] In another example, the scene representation system 200 can update the scene representation 204 in response to detecting that the FOV of the camera system has changed (e.g., in response to detecting that the image data currently captured by the camera system corresponds to a new portion of the scene). Additionally or alternatively, the scene representation system 200 can update the scene representation 204 on a fixed time schedule (e.g., every 0.25 seconds, every 0.5 seconds, every 1 second, etc.).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao with Reistmayr. Generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, as in Reistmayr, would benefit the Gao teachings by allowing for a way to have voxels which correspond to distances to be on a plane. Additionally, this is the application of a known technique, generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, to yield predictable results.
Regarding claim 3.
Gao and Reistmayr teach:
The method of claim 2, wherein the method further comprises: fitting a plurality of updated local planes to blocks of voxels in the updated voxel representation; and generating a third larger plane from merging a first subset of the plurality of updated local planes based on the predefined criteria (Reistmayr [0063] For instance, the geometry estimator 406 can identify 3D coordinates corresponding to each voxel within a group of blocks that have been merged to generate a merged plane equation.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao with Reistmayr. Generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, as in Reistmayr, would benefit the Gao teachings by allowing for a way to have voxels which correspond to distances to be on a plane. Additionally, this is the application of a known technique, generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, to yield predictable results.
Regarding claim 4.
Gao and Reistmayr teach:
The method of claim 1,
wherein determining the plurality of distance values by applying the signed distance function to the plurality of depth estimates comprises: continuously generating additional plurality of depth estimates based on continuous updates to one or more of: the series of posed camera images and the dataset of VIO points (Reistmayr [0066] In another example, the scene representation system 200 can update the scene representation 204 in response to detecting that the FOV of the camera system has changed (e.g., in response to detecting that the image data currently captured by the camera system corresponds to a new portion of the scene). Additionally or alternatively, the scene representation system 200 can update the scene representation 204 on a fixed time schedule (e.g., every 0.25 seconds, every 0.5 seconds, every 1 second, etc.).);
and averaging the additional plurality of depth estimates into the voxel representation (Reistmayr [0059] In some cases, generating a volumetric reconstruction of a scene can average and/or filter errors within a depth map, which may facilitate more accurate and/or more efficient processing of the information within the depth map.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao with Reistmayr. Generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, as in Reistmayr, would benefit the Gao teachings by allowing for a way to have voxels which correspond to distances to be on a plane. Additionally, this is the application of a known technique, generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, to yield predictable results.
Regarding claim 5.
Gao and Reistmayr teach:
The method of claim 1,
wherein the signed distance function is a truncated signed distance function (Reistmayr [0054] In a non-limiting example, the depth map system 300 can represent the distances using a signed distance function, such as a truncated signed distance function.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao with Reistmayr. Generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, as in Reistmayr, would benefit the Gao teachings by allowing for a way to have voxels which correspond to distances to be on a plane. Additionally, this is the application of a known technique, generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, to yield predictable results.
Regarding claim 6.
Gao and Reistmayr teach:
The method of claim 1,
wherein generating the first larger plane from merging the first subset of the plurality of local planes comprises a comparison of a surface normal vector of each local plane with a neighboring local plane (Reistmayr [0079] In some cases, the plane merger 404 can determine that two plane equations have at least the threshold degree of similarity based on comparing the plane parameters of the plane equations.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao with Reistmayr. Generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, as in Reistmayr, would benefit the Gao teachings by allowing for a way to have voxels which correspond to distances to be on a plane. Additionally, this is the application of a known technique, generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, to yield predictable results.
Regarding claim 16.
A system for detecting planes in an augmented reality environment, the system comprising:
one or more processors;
a memory storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
generating depth estimates from one or more of: a series of posed camera images and a dataset of visual inertial odometry (VIO) points (Gao [PG 21 Par 7] in an AR scene, depth estimation requires a coordinate system converted to a VIO (visual inertial odometer) to enable object identification. VIO can realize SLAM algorithm by fusing camera and IMU data, VIO itself can output sparse depth point (that is, VIO generates sparse depth, sparse depth can represent depth information of partial pixel point in target coordinate system). Thus, the relative depth of the network output can be mapped to the VIO coordinate system by linear transformation.);
generating distance values by applying a signed distance function to the depth estimates (Gao [Pg 5 Par 5] the relative depth, is the depth obtained by quantizing the depth of the pixel point in the image with a certain size as unit, the relative depth retains the relative relationship (e.g., proportional relationship) between the absolute depths of the pixel points, and eliminates the actual size (i.e., depth unit) on which the absolute depth is based, The value of the relative depth may be between 0 and 1, or between 0 and a designated value (e.g., 10), or from a smaller designated value (e.g., 0.5) to a larger designated value (e.g., 10), the larger the value is, the farther the distance from the origin is.);
configuring a voxel representation of the distance values (Reistmayr [0054] Depth information 306 can include any measurement or value that indicates and/or corresponds to a distance between a surface of a real world object and a point in physical space (e.g., a voxel). … In a non-limiting example, the depth map system 300 can represent the distances using a signed distance function, such as a truncated signed distance function.);
fitting local planes to blocks of voxels in the voxel representation, wherein each block comprises multiple voxels (Reistmayr [0063] For instance, the geometry estimator 406 can identify 3D coordinates corresponding to each voxel within a group of blocks that have been merged to generate a merged plane equation.);
generating larger planes by merging one or more of the local planes based on predefined criteria (Reistmayr [0071] For instance, the individual points within the projected coordinates 506 can each correspond to a coordinate (e.g., a voxel coordinate) projected onto a plane (e.g., a merged plane corresponding to one or the planar surfaces 504).);
and at least one of dynamically update or dynamically remove one or more of the larger planes in response to generating one or more updated depth estimates (Reistmayr [0063] In some cases, the geometry estimator 406 can determine geometric shapes corresponding to one or more merged plane equations. The determined geometric shapes can include planar regions corresponding to (or approximately corresponding to) object surfaces within the scene. The geometry estimator 406 can determine the planar regions in various ways and/or using various techniques. In one example, the geometry estimator 406 can identify each distance measurement corresponding to a merged plane equation. For instance, the geometry estimator 406 can identify 3D coordinates corresponding to each voxel within a group of blocks that have been merged to generate a merged plane equation.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao with Reistmayr. Generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, as in Reistmayr, would benefit the Gao teachings by allowing for a way to have voxels which correspond to distances to be on a plane. Additionally, this is the application of a known technique, generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, to yield predictable results.
Regarding claim 17.
The system of claim 16,
wherein the operations further comprise: continuously generate additional depth estimates based on continuous updates to one or more of: the series of posed camera images and the dataset of VIO points (Reistmayr [0066] In another example, the scene representation system 200 can update the scene representation 204 in response to detecting that the FOV of the camera system has changed (e.g., in response to detecting that the image data currently captured by the camera system corresponds to a new portion of the scene). Additionally or alternatively, the scene representation system 200 can update the scene representation 204 on a fixed time schedule (e.g., every 0.25 seconds, every 0.5 seconds, every 1 second, etc.).);
and averaging the additional depth estimates into the voxel representation (Reistmayr [0059] In some cases, generating a volumetric reconstruction of a scene can average and/or filter errors within a depth map, which may facilitate more accurate and/or more efficient processing of the information within the depth map.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao with Reistmayr. Generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, as in Reistmayr, would benefit the Gao teachings by allowing for a way to have voxels which correspond to distances to be on a plane. Additionally, this is the application of a known technique, generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, to yield predictable results.
Regarding claim 20.
A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a processor of a system, cause the system to perform operations comprising:
generating a plurality of depth estimates from one or more of: a series of posed camera images and a dataset of visual inertial odometry (VIO) points (Gao [PG 21 Par 7] in an AR scene, depth estimation requires a coordinate system converted to a VIO (visual inertial odometer) to enable object identification. VIO can realize SLAM algorithm by fusing camera and IMU data, VIO itself can output sparse depth point (that is, VIO generates sparse depth, sparse depth can represent depth information of partial pixel point in target coordinate system). Thus, the relative depth of the network output can be mapped to the VIO coordinate system by linear transformation.);
determining a plurality of distance values by applying a signed distance function to the plurality of depth estimates (Gao [Pg 5 Par 5] the relative depth, is the depth obtained by quantizing the depth of the pixel point in the image with a certain size as unit, the relative depth retains the relative relationship (e.g., proportional relationship) between the absolute depths of the pixel points, and eliminates the actual size (i.e., depth unit) on which the absolute depth is based, The value of the relative depth may be between 0 and 1, or between 0 and a designated value (e.g., 10), or from a smaller designated value (e.g., 0.5) to a larger designated value (e.g., 10), the larger the value is, the farther the distance from the origin is.);
configuring a voxel representation with the plurality of distance values (Reistmayr [0054] Depth information 306 can include any measurement or value that indicates and/or corresponds to a distance between a surface of a real world object and a point in physical space (e.g., a voxel). … In a non-limiting example, the depth map system 300 can represent the distances using a signed distance function, such as a truncated signed distance function.);
fitting a plurality of local planes to blocks of voxels in the voxel representation, wherein each block comprises multiple voxels (Reistmayr [0063] For instance, the geometry estimator 406 can identify 3D coordinates corresponding to each voxel within a group of blocks that have been merged to generate a merged plane equation.);
generating a first larger plane from merging a first subset of the plurality of local planes based on predefined criteria (Reistmayr [0071] For instance, the individual points within the projected coordinates 506 can each correspond to a coordinate (e.g., a voxel coordinate) projected onto a plane (e.g., a merged plane corresponding to one or the planar surfaces 504).);
generating a second larger plane from merging a second subset of the plurality of local planes based on the predefined criteria, wherein the first subset is distinct from the second subset, and wherein a first normal vector of the first larger plane is different from a second normal vector of the second larger plane (Reistmayr [0063] In some cases, the geometry estimator 406 can determine geometric shapes corresponding to one or more merged plane equations. The determined geometric shapes can include planar regions corresponding to (or approximately corresponding to) object surfaces within the scene. The geometry estimator 406 can determine the planar regions in various ways and/or using various techniques. In one example, the geometry estimator 406 can identify each distance measurement corresponding to a merged plane equation. For instance, the geometry estimator 406 can identify 3D coordinates corresponding to each voxel within a group of blocks that have been merged to generate a merged plane equation.);
and at least one of dynamically updating or dynamically removing one or more of the first larger plane and the second larger plane in response to generating one or more updated depth estimates (Reistmayr [0063] In some cases, the geometry estimator 406 can determine geometric shapes corresponding to one or more merged plane equations. The determined geometric shapes can include planar regions corresponding to (or approximately corresponding to) object surfaces within the scene. The geometry estimator 406 can determine the planar regions in various ways and/or using various techniques. In one example, the geometry estimator 406 can identify each distance measurement corresponding to a merged plane equation. For instance, the geometry estimator 406 can identify 3D coordinates corresponding to each voxel within a group of blocks that have been merged to generate a merged plane equation.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao with Reistmayr. Generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, as in Reistmayr, would benefit the Gao teachings by allowing for a way to have voxels which correspond to distances to be on a plane. Additionally, this is the application of a known technique, generating plans that have voxels that are fitted or even generated by the collections of voxels and also having merged planes, to yield predictable results.
Claim(s) 7 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al. (CN 116958489) in view of Reistmayr et al. (US 20220122326), Cheng et al. (CN 108399641), and Yu et al. (US 20210304422).
Regarding claim 7.
Gao and Reistmayr teach:
The method of claim 1,
Gao and Reistmayr fail to teach:
wherein the predefined criteria comprise at least one of a similarity threshold for surface normal vectors of the plurality of local planes and a root mean square error below a predetermined threshold (Cheng [ABSTRACT] if the similarity is smaller than the similarity threshold value, determining to re-detect ground..) (Yu [0048] To obtain the clique corresponding to the target object separate from the pseudo-dense cliques, a variance threshold to cut off low-density cliques, which may be determined based on a root mean square error (RMSE), may be employed.).
Cheng and Yu teach:
wherein the predefined criteria comprise at least one of a similarity threshold for surface normal vectors of the plurality of local planes and a root mean square error below a predetermined threshold (Cheng [ABSTRACT] A re-detect ground is determined if the similarity between the current image frame of the gravitational acceleration vector and the ground normal vector is less than similarity threshold.) (Yu [0048] To obtain the clique corresponding to the target object separate from the pseudo-dense cliques, a variance threshold to cut off low-density cliques, which may be determined based on a root mean square error (RMSE), may be employed.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao and Reistmayr with Cheng and Yu. Having a similarity threshold and also have a root mean square error threshold, as in Cheng and Yu, would benefit the Gao and Reistmayr teachings by having a threshold to compare to. Additionally, this is the application of a known technique, have a similarity threshold and also have a root mean square error threshold, to yield predictable results.
Regarding claim 18.
The system of claim 16,
wherein the predefined criteria comprise at least one of a similarity threshold for surface normal vectors of the local planes and a root mean square error below a predetermined threshold (Cheng [ABSTRACT] if the similarity is smaller than the similarity threshold value, determining to re-detect ground.) (Yu [0048] To obtain the clique corresponding to the target object separate from the pseudo-dense cliques, a variance threshold to cut off low-density cliques, which may be determined based on a root mean square error (RMSE), may be employed.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao and Reistmayr with Cheng and Yu. Having a similarity threshold and also have a root mean square error threshold, as in Cheng and Yu, would benefit the Gao and Reistmayr teachings by having a threshold to compare to. Additionally, this is the application of a known technique, have a similarity threshold and also have a root mean square error threshold, to yield predictable results.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al. (CN 116958489) in view of Reistmayr et al. (US 20220122326), and Chandler et al. (WO 2021013791).
Regarding claim 8.
Gao and ref teach:
The method of claim 1,
Gao and Reistmayr fail to teach:
wherein a block of voxels used in fitting the plurality local planes to blocks of voxels in the voxel representation comprises 8x8x8 voxels (Chandler [Page 32 Par 4] In a 64-tree, the principles are the same, however each voxel is subdivided into an 8x8x8 array of smaller voxels rather than the 4x4x4 array used in an octree.).
Chandler teaches:
wherein a block of voxels used in fitting the plurality local planes to blocks of voxels in the voxel representation comprises 8x8x8 voxels (Chandler [Page 32 Par 4] In a 64-tree, the principles are the same, however each voxel is subdivided into an 8x8x8 array of smaller voxels rather than the 4x4x4 array used in an octree.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao and Reistmayr with Chandler. Having each voxel is subdivided into an 8x8x8 array of smaller voxels, as in Chandler, would benefit the Gao and Reistmayr teachings by being able to have each block of voxels subdivided. Additionally, this is the application of a known technique, having each voxel is subdivided into an 8x8x8 array of smaller voxels, to yield predictable results.
Claim(s) 9 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al. (CN 116958489) in view of Reistmayr et al. (US 20220122326), and Powers et al. (US 10339716).
Regarding claim 9.
Gao and Reistmayr teach:
The method of claim 1,
Gao and Reistmayr fail to teach:
wherein a size of a block of voxels used in fitting the plurality of local planes to blocks of voxels is based on a distance in the voxel representation (Powers [0036] In some particular implementations, the size of a voxel block may vary based on the distance from the current pose of the mobile device.).
Powers teaches:
wherein a size of a block of voxels used in fitting the plurality of local planes to blocks of voxels is based on a distance in the voxel representation (Powers [0036] In some particular implementations, the size of a voxel block may vary based on the distance from the current pose of the mobile device.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao and Reistmayr with Powers. Having the size of a voxel block may vary based on the distance from the current pose, as in Powers, would benefit the Gao and Reistmayr teachings by being able to have each block of voxels different based on distances. Additionally, this is the application of a known technique, having the size of a voxel block may vary based on the distance from the current pose, to yield predictable results.
Regarding claim 12.
Gao and Reistmayr teach:
The method of claim 1,
Gao and Reistmayr fail to teach:
further comprising: extending the first larger plane to neighboring blocks of voxels when the neighboring blocks of voxels meet the predefined criteria (Powers [0068] (e.g., the 27 indices assigned to the 27 voxels are determined by v=x+y*3+z*9, where x, y, z are the voxel coordinates in the extended voxel block).).
Powers teaches:
further comprising: extending the first larger plane to neighboring blocks of voxels when the neighboring blocks of voxels meet the predefined criteria (Powers [0068] (e.g., the 27 indices assigned to the 27 voxels are determined by v=x+y*3+z*9, where x, y, z are the voxel coordinates in the extended voxel block).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao and Reistmayr with Powers. Having the size of a voxel block may vary based on the distance from the current pose, as in Powers, would benefit the Gao and Reistmayr teachings by being able to have each block of voxels different based on distances. Additionally, this is the application of a known technique, having the size of a voxel block may vary based on the distance from the current pose, to yield predictable results.
Claim(s) 10 and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al. (CN 116958489) in view of Reistmayr et al. (US 20220122326), Powers et al. (US 10339716), and Chandler et al. (WO 2021013791).
Regarding claim 10.
Gao, Reistmayr, and Powers teach:
The method of claim 9,
Gao, Reistmayr, and Powers fail to teach:
wherein the size of the block of voxels is larger than 8x8x8 voxels when the distance in the voxel representation is above a first threshold (Powers [0036] In some particular implementations, the size of a voxel block may vary based on the distance from the current pose of the mobile device. [0026] In addition to or in alternative of a distance threshold, the system may implement a time threshold to cause a new viewpoint bundle to be loaded into memory or the integration viewpoint bundle to be evaluated (for instance based on distance or drift error).) (Chandler [Page 32 Par 4] In a 64-tree, the principles are the same, however each voxel is subdivided into an 8x8x8 array of smaller voxels rather than the 4x4x4 array used in an octree.).
Powers and Chandler together teach:
wherein the size of the block of voxels is larger than 8x8x8 voxels when the distance in the voxel representation is above a first threshold (Powers [0036] In some particular implementations, the size of a voxel block may vary based on the distance from the current pose of the mobile device. [0026] In addition to or in alternative of a distance threshold, the system may implement a time threshold to cause a new viewpoint bundle to be loaded into memory or the integration viewpoint bundle to be evaluated (for instance based on distance or drift error).) (Chandler [Page 32 Par 4] In a 64-tree, the principles are the same, however each voxel is subdivided into an 8x8x8 array of smaller voxels rather than the 4x4x4 array used in an octree.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao and Reistmayr with Powers and Chandler. Having the size of a voxel block may vary based on the distance from the current pose and having each voxel is subdivided into an 8x8x8 array of smaller voxels, as in Powers and Chandler, would benefit the Gao and Reistmayr teachings by being able to have each block of voxels different based on distances. Additionally, this is the application of a known technique, having the size of a voxel block may vary based on the distance from the current pose and having each voxel is subdivided into an 8x8x8 array of smaller voxels, to yield predictable results.
Regarding claim 11.
Gao, Reistmayr, and Powers teach:
The method of claim 9,
Gao, Reistmayr, and Powers fail to teach:
wherein the size of the block of voxels is smaller than 8x8x8 voxels when the distance in the voxel representation is below a second threshold (Powers [0036] In some particular implementations, the size of a voxel block may vary based on the distance from the current pose of the mobile device. [0026] In addition to or in alternative of a distance threshold, the system may implement a time threshold to cause a new viewpoint bundle to be loaded into memory or the integration viewpoint bundle to be evaluated (for instance based on distance or drift error).) (Chandler [Page 32 Par 4] In a 64-tree, the principles are the same, however each voxel is subdivided into an 8x8x8 array of smaller voxels rather than the 4x4x4 array used in an octree.).
Powers and Chandler together teach:
wherein the size of the block of voxels is smaller than 8x8x8 voxels when the distance in the voxel representation is below a second threshold (Powers [0036] In some particular implementations, the size of a voxel block may vary based on the distance from the current pose of the mobile device. [0026] In addition to or in alternative of a distance threshold, the system may implement a time threshold to cause a new viewpoint bundle to be loaded into memory or the integration viewpoint bundle to be evaluated (for instance based on distance or drift error).) (Chandler [Page 32 Par 4] In a 64-tree, the principles are the same, however each voxel is subdivided into an 8x8x8 array of smaller voxels rather than the 4x4x4 array used in an octree.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao and Reistmayr with Powers and Chandler. Having the size of a voxel block may vary based on the distance from the current pose and having each voxel is subdivided into an 8x8x8 array of smaller voxels, as in Powers and Chandler, would benefit the Gao and Reistmayr teachings by being able to have each block of voxels different based on distances. Additionally, this is the application of a known technique, having the size of a voxel block may vary based on the distance from the current pose and having each voxel is subdivided into an 8x8x8 array of smaller voxels, to yield predictable results.
Claim(s) 13, 14, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al. (CN 116958489) in view of Reistmayr et al. (US 20220122326), and Ramirez et al. (20220205788).
Regarding claim 13.
Gao and Reistmayr teach:
The method of claim 1,
Gao and Reistmayr fail to teach:
further comprising sampling points from the first larger plane (Ramirez [0014] determine a two dimensional plane between the visual features in a sliding window and for a plurality of sampled points from the IMU, wheel speed sensors, and steering wheel angle;).
Ramirez teaches:
further comprising sampling points from the first larger plane (Ramirez [0014] determine a two dimensional plane between the visual features in a sliding window and for a plurality of sampled points from the IMU, wheel speed sensors, and steering wheel angle;).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao and Reistmayr with Ramirez. Sampling points, as in Ramirez, would benefit the Gao and Reistmayr teachings by being able to have multiple different data points. Additionally, this is the application of a known technique, sampling points, to yield predictable results.
Regarding claim 14.
Gao, Reistmayr, and Ramirez teach:
The method of claim 13,
further comprising using the sampled points to improve accuracy of the dataset of VIO points (Ramirez [0014] A vehicle-visual-inertial odometry system for a ground vehicle according to another exemplary embodiment of this disclosure includes, among other possible things, at least one camera on-board the vehicle obtaining images of object proximate the vehicle, an inertial measurement unit generating information indicative of vehicle movement, a wheel speed sensor generating information indicative of wheel speed and a controller configured to obtain an initial set of images with a camera on-board a vehicle; identify visual features within the initial set of images, obtain information indicative of vehicle movement with an inertial measurement unit, obtain information indicative of vehicle movement with the vehicle's wheel speed sensors and steering wheel angle sensor, determine a two dimensional plane between the visual features in a sliding window and for a plurality of sampled points from the IMU, wheel speed sensors, and steering wheel angle; fuse the identified features within the images and the vehicle movement from the IMU and vehicle sensors within the two-dimensional plane, and determine a vehicle position relative to an initial start location based on the visual features in the images and the vehicle movement information from the IMU and vehicle sensors.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao and Reistmayr with Ramirez. Sampling points, as in Ramirez, would benefit the Gao and Reistmayr teachings by being able to have multiple different data points. Additionally, this is the application of a known technique, sampling points, to yield predictable results.
Regarding claim 19.
The system of claim 16,
wherein the operations further comprise: sampling points from at least one of the larger planes (Ramirez [0014] determine a two dimensional plane between the visual features in a sliding window and for a plurality of sampled points from the IMU, wheel speed sensors, and steering wheel angle;);
and using the sampled points to improve at least one of: accuracy of the dataset of VIO points and accuracy of the depth estimates (Ramirez [0014] A vehicle-visual-inertial odometry system for a ground vehicle according to another exemplary embodiment of this disclosure includes, among other possible things, at least one camera on-board the vehicle obtaining images of object proximate the vehicle, an inertial measurement unit generating information indicative of vehicle movement, a wheel speed sensor generating information indicative of wheel speed and a controller configured to obtain an initial set of images with a camera on-board a vehicle; identify visual features within the initial set of images, obtain information indicative of vehicle movement with an inertial measurement unit, obtain information indicative of vehicle movement with the vehicle's wheel speed sensors and steering wheel angle sensor, determine a two dimensional plane between the visual features in a sliding window and for a plurality of sampled points from the IMU, wheel speed sensors, and steering wheel angle; fuse the identified features within the images and the vehicle movement from the IMU and vehicle sensors within the two-dimensional plane, and determine a vehicle position relative to an initial start location based on the visual features in the images and the vehicle movement information from the IMU and vehicle sensors.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao and Reistmayr with Ramirez. Sampling points, as in Ramirez, would benefit the Gao and Reistmayr teachings by being able to have multiple different data points. Additionally, this is the application of a known technique, sampling points, to yield predictable results.
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gao et al. (CN 116958489) in view of Reistmayr et al. (US 20220122326), Ramirez et al. (20220205788), and Chandler et al. (WO 2021013791).
Regarding claim 15.
Gao, Reistmayr, and Ramirez teach:
The method of claim 13,
Gao, Reistmayr, and Ramirez fail to teach:
further comprising using the sampled points to improve accuracy of the plurality of depth estimates (Chandler [Page 20 Par 1] A number of stereo depth extraction algorithms may be applied to estimate the pixel disparities, such as Global Matching, Semi-Global Matching and Local Matching algorithms. In a real-time context, Semi-Global Matching (SGM) generally provides an acceptable trade off between accuracy and real-time performance.).
Chandler teaches:
further comprising using the sampled points to improve accuracy of the plurality of depth estimates (Chandler [Page 20 Par 1] A number of stereo depth extraction algorithms may be applied to estimate the pixel disparities, such as Global Matching, Semi-Global Matching and Local Matching algorithms. In a real-time context, Semi-Global Matching (SGM) generally provides an acceptable trade off between accuracy and real-time performance.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gao, Reistmayr, and Ramirez with Chandler. Updating accuracy of estimates, as in Chandler, would benefit the Gao, Reistmayr, and Ramirez teachings by being able to be more accurate. Additionally, this is the application of a known technique, updating accuracy of estimates, to yield predictable results.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DENIS VASILIY MINKO whose telephone number is (571)270-5226. The examiner can normally be reached Monday-Thursday 8:30-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DENIS VASILIY MINKO/Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
Prosecution Insights