DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/10/2026 has been entered.
Response to Amendments
The amendments to claims 1, 11, and 19 are accepted and entered.
Claims 1-20 are pending regarding this application.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. As a result, of Hu et al. (CN 1046370055 B, see attached English translation for citations) has been included in the 103 rejection below to teach the added subject matter as included in the claims filed on 04/10/2026. Please see the 103 rejection of claims 1-20 regarding this matter.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 7, 11, 12, 17, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Robert et al. (WO 2017/096299 A1), hereinafter Robert, in view of Hu et al. (CN 1046370055 B, see attached English translation for citations), hereinafter Hu and Shreve et al. (U.S. Publication No. 2021/0142039 A1), hereinafter Shreve.
Regarding claim 1, Robert teaches a method for aligning multiple three-dimensional (3D) point clouds into a common 3D point cloud, comprising:
associating a first subset of points within a first 3D point cloud with a second subset of points within a second 3D point cloud (Robert “at step 302, a set of scans 304 (e.g., a first 3D point cloud and a second 3D point cloud) are acquired/input (e.g., via laser scanner 134/LiDAR or received from across a network 204). Such scans 304 are scans with normal (and such normal information may be retrieved). The two point clouds have a subset of points in common and there is no prior knowledge on an alignment between the two point clouds” para. [0065]) based on each point in the first subset of points having a threshold correspondence to a unique counterpart point in the second subset of points (Robert “Step 308 further includes determining matching pairs of descriptors (i.e., one descriptor from each point cloud in a pair). To determine matching pairs, a 4D kd-tree may be built using all of the descriptors. For the first descriptor, a nearest neighbor search may be performed in the tree for candidate descriptors from the second point cloud. Those descriptors within a threshold distance value are kept and stored” para. [0069]);
refining the first subset of points (Robert teaches removing outliers by creating hypotheses between matching pairs in the first and second point clouds. The hypotheses with the highest density are selected. This process is interpreted as the refining step in the claim language. See para. [0070]);
determining, based on point correspondences between the first improved subset of points and the second improved subset of points, a relative rotation and translation between the first 3D point cloud and the second 3D point cloud (Robert teaches descriptors which represent matching pairs in which “the descriptors are used to estimate rigid transformation hypotheses (i.e., at step 310). Each hypothesis represents a transformation between the first and second point cloud. Further, each hypothesis is based on the 3D information in the matching pair. Since there may be a lot of outliers, the transformations are accumulated into a fitted space and a search for zones with high density is conducted (e.g., similar to a classic Hough transform). Accordingly, one or more hypotheses are selected based on density. The most relevant transforms are then scored to define the correct alignment. In this regard, the selected hypotheses are validated based on a scoring at step 312. The scoring may be based on a translation as a first criterion and a rotation as a second criterion” para. [0070]; here, Robert teaches improving/refining the hypotheses and then validating the hypotheses (i.e. improved subset of points) based on a rotation and translation); and
aligning the first 3D point cloud and the second 3D point cloud within a common coordinate system based on the relative rotation and translation (Robert teaches a global registration process in para. [0064]; this process is interpreted as aligning the first and second point clouds within a common coordinate system, as it involves aligning two point clouds in a unified coordinate system by correcting position and orientation; Robert additionally teaches alignment based on translation as a first criterion and rotation as a second criterion in para. [0070]).
Robert fails to teach generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views. Additionally, while Robert teaches refining the hypotheses by eliminating outliers, Robert fails to teach refining the first subset of points to generate a first improved subset of points that includes fewer outlier point correspondences to a second subset of points than the first subset of points, wherein refining the first subset of points includes searching for tiepoint triangles in the first subset of points having correspondence to tiepoint triangles in the second subset of points and refining the second subset of points to generate a second improved subset of points that includes fewer outlier point correspondences to the first subset of points than the second subset of points, wherein refining the second subset of points includes searching for tiepoint triangles in the second subset of points having correspondence to tiepoint triangles in the first subset of points.
However, Hu teaches refining the first subset of points to generate a first improved subset of points that includes fewer outlier point correspondences to a second subset of points than the first subset of points, wherein refining the first subset of points includes searching for tiepoint triangles in the first subset of points having correspondence to tiepoint triangles in the second subset of points and refining the second subset of points to generate a second improved subset of points that includes fewer outlier point correspondences to the first subset of points than the second subset of points, wherein refining the second subset of points includes searching for tiepoint triangles in the second subset of points having correspondence to tiepoint triangles in the first subset of points (Hu teaches a first set of feature points (M1) in a first image (l1) and a second set of feature points (M2) in a second image (l2) and determining the matching points of M1 and M2. These respective matched points of the first and second images are determined to be equivalent to the claimed first and second subset of points. See para. [0075]-[0077] regarding this matter. Subsequently Hu teaches a process of using a similar method triangles to “purify” the matching point pairs, by forming triangles in each image that contain matching pair point (tiepoints), and eliminating any pairs which do not satisfy the formula as shown in para. [0077]. Since pairs of points are eliminated in this process, it is determined that BOTH the first subset of points and the second subset of points are refined in this process in order to create a first improved subset of points and a second improved subset of points. Additionally, the process of searching for tiepoint triangles occurs such that the tiepoint triangles in the first subset of points have correspondence to tiepoint triangles in the second subset of points, and vice versa).
Robert and Hu are both considered to be analogous to the claimed invention because they are in the same field of registering images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert to incorporate the teachings of Hu and include “refining the first subset of points to generate a first improved subset of points that includes fewer outlier point correspondences to a second subset of points than the first subset of points, wherein refining the first subset of points includes searching for tiepoint triangles in the first subset of points having correspondence to tiepoint triangles in the second subset of points and refining the second subset of points to generate a second improved subset of points that includes fewer outlier point correspondences to the first subset of points than the second subset of points, wherein refining the second subset of points includes searching for tiepoint triangles in the second subset of points having correspondence to tiepoint triangles in the first subset of points”. The motivation would have been that “eliminating the mismatched point pair is performed to achieve the purification of the feature point pairs”, as suggested by Hu in para. [0077]. Therefore, it would have been obvious to combine Robert with Hu to obtain the invention specified in the above claim limitations.
Robert and Hu fail to teach generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views.
However, Shreve teaches generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views (Shreve “the 3D sensor 904 is representative of a variety of different 3D scanning devices including, for example… a time-of-flight 3D laser scanner” para. [0052]; “the method shown in FIG. 1 comprises obtaining 102 a first 3D point cloud associated with a physical object having at least one articulatable part. The first point cloud is associated with the physical object prior to articulation of the articulatable part. The method comprises obtaining 104 a second 3D point cloud associated with the physical object after articulation of the articulatable part” para. [0025]; the difference between the first and second point clouds is interpreted as different time-of-flight views)
Robert, Hu, and Shreve are all considered to be analogous to the claimed invention because they are in the same field of registering images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert (as modified by Hu) to incorporate the teachings of Shreve and “generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views” and “the refinement of the two point clouds separately”. The motivation for including “generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views” would have been to “generat[e] 112 an output comprising at least the remaining points of the second point cloud associated with the articulatable part without the noise points”, as suggested by Shreve in para. [0025]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert and Hu with Shreve to obtain the invention specified in claim 1.
Regarding claim 2, Robert, Hu, and Shreve teach the method of claim 1, further comprising:
generating first Fast Point Feature Histograms (FPFHs) for points in the first subset of points and second FPFHs for points in the second subset of points (Shreve “The method further comprises computing 408 Fast Point Feature Histogram (FPFH) descriptors from sparse point clouds 1 and 2” para. [0033]),
wherein associating a first point of the first subset of points with a second point of the second subset of points is based on determining whether a distance between a first FPFH descriptor for the first point in the first FPFHs and a second FPFH descriptor for the second point in the second FPFHs is within a threshold difference (Shreve “Fast Point Feature Histogram descriptors are computed on sparse point cloud1 and sparse point cloud2. Having computed the FPFH descriptors, the FPFH descriptors are matched across point cloud1 and point cloud2, and then used to align point cloud1 and point cloud2 by estimating a 3D transformation matrix on homogenous coordinates” para. [0040]; since the descriptors are matched across both point clouds, it is inferred that there is some sort of threshold or criteria involved in the matching process. In addition, Shreve teaches using the FPFH descriptors to estimate a 3D transformation matrix wherein “if the reference point and the transformed point have an L2 norm below a specified threshold (e.g., 2 mm), the transformed point is treated as an inlier, and otherwise as an outlier” para. [0041]). The motivation here would have been to provide a course global registration between the two point clouds before performing the fine alignment, as suggested by Robert in para. [0042]. Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to combine Robert with Shreve to obtain the invention specified in claim 2.
Regarding claim 7, Robert, Hu, and Shreve teach the method of claim 1,
further comprising downsampling a first captured point cloud into the first 3D point cloud, and downsampling a second captured point cloud into the second 3D point cloud (Shreve “The method also comprises downsampling 406 point cloud1 and point cloud2 to produce sparse point clouds 1 and 2” para. [0033]). The motivation here would have been to produce sparse point clouds in order for the FPFH descriptors to be computed in the coarse alignment process, as suggested by Shreve in para. [0040]. Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filed to combine Robert with Shreve to obtain the invention specified in claim 7.
Regarding claim 11, Robert teaches a system comprising:
a memory (Robert “memory” para. [0046]); and
one or more processors coupled to the memory (Robert “the computer 102 operates by the general purpose processor 104A performing instructions defined by the computer program 110 under control of an operating system 108. The computer program 110 and/or the operating system 108 may be stored in the memory 106” para. [0049]), wherein the one or more processors are configured to:
associate a first subset of points within a first 3D point cloud with a second subset of points within a second 3D point cloud (Robert “at step 302, a set of scans 304 (e.g., a first 3D point cloud and a second 3D point cloud) are acquired/input (e.g., via laser scanner 134/LiDAR or received from across a network 204). Such scans 304 are scans with normal (and such normal information may be retrieved). The two point clouds have a subset of points in common and there is no prior knowledge on an alignment between the two point clouds” para. [0065]) based on each point in the first subset of points having a threshold correspondence to a unique counterpart point in the second subset of points (Robert “Step 308 further includes determining matching pairs of descriptors (i.e., one descriptor from each point cloud in a pair). To determine matching pairs, a 4D kd-tree may be built using all of the descriptors. For the first descriptor, a nearest neighbor search may be performed in the tree for candidate descriptors from the second point cloud. Those descriptors within a threshold distance value are kept and stored” para. [0069]);
refine the first subset of points (Robert teaches removing outliers by creating hypotheses between matching pairs in the first and second point clouds. The hypotheses with the highest density are selected. This process is interpreted as the refining step in the claim language. See para. [0070]);
determine, based on point correspondences between the first improved subset of points and the second improved subset of points, a relative rotation and translation between the first 3D point cloud and the second 3D point cloud (Robert teaches descriptors which represent matching pairs in which “the descriptors are used to estimate rigid transformation hypotheses (i.e., at step 310). Each hypothesis represents a transformation between the first and second point cloud. Further, each hypothesis is based on the 3D information in the matching pair. Since there may be a lot of outliers, the transformations are accumulated into a fitted space and a search for zones with high density is conducted (e.g., similar to a classic Hough transform). Accordingly, one or more hypotheses are selected based on density. The most relevant transforms are then scored to define the correct alignment. In this regard, the selected hypotheses are validated based on a scoring at step 312. The scoring may be based on a translation as a first criterion and a rotation as a second criterion” para. [0070]; here, Robert teaches improving/refining the hypotheses and then validating the hypotheses (i.e. improved subset of points) based on a rotation and translation); and
align the first 3D point cloud and the second 3D point cloud within a common coordinate system based on the relative rotation and translation (Robert teaches a global registration process in para. [0064]; this process is interpreted as aligning the first and second point clouds within a common coordinate system, as it involves aligning two point clouds by correcting position and orientation; Robert additionally teaches alignment based on translation as a first criterion and rotation as a second criterion in para. [0070]).
Robert fails to teach generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views. Additionally, while Robert teaches refining the hypotheses by eliminating outliers, Robert fails to teach refining the first subset of points to generate a first improved subset of points that includes fewer outlier point correspondences to a second subset of points than the first subset of points, wherein refining the first subset of points includes searching for tiepoint triangles in the first subset of points having correspondence to tiepoint triangles in the second subset of points and refining the second subset of points to generate a second improved subset of points that includes fewer outlier point correspondences to the first subset of points than the second subset of points, wherein refining the second subset of points includes searching for tiepoint triangles in the second subset of points having correspondence to tiepoint triangles in the first subset of points.
However, Hu teaches refining the first subset of points to generate a first improved subset of points that includes fewer outlier point correspondences to a second subset of points than the first subset of points, wherein refining the first subset of points includes searching for tiepoint triangles in the first subset of points having correspondence to tiepoint triangles in the second subset of points and refining the second subset of points to generate a second improved subset of points that includes fewer outlier point correspondences to the first subset of points than the second subset of points, wherein refining the second subset of points includes searching for tiepoint triangles in the second subset of points having correspondence to tiepoint triangles in the first subset of points (Hu teaches a first set of feature points (M1) in a first image (l1) and a second set of feature points (M2) in a second image (l2) and determining the matching points of M1 and M2. These respective matched points of the first and second images are determined to be equivalent to the claimed first and second subset of points. See para. [0075]-[0077] regarding this matter. Subsequently Hu teaches a process of using a similar method triangles to “purify” the matching point pairs, by forming triangles in each image that contain matching pair point (tiepoints), and eliminating any pairs which do not satisfy the formula as shown in para. [0077]. Since pairs of points are eliminated in this process, it is determined that BOTH the first subset of points and the second subset of points are refined in this process in order to create a first improved subset of points and a second improved subset of points. Additionally, the process of searching for tiepoint triangles occurs such that the tiepoint triangles in the first subset of points have correspondence to tiepoint triangles in the second subset of points, and vice versa).
Robert and Hu are both considered to be analogous to the claimed invention because they are in the same field of registering images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert to incorporate the teachings of Hu and include “refining the first subset of points to generate a first improved subset of points that includes fewer outlier point correspondences to a second subset of points than the first subset of points, wherein refining the first subset of points includes searching for tiepoint triangles in the first subset of points having correspondence to tiepoint triangles in the second subset of points and refining the second subset of points to generate a second improved subset of points that includes fewer outlier point correspondences to the first subset of points than the second subset of points, wherein refining the second subset of points includes searching for tiepoint triangles in the second subset of points having correspondence to tiepoint triangles in the first subset of points”. The motivation would have been that “eliminating the mismatched point pair is performed to achieve the purification of the feature point pairs”, as suggested by Hu in para. [0077]. Therefore, it would have been obvious to combine Robert with Hu to obtain the invention specified in the above claim limitations.
Robert and Hu fail to teach generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views.
However, Shreve teaches generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views (Shreve “the 3D sensor 904 is representative of a variety of different 3D scanning devices including, for example… a time-of-flight 3D laser scanner” para. [0052]; “the method shown in FIG. 1 comprises obtaining 102 a first 3D point cloud associated with a physical object having at least one articulatable part. The first point cloud is associated with the physical object prior to articulation of the articulatable part. The method comprises obtaining 104 a second 3D point cloud associated with the physical object after articulation of the articulatable part” para. [0025]; the difference between the first and second point clouds is interpreted as different time-of-flight views)
Robert, Hu, and Shreve are all considered to be analogous to the claimed invention because they are in the same field of registering images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert (as modified by Hu) to incorporate the teachings of Shreve and “generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views” and “the refinement of the two point clouds separately”. The motivation for including “generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views” would have been to “generat[e] 112 an output comprising at least the remaining points of the second point cloud associated with the articulatable part without the noise points”, as suggested by Shreve in para. [0025]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert and Hu with Shreve to obtain the invention specified in claim 11.
Regarding claim 12, Robert, Hu, and Shreve teach the system of claim 11, wherein the one or more processors are further configured to:
generate first Fast Point Feature Histograms (FPFHs) for points in the first subset of points and second FPFHs for points in the second subset of points (Shreve “The method further comprises computing 408 Fast Point Feature Histogram (FPFH) descriptors from sparse point clouds 1 and 2” para. [0033]),
and associate a first point of the first subset of points with a second point of the second subset of points is based on determining whether a distance between a first FPFH descriptor for the first point in the first FPFHs and a second FPFH descriptor for the second point in the second FPFHs is within a threshold difference (Shreve “Fast Point Feature Histogram descriptors are computed on sparse point cloud1 and sparse point cloud2. Having computed the FPFH descriptors, the FPFH descriptors are matched across point cloud1 and point cloud2, and then used to align point cloud1 and point cloud2 by estimating a 3D transformation matrix on homogenous coordinates” para. [0040]; since the descriptors are matched across both point clouds, it is inferred that there is some sort of threshold or criteria involved in the matching process. In addition, Shreve teaches using the FPFH descriptors to estimate a 3D transformation matrix wherein “if the reference point and the transformed point have an L2 norm below a specified threshold (e.g., 2 mm), the transformed point is treated as an inlier, and otherwise as an outlier” para. [0041]). The motivation here would have been to provide a course global registration between the two point clouds before performing the fine alignment, as suggested by Robert in para. [0042]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert and Hu with Shreve to obtain the invention specified in claim 12.
Regarding claim 17, Robert, Hu, and Shreve teach the system of claim 11,
wherein the one or more processors are further configured to downsample a first captured point cloud into the first 3D point cloud, and downsample a second captured point cloud into the second 3D point cloud (Shreve “The method also comprises downsampling 406 point cloud1 and point cloud2 to produce sparse point clouds 1 and 2” para. [0033]). The motivation here would have been to produce sparse point clouds in order for the FPFH descriptors to be computed in the coarse alignment process, as suggested by Shreve in para. [0040]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert and Hu with Shreve to obtain the invention specified in claim 17.
Regarding claim 19, Robert teaches a non-transitory computer-readable medium, comprising code executable by one or more processors (Robert teaches that “the computer 102 comprises a general purpose hardware processor 104A and/or a special purpose hardware processor 104B (hereinafter alternatively collectively referred to as processor 104) and a memory 106, such as random access memory (RAM).” para. [0046]) for aligning multiple three-dimensional (3D) point clouds into a common 3D point cloud, the code comprising code for:
associating a first subset of points within a first 3D point cloud with a second subset of points within a second 3D point cloud (Robert “at step 302, a set of scans 304 (e.g., a first 3D point cloud and a second 3D point cloud) are acquired/input (e.g., via laser scanner 134/LiDAR or received from across a network 204). Such scans 304 are scans with normal (and such normal information may be retrieved). The two point clouds have a subset of points in common and there is no prior knowledge on an alignment between the two point clouds” para. [0065]) based on each point in the first subset of points having a threshold correspondence to a unique counterpart point in the second subset of points (Robert “Step 308 further includes determining matching pairs of descriptors (i.e., one descriptor from each point cloud in a pair). To determine matching pairs, a 4D kd-tree may be built using all of the descriptors. For the first descriptor, a nearest neighbor search may be performed in the tree for candidate descriptors from the second point cloud. Those descriptors within a threshold distance value are kept and stored” para. [0069]);
refining the first subset of points (Robert teaches removing outliers by creating hypotheses between matching pairs in the first and second point clouds. The hypotheses with the highest density are selected. This process is interpreted as the refining step in the claim language. See para. [0070]);
determining, based on point correspondences between the first improved subset of points and the second improved subset of points, a relative rotation and translation between the first 3D point cloud and the second 3D point cloud (Robert teaches descriptors which represent matching pairs in which “the descriptors are used to estimate rigid transformation hypotheses (i.e., at step 310). Each hypothesis represents a transformation between the first and second point cloud. Further, each hypothesis is based on the 3D information in the matching pair. Since there may be a lot of outliers, the transformations are accumulated into a fitted space and a search for zones with high density is conducted (e.g., similar to a classic Hough transform). Accordingly, one or more hypotheses are selected based on density. The most relevant transforms are then scored to define the correct alignment. In this regard, the selected hypotheses are validated based on a scoring at step 312. The scoring may be based on a translation as a first criterion and a rotation as a second criterion” para. [0070]; here, Robert teaches improving/refining the hypotheses and then validating the hypotheses (i.e. improved subset of points) based on a rotation and translation); and
aligning the first 3D point cloud and the second 3D point cloud within a common coordinate system based on the relative rotation and translation (Robert teaches a global registration process in para. [0064]; this process is interpreted as aligning the first and second point clouds within a common coordinate system, as it involves aligning two point clouds by correcting position and orientation; Robert additionally teaches alignment based on translation as a first criterion and rotation as a second criterion in para. [0070]).
Robert fails to teach generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views. Additionally, while Robert teaches refining the hypotheses by eliminating outliers, Robert fails to teach refining the first subset of points to generate a first improved subset of points that includes fewer outlier point correspondences to a second subset of points than the first subset of points, wherein refining the first subset of points includes searching for tiepoint triangles in the first subset of points having correspondence to tiepoint triangles in the second subset of points and refining the second subset of points to generate a second improved subset of points that includes fewer outlier point correspondences to the first subset of points than the second subset of points, wherein refining the second subset of points includes searching for tiepoint triangles in the second subset of points having correspondence to tiepoint triangles in the first subset of points.
However, Hu teaches refining the first subset of points to generate a first improved subset of points that includes fewer outlier point correspondences to a second subset of points than the first subset of points, wherein refining the first subset of points includes searching for tiepoint triangles in the first subset of points having correspondence to tiepoint triangles in the second subset of points and refining the second subset of points to generate a second improved subset of points that includes fewer outlier point correspondences to the first subset of points than the second subset of points, wherein refining the second subset of points includes searching for tiepoint triangles in the second subset of points having correspondence to tiepoint triangles in the first subset of points (Hu teaches a first set of feature points (M1) in a first image (l1) and a second set of feature points (M2) in a second image (l2) and determining the matching points of M1 and M2. These respective matched points of the first and second images are determined to be equivalent to the claimed first and second subset of points. See para. [0075]-[0077] regarding this matter. Subsequently Hu teaches a process of using a similar method triangles to “purify” the matching point pairs, by forming triangles in each image that contain matching pair point (tiepoints), and eliminating any pairs which do not satisfy the formula as shown in para. [0077]. Since pairs of points are eliminated in this process, it is determined that BOTH the first subset of points and the second subset of points are refined in this process in order to create a first improved subset of points and a second improved subset of points. Additionally, the process of searching for tiepoint triangles occurs such that the tiepoint triangles in the first subset of points have correspondence to tiepoint triangles in the second subset of points, and vice versa).
Robert and Hu are both considered to be analogous to the claimed invention because they are in the same field of registering images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert to incorporate the teachings of Hu and include “refining the first subset of points to generate a first improved subset of points that includes fewer outlier point correspondences to a second subset of points than the first subset of points, wherein refining the first subset of points includes searching for tiepoint triangles in the first subset of points having correspondence to tiepoint triangles in the second subset of points and refining the second subset of points to generate a second improved subset of points that includes fewer outlier point correspondences to the first subset of points than the second subset of points, wherein refining the second subset of points includes searching for tiepoint triangles in the second subset of points having correspondence to tiepoint triangles in the first subset of points”. The motivation would have been that “eliminating the mismatched point pair is performed to achieve the purification of the feature point pairs”, as suggested by Hu in para. [0077]. Therefore, it would have been obvious to combine Robert with Hu to obtain the invention specified in the above claim limitations.
Robert and Hu fail to teach generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views.
However, Shreve teaches generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views (Shreve “the 3D sensor 904 is representative of a variety of different 3D scanning devices including, for example… a time-of-flight 3D laser scanner” para. [0052]; “the method shown in FIG. 1 comprises obtaining 102 a first 3D point cloud associated with a physical object having at least one articulatable part. The first point cloud is associated with the physical object prior to articulation of the articulatable part. The method comprises obtaining 104 a second 3D point cloud associated with the physical object after articulation of the articulatable part” para. [0025]; the difference between the first and second point clouds is interpreted as different time-of-flight views)
Robert, Hu, and Shreve are all considered to be analogous to the claimed invention because they are in the same field of registering images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert (as modified by Hu) to incorporate the teachings of Shreve and “generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views” and “the refinement of the two point clouds separately”. The motivation for including “generating a first and second 3D point cloud generated from differing time-of-flight (TOF) views” would have been to “generat[e] 112 an output comprising at least the remaining points of the second point cloud associated with the articulatable part without the noise points”, as suggested by Shreve in para. [0025]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert and Hu with Shreve to obtain the invention specified in claim 19.
Regarding claim 20, Robert, Hu, and Shreve teach the non-transitory computer-readable medium of claim 19, further comprising code for:
generating first Fast Point Feature Histograms (FPFHs) for points in the first subset of points and second FPFHs for points in the second subset of points (Shreve “The method further comprises computing 408 Fast Point Feature Histogram (FPFH) descriptors from sparse point clouds 1 and 2” para. [0033]),
wherein associating a first point of the first subset of points with a second point of the second subset of points is based on determining whether a distance between a first FPFH descriptor for the first point in the first FPFHs and a second FPFH descriptor for the second point in the second FPFHs is within a threshold difference (Shreve “Fast Point Feature Histogram descriptors are computed on sparse point cloud1 and sparse point cloud2. Having computed the FPFH descriptors, the FPFH descriptors are matched across point cloud1 and point cloud2, and then used to align point cloud1 and point cloud2 by estimating a 3D transformation matrix on homogenous coordinates” para. [0040]; since the descriptors are matched across both point clouds, it is inferred that there is some sort of threshold or criteria involved in the matching process. In addition, Shreve teaches using the FPFH descriptors to estimate a 3D transformation matrix wherein “if the reference point and the transformed point have an L2 norm below a specified threshold (e.g., 2 mm), the transformed point is treated as an inlier, and otherwise as an outlier” para. [0041]). The motivation here would have been to provide a course global registration between the two point clouds before performing the fine alignment, as suggested by Robert in para. [0042]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert and Hu with Shreve to obtain the invention specified in claim 20.
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Robert et al. (WO 2017/096299 A1), hereinafter Robert, in view of Hu et al. (CN 1046370055 B, see attached English translation for citations), hereinafter Hu, Shreve et al. (U.S. Publication No. 2021/0142039 A1), hereinafter Shreve, and Wang et al. (NPL: Point cloud simplification algorithm based on the feature of adaptive curvature entropy, 2021), hereinafter Wang.
Regarding claim 3, Robert, Hu, and Shreve teach the method of claim 1, further comprising:
extracting the first subset of points from the first 3D point cloud (Robert “Step 308 further includes determining matching pairs of descriptors (i.e., one descriptor from each point cloud in a pair). To determine matching pairs, a 4D kd-tree may be built using all of the descriptors. For the first descriptor, a nearest neighbor search may be performed in the tree for candidate descriptors from the second point cloud. Those descriptors within a threshold distance value are kept and stored” para. [0069]); and
extracting the second subset of points from the second 3D point cloud (Robert “a nearest neighbor search may be performed in the tree for candidate descriptors from the second point cloud. Those descriptors within a threshold distance value are kept and stored” para. [0069]),
wherein associating the first subset of points and the second subset of points includes evaluating the first subset of points and the second subset of points for descriptor correspondence (Robert “To determine matching pairs, a 4D kd-tree may be built using all of the descriptors. For the first descriptor, a nearest neighbor search may be performed in the tree for candidate descriptors from the second point cloud. Those descriptors within a threshold distance value are kept and stored” para. [0069]).
While Robert teaches a threshold descriptor (see para. [0069]), Robert, Hu, and Shreve fail to teach having at least a threshold descriptor entropy.
However, Wang teaches having at least a threshold descriptor entropy (Wang “The entropy of the curvature for any point in the point cloud is closely related to the number of its neighbors. The secondary refinement is implemented based on the initial simplification. Thus, it is very important to set a reasonable threshold T to filter the less important points so that more feature points can be preserved in the secondary simplification. The entropy threshold T is determined as: T = λS · log10K.” sec. 2.4)
Robert, Hu, Shreve, and Wang are all considered to be analogous to the claimed invention because they are in the same field of refining point clouds. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert (as modified by Hu and Shreve) to incorporate the teachings of Wang and include a threshold descriptor entropy. The motivation for doing so would have been to “improve the point cloud simplification accuracy”, as suggested by Wang in sec. 3.2. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert, Hu, and Shreve with Wang to obtain the invention specified in claim 3.
Regarding claim 13, Robert, Hu, and Shreve teach the system of claim 11, wherein the one or more processors are further configured to:
extract the first subset of points from the first 3D point cloud (Robert “Step 308 further includes determining matching pairs of descriptors (i.e., one descriptor from each point cloud in a pair). To determine matching pairs, a 4D kd-tree may be built using all of the descriptors. For the first descriptor, a nearest neighbor search may be performed in the tree for candidate descriptors from the second point cloud. Those descriptors within a threshold distance value are kept and stored” para. [0069]); and
extract the second subset of points from the second 3D point cloud (Robert “a nearest neighbor search may be performed in the tree for candidate descriptors from the second point cloud. Those descriptors within a threshold distance value are kept and stored” para. [0069]),
associate the first subset of points and the second subset of points includes evaluating the first subset of points and the second subset of points for descriptor correspondence (Robert “To determine matching pairs, a 4D kd-tree may be built using all of the descriptors. For the first descriptor, a nearest neighbor search may be performed in the tree for candidate descriptors from the second point cloud. Those descriptors within a threshold distance value are kept and stored” para. [0069]).
While Robert teaches a threshold descriptor (see para. [0069]), Robert, Hu, and Shreve fail to teach having at least a threshold descriptor entropy.
However, Wang teaches having at least a threshold descriptor entropy (Wang “The entropy of the curvature for any point in the point cloud is closely related to the number of its neighbors. The secondary refinement is implemented based on the initial simplification. Thus, it is very important to set a reasonable threshold T to filter the less important points so that more feature points can be preserved in the secondary simplification. The entropy threshold T is determined as: T = λS · log10K.” sec. 2.4)
Robert, Hu, Shreve, and Wang are all considered to be analogous to the claimed invention because they are in the same field of refining point clouds. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert (as modified by Hu and Shreve) to incorporate the teachings of Wang and include a threshold descriptor entropy. The motivation for doing so would have been to “improve the point cloud simplification accuracy”, as suggested by Wang in sec. 3.2. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert, Hu, and Shreve with Wang to obtain the invention specified in claim 13.
Claims 4-6 and 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Robert et al. (WO 2017/096299 A1), hereinafter Robert, in view of Hu et al. (CN 1046370055 B, see attached English translation for citations), hereinafter Hu, Shreve et al. (U.S. Publication No. 2021/0142039 A1), hereinafter Shreve, and El Banani et al. (NPL: UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering), hereinafter El Banani.
Regarding claim 4, Robert, Hu, and Shreve teach the method of claim 1, wherein refining the first subset of points and the second subset of points includes:
using a test on 33-dimensional descriptors of the first subset of points and the second subset of points to compare points in the first subset of points and the second subset of points having point correspondence (Shreve teaches generating FPFH descriptors which are inherently 33-dimensional and using a RANSAC methodology to compare the points in the first subset and the second subset based on point correspondence in para. [0040-0042]); and
retaining, in the first improved subset of points and the second improved subset of points, points that pass the test (Hu teaches retaining points in a first and second improved subset of points that satisfy a formula in para. [0077]. Shreve additionally teaches refining parameters in order to yield the fewest outliers and retaining the reference point and the transformed point that “have an L2 norm below a specified threshold (e.g., 2 mm), the transformed point is treated as an inlier, and otherwise as an outlier” para. [0041]).
Robert, Hu, and Shreve fail to specifically teach the test being a Lowe’s ratio test.
However, El Banani teaches using a Lowe’s ratio test to estimate weight correspondences between point clouds (El Banani “we use Lowe’s ratio test to estimate the weight for each correspondence. We ablate this component by instead using the feature distance between the corresponding points to rank the correspondences” sec. 4.2 Ablations, Ratio Test)
Robert, Hu, Shreve, and El Banani are all considered to be analogous to the claimed invention because they are in the same field of aligning point clouds. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert (as modified by Hu and Shreve) to incorporate the teachings of El Banani and include a Lowe’s ratio test. The motivation for doing so would have been that “Lowe’s ratio test shows incredible efficacy in determining correspondence weights; a function often undertaken by far more complex models in recent work”, as suggested by El Banani in sec. 4.2 Ablations, Ratio Test. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert, Hu, and Shreve with El Banani to obtain the invention specified in claim 4.
Regarding claim 5, Robert, Hu, and Shreve teach the method of claim 1.
While Robert, Hu, and Shreve teach retaining, in the first improved subset of points, the first portion of points and, in the second improved subset of points, the second portion of points, if a difference between the first distance and second distance is less than a threshold (Hu teaches retaining points in a first and second improved subset of points that satisfy a formula in para. [0077]. Shreve additionally teaches refinement based on “the RANSAC methodology [which] uses a sample consensus algorithm to find a set of parameters that yields the fewest outliers according to a specified distance criterion” para. [0041]) (Robert teaches “For the first descriptor, a nearest neighbor search may be performed in the tree for candidate descriptors from the second point cloud. Those descriptors within a threshold distance value are kept and stored” para. [0069]), Robert, Hu, and Shreve fail to teach wherein refining the first subset of points and the second subset of points includes: comparing a first distance between a first portion of points from the first subset of points with a second distance between a second portion of corresponding points from the second subset of points.
However, El Banani teaches wherein refining the first subset of points and the second subset of points includes: comparing a first distance between a first portion of points from the first subset of points with a second distance between a second portion of corresponding points from the second subset of points (El Banani teaches “for each point in p ∈ P, we would like to find the point qp such that qp = arg min q∈Q D(fp,fq), (1) where D(p, q) is a distance metric defined on the feature space” … “as a result, the quality of correspondence (p, qp) is not simply determined by D(p, qp), but rather between the ratio r which is defined as r = D(p, qp,1) D(p, qp,2) , (2) where qp,i is the i-th nearest neighbor to point p in Q” in sec. 3.2 Correspondence Estimation; here, the ratio is comparing a distance between point p in point cloud P to the ith nearest neighbor, q, in point cloud Q. Point cloud P is equivalent to a first subset of points and point cloud Q is equivalent to a second subset of points; it is also taught that the distance comparisons are conducted for both point cloud P (first subset) and point cloud Q (second subset): “we extract such correspondences for all points in both P and Q since correspondence is not guaranteed to be bijective. As a result, we have two sets of correspondences, CP→Q and CQ→P , where each set consists of N pairs” sec 3.2 Correspondence Estimation).
Robert, Hu, Shreve, and El Banani are all considered to be analogous to the claimed invention because they are in the same field of aligning point clouds. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert (as modified by Hu and Shreve) to incorporate the teachings of El Banani and include “wherein refining the first subset of points and the second subset of points includes: comparing a first distance between a first portion of points from the first subset of points with a second distance between a second portion of corresponding points from the second subset of points”. The motivation for doing so would have been to “best align the first input image to the second” and “increase the effectiveness and robust ness of a registration system”, as suggested by El Banani in sec. 4.1 Pairwise Registration. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert, Hu, and Shreve with El Banani to obtain the invention specified in claim 5.
Regarding claim 6, Robert, Hu, and Shreve teach the method of claim 1.
While Robert, Hu, and Shreve teach retaining, in the first improved subset of points, the first portion of points and, in the second improved subset of points, the second portion of points, if a difference between the first distance and second distance is less than a threshold (Hu teaches retaining points in a first and second improved subset of points that satisfy a formula in para. [0077]. Shreve additionally teaches refinement based on “the RANSAC methodology [which] uses a sample consensus algorithm to find a set of parameters that yields the fewest outliers according to a specified distance criterion” para. [0041]) (Robert teaches “For the first descriptor, a nearest neighbor search may be performed in the tree for candidate descriptors from the second point cloud. Those descriptors within a threshold distance value are kept and stored” para. [0069]), Robert, Hu, and Shreve fail to teach wherein refining the first subset of points and the second subset of points includes: for each first portion of points in the first subset of points and second portion of points in the second subset of points having separation distances less than a threshold, comparing the separation distances between a third point in the first subset of points and the first portion of points with counterpart separation distances between the corresponding third point in the second subset of points and the second portion of points; and retaining, in the first improved subset of points, the first portion of points and the third point in the first subset of points and, in the second improved subset of points, the second portion of points and the corresponding third point in the second subset of points, if the separation distances and the counterpart separation distances are less than a threshold.
However, El Banani teaches wherein refining the first subset of points and the second subset of points includes:
for each first portion of points in the first subset of points and second portion of points in the second subset of points having separation distances less than a threshold, comparing the separation distances between a third point in the first subset of points and the first portion of points with counterpart separation distances between the corresponding third point in the second subset of points and the second portion of points (El Banani teaches “for each point in p ∈ P, we would like to find the point qp such that qp = arg min q∈Q D(fp,fq), (1) where D(p, q) is a distance metric defined on the feature space” … “as a result, the quality of correspondence (p, qp) is not simply determined by D(p, qp), but rather between the ratio r which is defined as r = D(p, qp,1) D(p, qp,2) , (2) where qp,i is the i-th nearest neighbor to point p in Q” in sec. 3.2 Correspondence Estimation; here, the ratio is comparing a distance between point p in point cloud P to the ith nearest neighbor, q, in point cloud Q. Point cloud P is equivalent to a first subset of points and point cloud Q is equivalent to a second subset of points; El Banani teaches that it is most common to define a distance ratio threshold for inlier vs outliers in order to retain a number of correspondences in sec. 3.2 Correspondence Estimation, Ratio Test; since this process is repeated for every point in p, it can be inferred that there exists a third point in P that is compared with a distance from a counterpart third point in Q; here, the 2nd nearest neighbor to point p in Q (or point q in P) is interpreted as the third point in the second subset of points); and
retaining, in the first improved subset of points, the first portion of points and the third point in the first subset of points and, in the second improved subset of points, the second portion of points and the corresponding third point in the second subset of points, if the separation distances and the counterpart separation distances are less than a threshold (El Banani teaches, “since 0 ≤ rp ≤ 1 and a lower ratio indicates a better match, we weigh each correspondence by w = 1 – r”; here, if the third point (see above citation) is a better match than the second point, the third point is weighted higher than the second point and is retained if the weight is above a certain threshold; see 3.2 Correspondence Estimation, Ratio Test).
Robert, Hu, Shreve, and El Banani are all considered to be analogous to the claimed invention because they are in the same field of aligning point clouds. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert (as modified by Hu and Shreve) to incorporate the teachings of El Banani and include “wherein refining the first subset of points and the second subset of points includes: for each first portion of points in the first subset of points and second portion of points in the second subset of points having separation distances less than a threshold, comparing the separation distances between a third point in the first subset of points and the first portion of points with counterpart separation distances between the corresponding third point in the second subset of points and the second portion of points; and retaining, in the first improved subset of points, the first portion of points and the third point in the first subset of points and, in the second improved subset of points, the second portion of points and the corresponding third point in the second subset of points, if the separation distances and the counterpart separation distances are less than a threshold”. The motivation for doing so would have been to “best align the first input image to the second” and “increase the effectiveness and robust ness of a registration system”, as suggested by El Banani in sec. 4.1 Pairwise Registration. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert, Hu, and Shreve with El Banani to obtain the invention specified in claim 6.
Regarding claim 14, Robert, Hu, and Shreve teach the system of claim 11, wherein the one or more processors are configured to refine the first subset of points and the second subset of points at least in part by:
using a test on 33-dimensional descriptors of the first subset of points and the second subset of points to compare points in the first subset of points and the second subset of points having point correspondence (Shreve teaches generating FPFH descriptors which are inherently 33-dimensional and using a RANSAC methodology to compare the points in the first subset and the second subset based on point correspondence in para. [0040-0042]); and
retaining, in the first improved subset of points and the second improved subset of points, points that pass the test (Shreve teaches refining parameters in order to yield the fewest outliers and retaining the reference point and the transformed point that “have an L2 norm below a specified threshold (e.g., 2 mm), the transformed point is treated as an inlier, and otherwise as an outlier” para. [0041]).
Robert, Hu, and Shreve fail to specifically teach the test being a Lowe’s ratio test.
However, El Banani teaches using a Lowe’s ratio test to estimate weight correspondences between point clouds (El Banani “we use Lowe’s ratio test to estimate the weight for each correspondence. We ablate this component by instead using the feature distance between the corresponding points to rank the correspondences” sec. 4.2 Ablations, Ratio Test)
Robert, Hu, Shreve, and El Banani are all considered to be analogous to the claimed invention because they are in the same field of aligning point clouds. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert (as modified by Hu and Shreve) to incorporate the teachings of El Banani and include a Lowe’s ratio test. The motivation for doing so would have been that “Lowe’s ratio test shows incredible efficacy in determining correspondence weights; a function often undertaken by far more complex models in recent work”, as suggested by El Banani in sec. 4.2 Ablations, Ratio Test. Therefore, it would have been obvious to combine Robert, Hu, and Shreve with El Banani to obtain the invention specified in claim 14.
Regarding claim 15, Shreve and Robert teach the system of claim 11.
While Robert and Shreve teach retaining, in the first improved subset of points, the first portion of points and, in the second improved subset of points, the second portion of points, if a difference between the first distance and second distance is less than a threshold (Hu teaches retaining points in a first and second improved subset of points based on distances between matching points in corresponding triangles between the first and second portion of points in para. [0077]) (Shreve additionally teaches refinement based on “the RANSAC methodology [which] uses a sample consensus algorithm to find a set of parameters that yields the fewest outliers according to a specified distance criterion” para. [0041]) (Robert teaches “For the first descriptor, a nearest neighbor search may be performed in the tree for candidate descriptors from the second point cloud. Those descriptors within a threshold distance value are kept and stored” para. [0069]), Robert and Shreve fail to teach wherein refining the first subset of points and the second subset of points includes: comparing a first distance between a first portion of points from the first subset of points with a second distance between a second portion of corresponding points from the second subset of points.
However, El Banani teaches wherein the one or more processors are configured to refine the first subset of points and the second subset of points at least in part by: comparing a first distance between a first portion of points from the first subset of points with a second distance between a second portion of corresponding points from the second subset of points (El Banani teaches “for each point in p ∈ P, we would like to find the point qp such that qp = arg min q∈Q D(fp,fq), (1) where D(p, q) is a distance metric defined on the feature space” … “as a result, the quality of correspondence (p, qp) is not simply determined by D(p, qp), but rather between the ratio r which is defined as r = D(p, qp,1) D(p, qp,2) , (2) where qp,i is the i-th nearest neighbor to point p in Q” in sec. 3.2 Correspondence Estimation; here, the ratio is comparing a distance between point p in point cloud P to the ith nearest neighbor, q, in point cloud Q. Point cloud P is equivalent to a first subset of points and point cloud Q is equivalent to a second subset of points; it is also taught that the distance comparisons are conducted for both point cloud P (first subset) and point cloud Q (second subset): “we extract such correspondences for all points in both P and Q since correspondence is not guaranteed to be bijective. As a result, we have two sets of correspondences, CP→Q and CQ→P , where each set consists of N pairs” sec 3.2 Correspondence Estimation)
Robert, Hu, Shreve, and El Banani are all considered to be analogous to the claimed invention because they are in the same field of aligning points in images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert (as modified by Hu and Shreve) to incorporate the teachings of El Banani and include “wherein the one or more processors are configured to refine the first subset of points and the second subset of points at least in part by: comparing a first distance between a first portion of points from the first subset of points with a second distance between a second portion of corresponding points from the second subset of points”. The motivation for doing so would have been to “best align the first input image to the second” and “increase the effectiveness and robust ness of a registration system”, as suggested by El Banani in sec. 4.1 Pairwise Registration. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert, Hu, and Shreve with El Banani to obtain the invention specified in claim 15.
Regarding claim 16, Robert, Hu, and Shreve teach the system of claim 11.
While Robert, Hu, and Shreve teach retaining, in the first improved subset of points, the first portion of points and, in the second improved subset of points, the second portion of points, if a difference between the first distance and second distance is less than a threshold (Hu teaches retaining points in a first and second improved subset of points based on distances between matching points in corresponding triangles between the first and second portion of points in para. [0077]) (Shreve additionally teaches refinement based on “the RANSAC methodology [which] uses a sample consensus algorithm to find a set of parameters that yields the fewest outliers according to a specified distance criterion” para. [0041]) (Robert teaches “For the first descriptor, a nearest neighbor search may be performed in the tree for candidate descriptors from the second point cloud. Those descriptors within a threshold distance value are kept and stored” para. [0069]), Robert and Shreve fail to teach wherein refining the first subset of points and the second subset of points includes: for each first portion of points in the first subset of points and second portion of points in the second subset of points having separation distances less than a threshold, comparing the separation distances between a third point in the first subset of points and the first portion of points with counterpart separation distances between the corresponding third point in the second subset of points and the second portion of points; and retaining, in the first improved subset of points, the first portion of points and the third point in the first subset of points and, in the second improved subset of points, the second portion of points and the corresponding third point in the second subset of points, if the separation distances and the counterpart separation distances are less than a threshold.
However, El Banani teaches wherein the one or more processors are configured to refine the first subset of points and the second subset of points at least in part by:
for each first portion of points in the first subset of points and second portion of points in the second subset of points having separation distances less than a threshold, comparing the separation distances between a third point in the first subset of points and the first portion of points with counterpart separation distances between the corresponding third point in the second subset of points and the second portion of points (El Banani teaches “for each point in p ∈ P, we would like to find the point qp such that qp = arg min q∈Q D(fp,fq), (1) where D(p, q) is a distance metric defined on the feature space” … “as a result, the quality of correspondence (p, qp) is not simply determined by D(p, qp), but rather between the ratio r which is defined as r = D(p, qp,1) D(p, qp,2) , (2) where qp,i is the i-th nearest neighbor to point p in Q” in sec. 3.2 Correspondence Estimation; here, the ratio is comparing a distance between point p in point cloud P to the ith nearest neighbor, q, in point cloud Q. Point cloud P is equivalent to a first subset of points and point cloud Q is equivalent to a second subset of points; El Banani teaches that it is most common to define a distance ratio threshold for inlier vs outliers in order to retain a number of correspondences in sec. 3.2 Correspondence Estimation, Ratio Test; since this process is repeated for every point in p, it can be inferred that there exists a third point in P that is compared with a distance from a counterpart third point in Q; here, the 2nd nearest neighbor to point p in Q (or point q in P) is interpreted as the third point in the second subset of points); and
retaining, in the first improved subset of points, the first portion of points and the third point in the first subset of points and, in the second improved subset of points, the second portion of points and the corresponding third point in the second subset of points, if the separation distances and the counterpart separation distances are less than a threshold (El Banani teaches, “since 0 ≤ rp ≤ 1 and a lower ratio indicates a better match, we weigh each correspondence by w = 1 – r”; here, if the third point (see above citation) is a better match than the second point, the third point is weighted higher than the second point and is retained if the weight is above a certain threshold; see 3.2 Correspondence Estimation, Ratio Test).
Robert, Hu, Shreve, and El Banani are all considered to be analogous to the claimed invention because they are in the same field of aligning points in images. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert (as modified by Hu and Shreve) to incorporate the teachings of El Banani and include “wherein refining the first subset of points and the second subset of points includes: for each first portion of points in the first subset of points and second portion of points in the second subset of points having separation distances less than a threshold, comparing the separation distances between a third point in the first subset of points and the first portion of points with counterpart separation distances between the corresponding third point in the second subset of points and the second portion of points; and retaining, in the first improved subset of points, the first portion of points and the third point in the first subset of points and, in the second improved subset of points, the second portion of points and the corresponding third point in the second subset of points, if the separation distances and the counterpart separation distances are less than a threshold”. The motivation for doing so would have been to “best align the first input image to the second” and “increase the effectiveness and robust ness of a registration system”, as suggested by El Banani in sec. 4.1 Pairwise Registration. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert, Hu, and Shreve with El Banani to obtain the invention specified in claim 16.
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Robert et al. (WO 2017/096299 A1), hereinafter Robert, in view of Hu et al. (CN 1046370055 B, see attached English translation for citations), hereinafter Hu, Shreve et al. (U.S. Publication No. 2021/0142039 A1), hereinafter Shreve, and Ebihara (U.S. Publication No. 2020/0257888 A1).
Regarding claim 8, Robert, Hu, and Shreve teach the method of claim 1,
further comprising aligning one or more 3D point clouds with the first 3D point cloud in a common coordinate system using relative rotations and translations calculated via motion synchronization (Robert teaches a scalable global registration process which is interpreted as equivalent to motion synchronization in para. [0043]).
Shreve, Hu, and Robert fail to teach bundle adjustment.
However, Ebihara teaches “bundle adjustment” in para. [0089], Ebihara explains the process of bundle adjustment in para. [0090].
Robert, Hu, Shreve, and Ebihara are all considered to be analogous to the claimed invention because they are in the same field of mapping points between three-dimensional objects. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert (as modified by Hu and Shreve) to incorporate the teachings of Ebihara and include the teaching of bundle adjustment. The motivation for doing so would have been to accurately determine “the three-dimensional position of a camera held when each of a plurality of image frames is captured and a three-dimensional shape of a target subject are calculated based on a movement of the target subject included in the plurality of image frames”, as suggested by Ebihara in para. [0090], in order to better align the point clouds taught by Robert, Hu, and Shreve. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert, Hu, and Shreve with Ebihara to obtain the invention specified in claim 8.
Regarding claim 18, Robert, Hu, and Shreve teach the system of claim 11,
wherein the one or more processors are further configured to align one or more 3D point clouds with the first 3D point cloud in a common coordinate system using relative rotations and translations calculated via motion synchronization (Robert teaches a scalable global registration process which is interpreted as equivalent to motion synchronization in para. [0043]).
Shreve, Hu, and Robert fail to teach bundle adjustment.
However, Ebihara teaches “bundle adjustment” in para. [0089], Ebihara explains the process of bundle adjustment in para. [0090].
Robert, Hu, Shreve, and Ebihara are all considered to be analogous to the claimed invention because they are in the same field of mapping points between three-dimensional objects. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert and Shreve to incorporate the teachings of Ebihara and include the teaching of bundle adjustment. The motivation for doing so would have been to accurately determine “the three-dimensional position of a camera held when each of a plurality of image frames is captured and a three-dimensional shape of a target subject are calculated based on a movement of the target subject included in the plurality of image frames”, as suggested by Ebihara in para. [0090], in order to better align the point clouds taught by Robert and Shreve. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert, Hu, and Shreve with Ebihara to obtain the invention specified in claim 18.
Claims 9 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Robert et al. (WO 2017/096299 A1), hereinafter Robert, in view of Hu et al. (CN 1046370055 B, see attached English translation for citations), hereinafter Hu, Shreve et al. (U.S. Publication No. 2021/0142039 A1), hereinafter Shreve, and Fu et al. (U.S. Publication No. 2018/0174325 A1), hereinafter Fu.
Regarding claim 9, Robert, Hu, and Shreve teach the method of claim 1, and the common coordinate system (Robert teaches a global registration process in para. [0064]; this process is interpreted as aligning the first and second point clouds within a common coordinate system, as it involves aligning two point clouds by correcting position and orientation).
Robert, Hu, and Shreve fail to teach computing dimensional measurements of an object.
However, Fu teaches computing dimensional measurements of an object (Fu teaches a point cloud alignment process in which the geometry of an objected is measured, wherein the measurements include a length, width, and height (see para. [0118])).
Robert, Hu, Shreve, and Fu are all considered to be analogous to the claimed invention because they are in the same field of mapping points between three-dimensional objects. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Robert and Shreve to incorporate the teachings of Fu and include computing dimensional measurements of an object. The motivation for doing so would have been to accurately determine “characteristics of the packages such as, for example, a weight of a package, a shape of package, and/or one or more dimensions of a package”, as suggested by Fu in para. [0019]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Robert, Hu, and Shreve with Fu to obtain the invention specified in claim 9.
Regarding claim 10, Robert, Hu, Shreve, and Fu teach the method of claim 9,
further comprising displaying an indication of the dimensional measurements (Fu teaches that “the controller 220 and/or the freight dimensioner 130 communicates data to one or more display device to render the dimensions of the items represented by the second portion 1050” in para. [0110]; see also para. [0141]). Similar motivations as applied to claim 9 can be applied here to claim 10.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Li (CN 103903249 B) teaches a method of determining tiepoint triangles in order to match images.
Yang et al. (CN113205548A) teaches a method of determining matched homonymous triangles according to the similarity of the side length and the feature descriptors of the fast point feature histogram.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYLA G ALLEN whose telephone number is (703)756-5315. The examiner can normally be reached M-F 7:30am - 4:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a
USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use
the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor,
John Villecco can be reached on (571) 272-7319. The fax phone number for the organization where this
application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from
Patent Center. Unpublished application information in Patent Center is available to registered users. To
file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and
https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/Kyla Guan-Ping Tiao Allen/
Examiner, Art Unit 2661
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661
Prosecution Insights