Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is in response to the amendment filed 02/02/2026. Claims 1, 3, 4, and 8 have been amended while claim 7 has been cancelled. Amendments have been fully considered but are not persuasive. Claims 1-4, 6 and 17-18 remain rejected in the application while claims 5 and 8-16 are objected to.
Response to Arguments
In response to applicant’s arguments regarding Examiner indicating claim 7 being allowable. Arguments fully considered but is not persuasive. Examiner indicated that ALL of the limitations within claim 7 would be allowable if placed into independent form which includes the claims it depends from such as claim 3 and 4. Applicant failed to include the limitations of claims 3 and 4 which claim 7 depends on.
In response to applicant’s arguments regarding Fu failing to teach generating a centerline parameter set. Arguments fully considered but is not persuasive. Fu explicitly teaches those limitations [Fu: 0063 “The information of the detected cylinder shapes includes: axis vectors, centers, diameters, lengths. This information can be used for further pipe run modeling”].
In response to applicant’s arguments regarding Fu failing to teach the parameter set for each point cloud patch/seed point. Arguments fully considered but is not persuasive. Fu explicitly teaches those limitations [Fu: 0062 “Another seed is selected to detect a new shape (i.e., at step 204). This process is iterated until all the points have been marked as “visited' (i.e., once all points have been checked as determined at step 202)”] (Fu explicitly teaches until all the points have been checked). Claims 1-4, 6 and 17-18 remain rejected in the application while claims 5 and 8-16 are objected to.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6, 17, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Fu et al. (U.S. Patent Publication No. 2011/0304628), in view of Qi et al. (Qi, C. R., Yi, L., Su, H., & Guibas, Leonidas J. (2017). PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. ArXiv.org).
Regarding claim 1, Fu discloses a method of building information modeling (BIM) reconstruction for a piping system using at least one processor, the method comprising: for each of a plurality of point cloud patches from point cloud data (interpreted as a computer implemented method that reconstructs a model of a piping system suitable for BIM, which is generating a structured 3D model of the piping system from data)[Fu: 0006 “The present invention relates generally to three-dimensional (3D) modeling, and in particular, to a method, system, apparatus, and article of manufacture for reconstructing a pipeline in a 3D computer-aided design”](teaches reconstructing a pipeline) obtaining the point cloud patch with respect to a seed point for the point cloud patch from the point cloud data obtained with respect to the piping system, the point cloud patch comprising the seed point and a first plurality of neighbor points with respect to the seed point (interpreted as a 3D point cloud of the piping system is available, a seed point is selected in that cloud. A local subset of points is then formed consisting of the seed point plus multiple neighboring points around it)[Fu: 0054 “a seed point is randomly selected (from the unchecked points) at step 204. A set of points {P} in the neighborhood of the seed point are found”](teaches the algorithm obtains point cloud data then selects a seed point and finds a set of points in the neighborhood of the seed points); and determining a pipe centerline point associated with the seed point based on the point cloud sub-patch and based on determining that the seed point is a pipe point (interpreted as once the seed point is known to be on a pipe and the neighbors in the sub patch belonging to the same pipe segment are selected, the method computes a pipe centerline point) [Fu: 0081 “The information of the detected cylinder shapes includes: axis vectors, centers, diameters, and lengths. The predecessor information is still needed for all of the pipes constituting a pipeline”][Fu: 0110 “Although the "pipes' shown in FIG. 13 are not real pipe components, the primary information of the pipe components (including the center line and the outer radius) have been extracted”](teaches primitive fitting process using the neighborhood of the seed point to fit a cylinder and notes that the primary information of pipe components include the center line and outer radius), the method further comprises: generating a centerline parameter set comprising [Fu: 0063 “axis vectors, centers, diameters”](teaches collecting matched pipe/cylinder shapes into a final set), for each of the plurality of point cloud patches comprising the seed point determined to be a pipe point (interpreted as the method repeats the seed point neighborhood process over the point cloud and only keeps valid pipe/cylinder)[Fu: 0062 “seed point is selected”], the pipe centerline point determined associated with the seed point for the point cloud patch [Fu: 0110 “components, the primary information of the pipe components (including the center line and the outer radius)”], a pipe radius determined associated with the seed point for the point cloud patch and a pipe flow direction determined associated with the seed point for the point cloud patch [Fu: 0063 “The information of the detected cylinder shapes includes: axis vectors, centers, diameters, lengths”], wherein the pipe radius determined and the pipe flow direction determined based on the point cloud patch are associated with the pipe centerline point determined based on the point cloud patch [Fu: 0063 “The information of the detected cylinder shapes includes: axis vectors, centers, diameters, lengths”] but fails to explicitly disclose embedding, for each of the first plurality of neighbor points, neighborhood features with respect to the neighbor point to the neighbor point to form a neighborhood feature embedded point cloud patch; generating a point cloud sub-patch comprising a second plurality of neighbor points from the first plurality of neighbor points of the neighborhood feature embedded point cloud patch, each of the second plurality of neighbor points being determined to belong to a same pipe segment as the seed point using a first machine learning model; determining whether the seed point is a pipe point based on the point cloud sub-patch using a second machine learning model.
However, Qi discloses embedding, for each of the first plurality of neighbor points, neighborhood features with respect to the neighbor point to the neighbor point to form a neighborhood feature embedded point cloud patch (interpreted as for each neighbor point in the patch, the method computes feature values describing its local neighborhood (like normal, flatness, local geometry), those neighborhood features are then embedded into that point)(Qi: Section 1 “We first partition the set of points into overlapping local regions by the distance metric of the underlying space. Similar to CNNs, we extract local features capturing fine geometric structures from small neighborhoods; such local features are further grouped into larger units”)(Qi: Section 3.2 “The set abstraction level is made of three key layers: Sampling layer, Grouping layer and PointNet layer. The Sampling layer selects a set of points from input points, which defines the centroids of local regions. Grouping layer then constructs local region sets by finding “neighboring” points around the centroids of local regions. Grouping layer then constructs local region sets by finding “neighboring” points around the centroids. PointNet layer uses a mini-PointNet to encode local region patterns into feature vectors.”)(teaches that PointNet++ set abstraction takes a point cloud, picks centroids (seed like points) and for each centroid, the grouping layer gathers neighboring points into a local region and the pointnet layer applies a neural network to that local region and outputs local feature vectors encoding its geometric structure); generating a point cloud sub-patch comprising a second plurality of neighbor points from the first plurality of neighbor points of the neighborhood feature embedded point cloud patch, each of the second plurality of neighbor points being determined to belong to a same pipe segment as the seed point using a first machine learning model (interpreted as starting from the feature embedded patch around the seed, a first ML model is used to decide for each neighbor whether it belongs to the same pipe segment as the seed. Those neighbors classified as same segment form a sub patch) (Qi: Section 1 “We first partition the set of points into overlapping local regions by the distance metric of the underlying space. Similar to CNNs, we extract local features capturing fine geometric structures from small neighborhoods; such local features are further grouped into larger units”)(Qi: Section 3.2 “Grouping layer then constructs local region sets by finding “neighboring” points around the centroids. PointNet layer uses a mini-PointNet to encode local region patterns into feature vectors.”)(Qi: Section C.1 “we evaluate our approach on part segmentation task assuming category label for each shape is already known. Taken shapes represented by point clouds as input, the task is to predict a part label for each point”)(teaches pointnet defines per centroid local regions (patches) by grouping neighboring points, and uses pointnet to produce learned features for those regions and then uses those features in a per point segmentation head); determining whether the seed point is a pipe point based on the point cloud sub-patch using a second machine learning model (within the context of this reference, ‘pipe point’ is treated as one semantic class)(Qi: Abstract “Experiments show that our network called PointNet++ is able to learn deep point set features efficiently and robustly. In particular, results significantly better than state-of-the-art have been obtained on challenging benchmarks of 3D point clouds.”)(Qi: Section 3.1 “Illustration of our hierarchical feature learning architecture and its application for set segmentation and classification using points in 2D Euclidean space as an example. Single scale point grouping is visualized here”)(Qi: Section 3.4 “However in set segmentation task such as semantic point labeling, we want to obtain point features for all the original points. One solution is to always sample all points as centroids in all set abstraction levels”)(teaches using the segmentation head and how it functions like the second ML model trained to determine whether the seed point is a pipe point).
Fu and Qi are considered to be analogous to the claimed invention because they are in the same field of 3D point cloud processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Fu to incorporate Qi’s teachings of utilizing local neighborhood feature embeddings. The motivation for such a combination would provide the benefit of improved robustness and automation.
Regarding claim 6, Fu discloses the method according to claim 1, wherein said obtaining the point cloud patch with respect to the seed point comprises extracting the plurality of neighbor points with respect to the seed point from the point cloud data (interpreted on how obtaining the point cloud patch step of claim 1 is completed) [Fu: 0054 “A determination is first made at step 202 whether all the points in the point cloud have been checked. If not, a seed point is randomly selected (from the unchecked points) at step 204. A set of points {P} in the neighborhood of the seedpoint are found at Step 206. Accordingly, in this approach, the extraction of a primitive shape is performed using random sampling in a set of points.”][Fu: 0113 “At step 1402, point cloud data is obtained. 0113. At step 1404, one or more primitive geometric shapes are detected in the point cloud data. To detect the shapes, a seed point is selected from the point cloud data. A set of points in the neighborhood of the seed point are found.”](teaches how point cloud patch is obtained), but fails to explicitly disclose using a ball query method or a k-nearest neighbors (k-NN) method.
However, Qi discloses using a ball query method or a k-nearest neighbors (k-NN) method (Qi: Page 3, “Ball query finds all points that are within a radius to the query point (an upper limit of K is set in implementation). An alternative range query is K nearest neighbor (kNN) search which finds a fixed number of neighboring points”).
Fu and Qi are considered to be analogous to the claimed invention because they are in the same field of point cloud processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Fu to incorporate Qi’s teachings of utilizing the KNN method. The motivation for such a combination would provide the benefit of efficient neighbor selection operators.
Claims 17 and 18 are system and non-transitory computer readable claims corresponding to claim 1 without any additional limitations. Thus, claims 17 and 18 are rejected for the same reasons as claim 1 above.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Fu et al. (U.S. Patent Publication No. 2011/0304628), in view of Qi et al. (Qi, C. R., Yi, L., Su, H., & Guibas, Leonidas J. (2017). PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. ArXiv.org), in further view of Rejeb Sfar et al. (U.S. Patent Publication No. 2022/0189070 A1).
Regarding claim 2, Fu and Qi disclose the method according to claim 1, but fail to explicitly disclose wherein the neighborhood features with respect to the neighbor point are latent space features learned with respect to a local neighborhood of the neighbor point.
However, Rejeb Sfar discloses wherein the neighborhood features with respect to the neighbor point are latent space features learned with respect to a local neighborhood of the neighbor point [Rejeb Sfar: 0006 “It is therefore provided a computer - implemented method of machine learning , for learning a neural network configured for encoding a super - point of a 3D point cloud into a latent vector . The method comprises providing a dataset of super - points . Each super - point is a set of points of a 3D point cloud . The set of points represents at least a part of an object”](Rejeb Sfar teaches a super point is a subset of points of the 3D point cloud which corresponds to local neighborhood patch in point cloud terms. The resulting latent vector is a latent space feature learned with respect to that local neighborhood).
Fu, Qi, and Rejeb Sfar are considered to be analogous to the claimed invention because they are in the same field of 3D point cloud processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Fu and Qi to incorporate Rejeb Sfar’s teachings of utilizing latent vectors. The motivation for such a combination would provide the benefit of more expressive and informative local descriptors, improving robustness of BIM reconstruction for piping systems.
Claims 3 and 4 are rejected under 35 U.S.C. 103 as being unpatentable over Fu et al. (U.S. Patent Publication No. 2011/0304628), in view of Qi et al. (Qi, C. R., Yi, L., Su, H., & Guibas, Leonidas J. (2017). PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. ArXiv.org), in further view of Le et al. (U.S. Patent No. 11,682,166).
Regarding claim 3, Fu and Qi disclose the method according to claim 1, Fu further discloses determining the pipe centerline point associated with the seed point based on the seed point, the pipe radius determined and the point normal determined [Fu: 0110 “Some pipelines with tiny radii are not modeled because of the deficiency of the points. Although the "pipes' shown in FIG. 13 are not real pipe components, the primary information of the pipe components (including the center line and the outer radius) have been extracted”] (teaches that for the detected pipes, the primary information including the center line and the outer radius has been extracted), but fails to explicitly disclose wherein said determining the pipe centerline point associated with the seed point comprises:
determining the pipe radius associated with the seed point based on the point cloud sub-patch using a third machine learning model; determining a point normal associated with the seed point based on the point cloud sub-patch using a fourth machine learning model.
However, Le discloses determining the pipe radius associated with the seed point based on the point cloud sub-patch using a third machine learning model (Le: Col. 1, Lines 29-34 “a set of 3D primitives are fit to a 3D point cloud using a cascaded primitive fitting network with a global primitive fitting network that evaluates the entire 3D point cloud and a local primitive fitting network that evaluates local patches formed by clusters of points from the 3D point cloud”) (Le: Col. 2, Lines 41-48 “the local primitive fitting network regresses a representation of smaller (local) primitives that fit the local structure of each of the local patches. The representations of the global and local primitives are merged into a representation of a combined, multi-scale set of fitted primitives, and representative primitive parameters such as dimensions, angles, and/or positions are computed for each fitted primitive in the combined set.”) (teaches local primitive fitting network which is a neural network that evaluates local patches formed by clusters of points from the point cloud and those local patches are the same type of object as the point cloud sub patch); determining a point normal associated with the seed point based on the point cloud sub-patch using a fourth machine learning model (Le: Col. 2, Lines 57-62 “A recent work proposed a supervised learning-based framework called Supervised Primitive Fitting Network (SPFN) that learns a configuration of 3D primitives that represents a 3D point cloud. Instead of directly regressing representative primitive parameters”) (Le: Col. 8, Lines 34-37 “In some embodiments, global primitive fitting network 245 is implemented with a Supervised Primitive Fitting Network (SPFN), which includes a PointNet++ architecture that predicts per-point features Wlob, Tglob, and Nglob.”) (Le: Col. 11, Lines 3-11 “In some embodiments, local primitive fitting network 260 is implemented with a Supervised Primitive Fitting Network (SPFN), which includes a PointNet++ architecture that predicts point-to-primitive membership Wloc per-point primitive type Tioe and unoriented point normals Nlec for each local patch. In an example implementation, WIoc {0, 13hxK toe, Nioce R"3, and TICE [0, 1]XL, where the local patch has n points”)(teaches the SPFN as predicting per point features including surface normal, the local primitive fitting network 260 is explicitly applied to local patches, and the point normal associated with the seed point is predicted by an ML model using the patch as input).
Fu, Qi, and Le are considered to be analogous to the claimed invention because they are in the same field of point cloud processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Fu and Qi to incorporate Le’s teachings of using neural networks on local point cloud neighborhoods. The motivation for such a combination would provide the benefit of improved automation and accuracy.
Regarding claim 4, Fu and Le disclose the method according to claim 3, but fail to explicitly disclose further comprising determining the pipe flow direction associated with the seed point based on the point cloud sub-patch using a fifth machine learning model.
However, Qi discloses further comprising determining the pipe flow direction associated with the seed point based on the point cloud sub-patch using a fifth machine learning model (Qi: Section 1 “The basic idea of PointNet is to learn a spatial encoding of each point”)(Qi discloses machine learning models and using a fifth model is an overly obvious limitation that does not make it novel over Qi).
Fu, Qi, and Le are considered to be analogous to the claimed invention because they are in the same field of point cloud processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Fu and Le to incorporate Qi’s teachings of using machine learning models. The motivation for such a combination would provide the benefit of improved automation and accuracy.
Allowable Subject Matter
Claims 5, 8-16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHMED TAHA whose telephone number is (571)272-6805. The examiner can normally be reached 8:30 am - 5 pm, Mon - Fri. Examiner interviews are available via telephone, in person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, XIAO WU can be reached at (571)272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786- 9199 (IN USA OR CANADA) or 571-272-1000.
/AHMED TAHA/Examiner, Art Unit 2613
/XIAO M WU/Supervisory Patent Examiner, Art Unit 2613
Prosecution Insights