DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Responsive to the communication dated 04/23/2026
Claims 1 and 4-15 are presented for examination
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/23/2026 has been entered.
Response to Arguments- 35 USC § 101
Applicant's arguments filed 03/24/2026 have been fully considered but they are not persuasive.
Applicant argues that the claims are not directed to a mental process because a person “cannot mentally compare incoming three- dimensional point cloud data with a two-dimensional building plan in real time as a scanner moves through a building.”
Examiner responds by firstly explaining that this interpretation is considerably narrower than what is required by the claim language; no mention of point cloud data nor a requirement that the comparison happen in real time is present in the claims. Under the broadest reasonable interpretation of the claims (BRI) “wherein the three- dimensional building data is acquired by means of a scanning unit” includes any kind of 3D building data; while this could include a point cloud, the breadth of this language also includes any other kind of 3D data generated utilizing physical scans. Similarly, that “comparing is performed at least partly during the acquiring such that the three-dimensional building data are compared with the two-dimensional, digitized building plan at least partly while the scanning unit acquires the three-dimensional building data” does not require that this comparison is done continuously nor in real time; the BRI of the claim language merely requires that at least once during the entire scanning process at least a portion of the comparison is completed. While continuous, real-time comparisons would read on this, by virtue of the breadth of the claim a person glancing at a digitized floorplan while walking a scanning trolley through a building and later mentally comparing what they can remember from the floorplan to a rendering of the 3D scan data reads equally as well on “the three-dimensional building data are compared with the two-dimensional, digitized building plan at least partly while the scanning unit acquires the three-dimensional building data” Note that the use of the word ‘partly’ in particular broadens the claim to where only a single step of the comparison process happening during the scan itself (e.g. a quick glance at the floor plan map) then later performing the vast majority of the comparison reads on the claim language.
Further, such a comparison between 3D data and 2D imagery is an inherent capability of the human mind; the ability to associate and compare a 2D layout with 3D geometry or environments is the basic function that has made maps an indispensable tool for navigation for thousands of years of human history. It is this ability that also allows people to visualize how an object might look from a certain perspective despite observing it from different perspective.
Additionally, even if this comparison happening in real time were required by the claims, the ability to perform this comparison at such a speed is merely the result of applying a general purpose computer to perform this mental process. (MPEP 2106.05(f)(2): Whether the claim invokes computers or other machinery merely as a tool to perform an existing process. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, "claiming the improved speed or efficiency inherent with applying the abstract idea on a computer" does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015).)
Applicant argues that the claims integrate the abstract idea into a practical solution and provide significantly more because they allow building information models to be generated “in a simple manner with a low degree of computational outlay” and include “essential features like object recognition, synchronization, and the like.”
Examiner responds by firstly explaining that the alleged improvements of the model being generated in a “simple manner” having “a low degree of computational outlay” are enabled by the use of object recognition and information extraction recited at an extremely high level of generality ([Par 20] “It is an advantage of embodiments of the present invention that a building information model can be generated in a simple manner and with a low degree of computational outlay. In particular, the essential and otherwise complex processing step of object recognition is automated by virtue of information being extracted based on the point clouds of the 3D data and the building plan during the acquisition of the three-dimensional data and the resulting generation of the building information model. This reduces the outlay of the manual post- processing of the building information model/BIM model.”) In other words, these improvements are enabled by the mental process of observing visual data and recognizing objects and features, something the human mind is particularly well suited for. Even with somewhat more abstract visual representations such as point clouds, the human mind is extremely adept at pattern recognition and would easily be able to pick out objects and recognizable features, something people have been doing with much less detailed data since the dawn of modern man (read: constellations, faces in mountains, shapes in clouds, etc.)
The argued ”synchronization” feature is interpreted as referring to the comparison between the 2D plans and 3D data; as noted previously, such a comparison between 3D data and 2D imagery is an inherent capability of the human mind; the ability to associate and compare a 2D layout with 3D geometry or environments is the basic function that has made maps an indispensable tool for navigation for thousands of years of human history. It is this ability that also allows people to visualize how an object might look from a certain perspective despite observing it in different perspective. Further, the claims lack sufficient specifics in relation to the actual embodiments of the computer elements that would suggest they are anything more than generic, general purpose computer elements.
Taken as a whole, the alleged improvements are solely furnished by the abstract idea itself and the remaining additional elements do not recite sufficient specificity to conclude that they provide a practical application nor significantly more than the judicial exception. Further, the use of a generically recited “scanning unit” to obtain scan data, even in view of the specification’s explanation that [Page 4 line 25] “A "scanning unit" can be understood to mean a scanner or laser scanner, for example.” is still at a very high level of generality that does not suggest any kind of non-generic, unconventional hardware configuration.
Further, this kind of 3D building scanning is an example of a well-understood, routine, conventional activity, as evidenced by the following:
Scan-To-Bim Procedure for an Old Industrial Plant ([Page 1023 Par 1 -Page 1024 Par 2])
From BIM to Scan Planning and Optimization for Construction Control ([Abstract, Page 1 Par 1 – Page 4 Par 6])
A Survey of Applications With Combined BIM and 3D Laser Scanning in the Life Cycle of Buildings ([Abstract, Page 5627 Col 2 Par 1 – Page 5628 Col 2 Par 3])
A Survey of Mobile Laser Scanning Applications and Key Techniques over Urban Areas ([Abstract, Page 1 Par 1 -Page 2 Par 1])
Response to Arguments- 35 USC § 103
Applicant's arguments filed 03/24/2026 have been fully considered but they are not persuasive.
Applicant argues that the combination of Zhang, Lim, and Son does not teach reading in two-dimensional data, generating the building information model, integrating object data into the building information model, and outputting the generated building information model in order to work on the building nor any 2D building plan and does not teach or suggest any comparison with a 2D building plan, as well as arguing that Zhang does not teach performing the comparisons while scanning nor scanning the building along a trajectory with a starting point determined based on the 2D building plan.
Examiner responds by firstly explaining that Lim and Son are no longer relied upon in the present rejection, and therefore arguments about the applicability of these references in particular are moot. Further, these features are taught by the combination of Zhang and new references Scan-To-Bim Procedure for an Old Industrial Plant (Hereinafter Guida), Floorplan-based Localization and Map Update Using LiDAR Sensor (Hereinafter Song), and From BIM to Scan Planning and Optimization for Construction Control (Hereinafter Frias). Particularly,
Zhang teaches reading in a ([Page 109 Col 1 Par 4] “This section describes the steps adopted in this study to capture, process, model, and integrate 3D laser scanner data about a construction process. The flowchart of the steps is presented in Fig. 1.” [Fig. 1] Shows the system flowchart, including a first step of creating a digital model of the building and defining objects within it [Page 109 Col 1 Par 5] “First, a 3D model of the structure is developed and the activities of the project are defined, using a 3D modeling software, such as MicroStation, Revit, or ArchiCad. For example, as was done in this study, the volume/surface area of the objects involved in each activity can be calculated by MicroStation V8i using the object's 3D coordinates.” [Page 112 Col 2 Par 1-2] “The dimensions of the completed structure are shown in Fig. 3. 3.1. 3D model and work schedule The project consisted of five activities, namely Column A, Column B, Column C, Column D, and Slab. A 3D model of this simple structure was produced by MicroStation V8i using the State plane coordinate system.”)
PNG
media_image1.png
804
537
media_image1.png
Greyscale
b) acquiring three-dimensional building data of at least a part of the building, wherein the three- dimensional building data is acquired by means of a scanning unit, ([Page 109 Col 2 Par 1] “3D point cloud data is captured using a laser scanner that comes typically with its own software”)
wherein the three-dimensional building data ([Page 109 Col 2 Par 1] “3D point cloud data is captured using a laser scanner that comes typically with its own software”)
c) comparing the acquired three-dimensional building data with the ([Page 109 Col 2 Par 6 – Page 110 Col 1 Par 1] “In the scan data, there are numerous points, including many points that are not related to the object under consideration. So, if the objective is to monitor the percentage of completion of an object, the points associated with the object need to be extracted from the original data. A java script was developed to count the number of points in the related portions of the point clouds. To account for construction and laser scanner tolerances, the true coordinates of the object in the 3D model are augmented by 0.7% increasing the volume of the object by 2% following [36] observation in their laser scanner experiments. The points (N1) in the point cloud that fall within the augmented object are counted. If no points are detected on any of the faces of the object, it is concluded that the percentage of completion is 0% (Fig. 2a). If points are detected within the boundaries of the object, a decision needs to be made about whether these points belong to the object being considered or to other objects or trash that are situated in the location where the 3D model indicates the object being considered should be. If the points represent the object considered, the points should appear on all the faces of the object. If the points represent trash or another object, whose shape is not the exact same shape as the object considered, the points do not appear on all faces of the object considered.”) ([Page 109 Col 2 Par 1] “3D point cloud data is captured using a laser scanner that comes typically with its own software”)
…
e) ([Page 109 Col 2 Par 1] “3D point cloud data is captured using a laser scanner that comes typically with its own software”) f) ([Fig.2] Details the method of determining if objects in the point cloud match those in the plan, as well as their completion amount) and the assigned building object data ([Page 109 Col 1 Par 5] For example, as was done in this study, the volume/surface area of the objects involved in each activity can be calculated by MicroStation V8i using the object's 3D coordinates.” [Page 112 Col 2 Par 1-2] “The dimensions of the completed structure are shown in Fig. 3. 3.1. 3D model and work schedule The project consisted of five activities, namely Column A, Column B, Column C, Column D, and Slab. A 3D model of this simple structure was produced by MicroStation V8i using the State plane coordinate system.”)
Guida teaches and the three- dimensional building data are acquired proceeding from the starting point, ([Page 1021 Par 2-3] “The quality of the data collected with the SLAM approach depends largely on how the acquisition is performed. For the case study, a few simple rules were observed. The area of interest is previously inspected to remove any obstacles and to identify critical sectors not identified in the design phase. The detection is performed walking slowly, with a speed of about 0.5 m/s, to have a good coverage and a high-resolution data. The artificial targets, used to materialise the photogrammetric GCPs, are scanned with the special accessory to store their coordinates, which are indispensable during the integration phase and to check for any drift. Since the surface is small, the acquisition is resolved with a single path, taking care to create intersections of the same (Fig. 2) Regarding drone acquisitions, two flight are prepared, both automatic and with double grid: a first one for the acquisition of nadir photogrammetric images and a second one, with the optical axis tilted about 45°, to survey the vertical walls and any shadow cones. The flight lines are designed using the DJI Ground- Station software package. The height is calculated in the DJI Ground- Station software using elevation data derived from Google Earth. Parallel flights lines are programmed to have an image overlap of 60% and sidelap of 60%, setting the proper camera parameters (dimensions of the sensor, focal length and flight height). In the nadir flights, 93 and 94 images are acquired for the first (from North to South) and second (from West to East) grid, respectively. For the oblique frames, 45 images are collected for the first grid and 50 for the second. The image acquisition is planned bearing in mind the project requirements - a Ground Sampling Distance (GSD) of about 1 cm - and, at the same time, with the aim of guaranteeing a high level of automation in the following phases.” [Page 1022 Par 1] “smoothing approaches estimate the full trajectory of the instrument from the full set of measurements. They address the so-called full SLAM problem and typically rely on least-square error minimization techniques. GeoSLAM algorithm can perform both an open-loop incremental solution for online SLAM and a closed-loop global registration for full SLAM (as in the case study). However, it is appropriate to introduce the general characteristics of the algorithm to understand its performance. For GeoSLAM formulation, the trajectory can describe the position of the sensor during data acquisition and can project raw laser measurements (2D laser profiles or segments) into a registered 3D point cloud when necessary. Data processing is an incremental and iterative procedure following a framework like the iterative closest point (ICP) algorithm.” [Page 4 Par 4] “Regarding photogrammetry, data treatment is performed by AgisoftMetashape, 1.6.5 version. Its workflow is based on two steps: “Align Photos” and “Build Dense Cloud”. At the first step an algorithm evaluates the camera internal parameters (focal length, position of the principal point, radial and tangential distortions), the camera positions for each photo and the “Sparse Cloud”. In the next phase, a greater pixel number is re-projected for each aligned camera, creating the “Dense Cloud”. The extracted point cloud has more than 48 million points, with average GCPs errors of about 2.8 cm.” [Fig. 2] Shows the scanning trajectory as well as a horizontal slice of the generated point cloud. The start/end point can be seen at the top left of the map; see annotated arrow pointing to the start of the scanning trajectory line)
PNG
media_image2.png
690
1266
media_image2.png
Greyscale
generating the building information model from the three-dimensional building data, integrating object data into the building information model, and outputting the generated building information model in order to work on the building. ([Page 1023 Par 1 - 2] “Once the application of the model has been established, i.e. to conserve the heritage through re-functionalisation and maintenance, the level of development (LOD) to be achieved is determined equal to LOD G, corresponding to the degree of detail of an upgraded object, and a level of accuracy (LOA) necessary LOA20, according to the USIBD classification, which allows geometric deviations between the model and the point cloud between 15 and 50 mm. The software used for the BIM modelling is Autodesk Revit which, thanks to the compatibility with Autodesk Recap PRO, allows the direct import of the point cloud in .rcp format… through the recognition of the intersections between the surfaces detected based on the arrangement of points in the cloud, it is possible to represent the lines that interpolate these points, resulting in the basis for the construction of other elements (Fig. 3)… It is therefore proposed to use these shape elements as the basis for modelling the walls as system families, using the “wall from surfaces” property, thus obtaining intelligent objects, to which the specific semantics of wall is associated, that faithfully reproduce the irregular course of the surfaces. Although HBIM is assuming an important role for the study and quantification of degradation phenomena by guaranteeing a detailed geometric representation of the artefacts [10], there is a lack of parameters related to the state of preservation. A solution to the problem has been obtained with the creation of parameters with which to populate the model with reliable and updatable information. The proposed experimentation foresees the representation of each degradation phenomenon through a mapping obtained by positioning the various adaptive points of the element, on the basis of the point cloud. In accordance with the indications of the UNI 11182 standard, the screens are imported into the software, associating graphic and textual information to them in the form of shared parameters. For a complete characterisation of the project parameters, global parameters are created. In this way, a hybrid graphical-informative database is configured, which comprehensively describes the alteration processes affecting the building and can be continuously updated. The development of proprietary libraries has made it possible to codify an approach that could become standardised and usable for future applications.” [Fig. 3] Shows the BIM model generated using the point-cloud with the raw point-cloud data overlaid)
PNG
media_image3.png
830
1298
media_image3.png
Greyscale
Guida is analogous art because it is within the field of scan-based BIM model development. It would have been obvious to one of ordinary skill in the art to combine Guida with Zhang before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to extend the system to not only be useful for new constructions, but to also assist in renovations, repair, and maintenance of existing structures. Guida notes how, while BIM systems are incredibly useful, their application to the renovation of existing structures is marred by limitations present in most BIM software, which are largely designed with only entirely new constructions in mind ([Page 1024 Par 1] “Almost all BIM software is designed for new constructions, so it may happen that some elements, not present in the software libraries, will have to be created by generating new parametric families [9] which took a lot of time due to the large number of elements and the amount of information, where the type and quality of the information input is proportional to the expected LOD. The most interesting challenge is the generation of custom parametric masonry families. The software does not allow the creation of walls that are not perfectly vertical. The geometries are modelled using Revit’s “conceptual masses” tool, families specific to the project.”) To this end, Guida introduces features that allow the easy integration of BIM systems with existing structures, allowing preservation and maintenance to be tracked alongside traditional BIM capabilities ([Page 102 Par 1-2] “It is therefore proposed to use these shape elements as the basis for modelling the walls as system families, using the “wall from surfaces” property, thus obtaining intelligent objects, to which the specific semantics of wall is associated, that faithfully reproduce the irregular course of the surfaces. Although HBIM is assuming an important role for the study and quantification of degradation phenomena by guaranteeing a detailed geometric representation of the artefacts [10], there is a lack of parameters related to the state of preservation. A solution to the problem has been obtained with the creation of parameters with which to populate the model with reliable and updatable information. The proposed experimentation foresees the representation of each degradation phenomenon through a mapping obtained by positioning the various adaptive points of the element, on the basis of the point cloud. In accordance with the indications of the UNI 11182 standard, the screens are imported into the software, associating graphic and textual information to them in the form of shared parameters. For a complete characterisation of the project parameters, global parameters are created. In this way, a hybrid graphical-informative database is configured, which comprehensively describes the alteration processes affecting the building and can be continuously updated.”) Overall, one of ordinary skill in the art would have recognized that combining Guida with Zhang would enable the construction tracking system of Zhang to be extended to be useful both in the construction of entirely new structures and the maintenance and renovation of existing structures, ultimately significantly increasing the applicability and utility of the system to a variety of scenarios in which it would not have been otherwise been particularly well suited.
Guida is analogous art because it is within the field of scan-based BIM model development. It would have been obvious to one of ordinary skill in the art to combine Guida with Zhang before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to extend the system to not only be useful for new constructions, but to also assist in renovations, repair, and maintenance of existing structures. Guida notes how, while BIM systems are incredibly useful, their application to the renovation of existing structures is marred by limitations present in most BIM software, which are largely designed with only entirely new constructions in mind ([Page 1024 Par 1] “Almost all BIM software is designed for new constructions, so it may happen that some elements, not present in the software libraries, will have to be created by generating new parametric families [9] which took a lot of time due to the large number of elements and the amount of information, where the type and quality of the information input is proportional to the expected LOD. The most interesting challenge is the generation of custom parametric masonry families. The software does not allow the creation of walls that are not perfectly vertical. The geometries are modelled using Revit’s “conceptual masses” tool, families specific to the project.”) To this end, Guida introduces features that allow the easy integration of BIM systems with existing structures, allowing preservation and maintenance to be tracked alongside traditional BIM capabilities ([Page 102 Par 1-2] “It is therefore proposed to use these shape elements as the basis for modelling the walls as system families, using the “wall from surfaces” property, thus obtaining intelligent objects, to which the specific semantics of wall is associated, that faithfully reproduce the irregular course of the surfaces. Although HBIM is assuming an important role for the study and quantification of degradation phenomena by guaranteeing a detailed geometric representation of the artefacts [10], there is a lack of parameters related to the state of preservation. A solution to the problem has been obtained with the creation of parameters with which to populate the model with reliable and updatable information. The proposed experimentation foresees the representation of each degradation phenomenon through a mapping obtained by positioning the various adaptive points of the element, on the basis of the point cloud. In accordance with the indications of the UNI 11182 standard, the screens are imported into the software, associating graphic and textual information to them in the form of shared parameters. For a complete characterisation of the project parameters, global parameters are created. In this way, a hybrid graphical-informative database is configured, which comprehensively describes the alteration processes affecting the building and can be continuously updated.”) Overall, one of ordinary skill in the art would have recognized that combining Guida with Zhang would enable the construction tracking system of Zhang to be extended to be useful both in the construction of entirely new structures and the maintenance and renovation of existing structures, ultimately significantly increasing the applicability and utility of the system to a variety of scenarios in which it would not have been otherwise been particularly well suited.
Song makes obvious reading in two-dimensional building plan data; ([Page 31 Col 1 Par 1] “To solve the problem mentioned above, in this paper, we proposed a matching algorithm that matches 3D line information extracted from 3D LiDAR with lines from the floorplan that are essential to exist for every building and MCL algorithm using results of matching. Line information with accurate metrics was extracted using a LiDAR sensor. Especially, the line was extracted only from the ceiling where floor information was removed for using only lines that were robust in the movement of furniture and other dynamic objects. Moreover, by using already existing floor plans, it is not necessary to make maps in advance. It also has the advantage of accurately matching the location with the actual map. As a result, based on the existing floorplan map shown in Fig. 1, MCL was possible by matching with the current line. Moreover, it was also possible to accumulate currently extracted lines on the map.” [Fig. 1] Shows imported floorplan data [Fig. 2] Shows the framework of the system, which clearly involves importing floorplan data)
PNG
media_image4.png
447
651
media_image4.png
Greyscale
PNG
media_image5.png
486
1392
media_image5.png
Greyscale
comparing the acquired three-dimensional building data with the two-dimensional, digitized building plan wherein comparing is performed at least partly during the acquiring such that the three-dimensional building data are compared with the two-dimensional, digitized building plan at least partly while the scanning unit acquires the three-dimensional building data; ([Abstract] “In this paper, we propose a novel localization and map update method for indoor using the LIDAR sensor and floorplan. Existing indoor localization algorithms need a previously generated 3D map and match those maps to the actual structure to get the precise location because there is no position reference like GPS. To solve this problem, the localization and map update method based on the floorplan, which generally exists, is proposed. For this, 3D LiDAR point clouds are accumulated, and ceiling parts are extracted, which is less sensitive to environmental changes such as furniture. Thereafter, the lines are extracted from the border of the ceiling parts, and the position is estimated through the Monte Carlo Localization algorithm using the comparison with floorplan lines.” [Page 31 Col 1 Par 1 – Col 2 Par 1] “To solve the problem mentioned above, in this paper, we proposed a matching algorithm that matches 3D line information extracted from 3D LiDAR with lines from the floorplan that are essential to exist for every building and MCL algorithm using results of matching. Line information with accurate metrics was extracted using a LiDAR sensor. Especially, the line was extracted only from the ceiling where floor information was removed for using only lines that were robust in the movement of furniture and other dynamic objects. Moreover, by using already existing floor plans, it is not necessary to make maps in advance. It also has the advantage of accurately matching the location with the actual map. As a result, based on the existing floorplan map shown in Fig. 1, MCL was possible by matching with the current line. Moreover, it was also possible to accumulate currently extracted lines on the map. … Therefore, it is efficient to use the floorplan for reference of an indoor environment and use it directly for localization.” [Page 32 Col 1 Par 1-2] “When the pointcloud accumulated until the current time t is called aPt, the origin of the pointcloud can be calculated from the previous timestep pose gTt−1 and the relative pose Tt−1 t , which is the result of NDT matching. Therefore, it can be said that gTt = Tt−1 t gTt−1. At this time, the pointcloud to be used in the next time step, t + 1, can be said to be ((Tt−1 t )−1 aPt) ∪ Pt and prevent the increase in computation cost due to an excessively dense pointcloud, a voxelized sparse point cloud is used. As shown in Fig. 4, the initial accumulated point cloud has only a single point cloud, and as time passes, the point clouds are gradually accumulated. The exact position of the next timestep can be estimated through this, and the accumulated pointcloud can be obtained. Since using NDT in this paper is to accumulate local point clouds, there is no need to accumulate all point clouds. Therefore, the accumulation of pointcloud targets only the pointcloud within time τ, the oldest element among the point clouds accumulated up to time t is the element included in Pt−τ” [Page 32 Col 2 Par 4 – Page 33 Col 1 Par 1] “In order to estimate the location by matching the line extracted through the above process with lines of the map, the map lines with the highest correspondence with the line currently extracted from the current location T of the robot should be searched. Since the previously extracted lines are expressed based on the robot’s local frame, they have to be expressed based on the robot’s current position gTt(gxs =g Tt·xs, gxe =g Tt·xe and gθ = θ(gTt)+θ).Then, to determine the similarity with the map lines, all map lines and the currently extracted lines are compared. Calculate the angle difference between the currently extracted line and the map lines, and calculate the center point distance between the map lines whose difference is less than θthres. Assuming that the currently extracted i’th line and the j’th line of the map are compared, if the distance is dij, the final distance to the line is determined through the dot product dij · ni with the normal vector ni of the currently extracted i’th line. … The particle with the highest similarity compared to the map line among the particles was used as the current position.” [Fig. 2] Shows the framework of the system, which clearly involves importing floorplan data and comparing it to the 3D lidar data. [Examiner’s note: As explained in the paragraphs cited above, the comparison process between the currently accumulated 3D lidar pointcloud data and the map/floorplan relies on the robot’s current position and the current time while generating the pointcloud; in other words, the comparison happens in real time during the acquisition of 3D building data.])
Song is analogous art because it is within the field of building scanning and model generation. It would have been obvious to one of ordinary skill in the art to combine Song with Zhang and Guida before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to better localize the scanning unit. As noted by Song, the ability to accurately determine the position of a scanning unit, an essential component for building scanning, is severely impacted by the unique conditions of indoor scanning, particularly in visually regular locations or locations without GPS or other signals, with Song noting the significant drawbacks of previous attempts to overcome these accuracy issues ([Page 30 Col 1 Par 1 – Col 2 Par 4] “Recently, simultaneous localization and mapping (SLAM) technology, which uses LiDAR technology to localize and mapping indoors and outdoors, has been used for various mobile robots. However, there is no GPS signal in indoor, mobile robots cannot have an exact location reference, which leads to inaccurate location recognition results [1]. … In order to perform indoor localization, information that can be obtained at indoors is used, usually radio signals. A typical approach is to measure the strength of existing wifi networks and use them to estimate locations [3] [4]. Although wifi is typically installed, it is not a common approach since multiple APs can exist, and wifi does not exist in all locations. There is also a disadvantage that wifi signals should be mapped on the entire building in advance. Similarly, there is a method that utilizes UWB sensors that use a wide radio bandwidth [2]. Those methods estimate the location of the mobile robot by obtaining distance information from multiple pre-installed UWB sensors. This allows for approximate indoor location estimation using less energy, but the UWB sensor must always be connected with an energy source, and sensor must be installed in advance, and the exact location of the installed UWB must be known. In order to compensate for the preinstallation, which is a disadvantage of the above method, another method [8] has been proposed to recognize and utilize changes in the earth’s magnetic field that occur in rebars of indoor structures. The magnetic method can correct the position by recognizing the change in magnetic values, but the disadvantage is that the change in the magnetic field can be caused by changes in the location of other iron objects such as tables, chairs, and computers. Localization using conventional RGB features has the disadvantage of not working well in plain indoor structures. To compensate for this disadvantage, a line visual SLAM algorithm [5] has also been proposed that utilizes line information that exists a lot in artificial structures. However, due to RGB image characteristics, even if calibrated through IMU, the exact metric is unknown. Also, additional works are inevitable to match localization and mapping results with the actual structure. Using Monte Carlo localization, a traditional indoor position recognition technology using LiDAR, [10] uses an existing map and 2D LiDAR to perform localization. Because it performs two-dimensional comparisons at robot height, it can produce inaccurate results and has a disadvantage that existing maps are needed”) To this end, Song presents a method for accurately determining the location of a sensor within a space using 3D lidar and existing building floorplans ([Page 31 Col 1 Par 1 -Col 2 Par 1] “To solve the problem mentioned above, in this paper, we proposed a matching algorithm that matches 3D line information extracted from 3D LiDAR with lines from the floorplan that are essential to exist for every building and MCL algorithm using results of matching. Line information with accurate metrics was extracted using a LiDAR sensor. Especially, the line was extracted only from the ceiling where floor information was removed for using only lines that were robust in the movement of furniture and other dynamic objects. Moreover, by using already existing floor plans, it is not necessary to make maps in advance. It also has the advantage of accurately matching the location with the actual map. As a result, based on the existing floorplan map shown in Fig. 1, MCL was possible by matching with the current line. Moreover, it was also possible to accumulate currently extracted lines on the map. … most buildings have floorplans for each floor, and in general, As shown in Fig. 3, it exists as a computer-aided design (CAD) file, so it is easy to manage and use. Therefore, it is efficient to use the floorplan for reference of an indoor environment and use it directly for localization.”) Overall, one of ordinary skill in the art would have recognized that combing Song with Zhang and Guida would result in much more accurate location determination for the unit used for 3D scanning, this more accurate location determination naturally leading to more accurate 3D scans which rely on this location as a basis for geometry calculation.
Frias makes obvious wherein a starting point of the scan trajectory is determined based on the two-dimensional, digitized building plan; ([Abstract] “The method starts by extracting floor plans from the BIM model according to the planned construction status, and including geometry and semantics of the building elements considered for construction control. The navigable space is defined from a binary map considering a security distance to building elements. After a grid-based and a triangulation-based distribution are implemented for generating scan position candidates, a visibility analysis is carried out to determine the optimal number and position of scans. The optimal route to visit all scan positions is addressed by using a probabilistic ant colony optimization algorithm.”)
Frias is analogous art because it is within the field of building scanning and model development. It would have been obvious to one of ordinary skill in the art to combine Frias with Zhang, Guida, and Song before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to make accurate scanning a much easier, less time-consuming task; this is noted by Song as a significant issue with previous systems ([Page 2 Par 4] “Generally, scanning is a time-consuming task, so minimizing the number of scanning operations is essential for efficient scanning planning. In addition, data acquisition must be successful in terms of integrity.”) To this end, Song presents a method for optimizing the scanning route, resulting in a scanning path that balances sensor visibility and total route length to minimizes scan time while maintaining high accuracy ([Page 2 Par 5] “In this work, a method to determinate the optimal scan positions and the optimal route followed by a stop &go system based on the use of BIM models, and considering data completeness as stopping criteria, is presented. … floor plans according to the planned construction status considered for construction control. The well-known DXF standard containing geometric information of the building elements is used to calculate candidates to scan positions, which are subsequently submitted to a visibility analysis using a ray-tracing algorithm. Next, scan positions are optimized based on visibility and data completeness as stopping criteria, and a probabilistic ant colony optimization algorithm is implemented to obtain a suboptimal route in a reasonable time.”) Overall, one of ordinary skill in the art would have recognized that combining Frias with Zhang, Guida, and Song would result in a scanning system that took significantly less time to complete the scanning process.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“a scanning unit”
“a comparison unit”
“a detection unit”
“an integration unit”
“an output unit”
in claims 1 and 14.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 10 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “substantially corresponding number” in claim 10 is a relative term which renders the claim indefinite. The term “substantially corresponding” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. What degree of similarity between two numbers is sufficient to conclude that they are “substantially corresponding?” Is a difference of .01 “substantially corresponding?” a difference of 1? 100? What is considered “substantially corresponding” is a subjective opinion that may differ between person to person, ultimately rendering this claim indefinite.
Further, Claim 10 recites the limitation "adjacent data points in the vertical direction have a substantially corresponding number.” There is insufficient antecedent basis for this limitation in the claim. Particularly, what this “number” refers to is unclear and not explained in the claim (read: number of what?) As such, the claim is rendered indefinite.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1 and 4-15 are rejected under 35 U.S.C. 101 because they are directed to an abstract idea without significantly more.
Claim 1 (Statutory Category – Process)
Step 2A – Prong 1: Judicial Exception Recited?
Yes, the claim recites a mental process, specifically:
MPEP 2106.04(a)(2)(Ill): “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, Judgments, and opinions.”
Further, the MPEP recites “The courts do not distinguish between mental processes that are performed entirely in the human mind and mental processes that require a human to use a physical aid (e.g., pen and paper or a slide rule) to perform the claim limitation.”
A method for generating a building information model for working on a building, comprising the following method steps: … c) comparing the acquired three-dimensional building data with the two-dimensional, digitized building plan, wherein at least a subregion of the building plan at a position of the scanning unit is evaluated, and outputting comparison data, wherein comparing is performed at least partly during the acquiring such that the three-dimensional building data are compared with the two-dimensional, digitized building plan at least partly while the scanning unit acquires the three-dimensional building data d) detecting at least one building object in the building plan based on the comparison data and outputting building object data assigned to the detected building object.
Comparing two representations of a building and recognizing objects within it is a mental process equivalent to observing the two representations and judging their similarities and differences. Detecting features within these representations is a matter of mentally observing and recognizing these features, such as walls and doorways. Further, such a comparison between 3D data and 2D imagery is an inherent capability of the human mind; the ability to associate and compare a 2D layout with 3D geometry or environments is the basic function that has made maps an indispensable tool for navigation for thousands of years of human history. It is this ability that also allows people to visualize how an object might look from a certain perspective despite observing it from a different perspective.
Doing this comparison with a “digitized” building plan amounts to no more than mere instructions to apply the judicial exception.
“Outputting” this data merely comprises of mentally noting the attributes of these features, such as the dimensions of a detected wall, and indicating them on paper with a pencil.
Should it be found that outputting this data is not a mental process, it is also an example of insignificant post-solution activity.
e) generating the building information model from the three-dimensional building data, f) integrating the detected building object and the assigned building object data into the building information model, and g) outputting the generated building information model in order to work on the building.
Generating and “outputting” such a model is a mental process equivalent to observing the data and drawing a model based on those observations with a pencil and paper. Integrating the detected objects into the model merely consists of including those objects in the representation of the model.
Doing this with a digital building information model in a true 3D environment amounts to no more than mere instructions to apply the judicial exception. It should be noted, however, that that the claims do not require that the building information model be 3D, merely that it is based on the 3D building data.
Should it be found that the generating and integrating steps are not a mental process, they are also an example of mere data gathering and mere instructions to apply.
Should it be found that outputting this data is not a mental process, it is also an example of insignificant post-solution activity.
Step 2A – Prong 2: Integrated into a Practical Solution?
Insignificant Extra-Solution Activity (MPEP 2106.05(g)) has found mere data gathering and
post solution activity to be insignificant extra-solution activity.
Data gathering:
a) reading in a two-dimensional, digitized building plan of the building, wherein the building plan comprises building objects and building object data assigned to the building objects,
Reading in this data merely gathers said data in a generic manner, and therefore amounts to no more than mere data gathering.
b) acquiring three-dimensional building data of at least a part of the building, wherein the three- dimensional building data is acquired by means of a scanning unit, wherein the three-dimensional building data are acquired by scanning the building along a trajectory, wherein a starting point of the trajectory is determined based on the two-dimensional, digitized building plan and the three- dimensional building data are acquired proceeding from the starting point,
Scanning in this data in a generic manner merely acts to gather this data, and therefore amounts to no more than mere data gathering. Specifying that it is gathered along a certain trajectory merely further clarifies the context in which the data is gathered.
e) generating the building information model from the three-dimensional building data, f) integrating the detected building object and the assigned building object data into the building information model
“Generating” this model and “integrating” the building object into the model, when recited at such a high level of generality, amounts to no more than gathering data representative of a building information model with certain elements contained therein.
Should it be found that this is not an example of mere data gathering, it is also an example of mere instructions to apply.
Post-Solution Activity:
outputting comparison data …g) outputting the generated building information model in order to work on the building.
Outputting the data and model in such a generic manner amounts to no more than presenting the results of the mental process, and therefore amounts to no more than insignificant post-solution activity.
Mere Instructions to Apply (MPEP 2106.05(f)) has found that merely applying a judicial exception such as an abstract idea, as by performing it on a computer, does not integrate the claim into a practical solution.
Mere Instructions to Apply:
e) generating the building information model from the three-dimensional building data, f) integrating the detected building object and the assigned building object data into the building information model
Applying a computer to generically generate a model and integrate elements into that model at a high level of generality is simply the act of instructing a computer to perform generic functions to perform that generation and integration, which is merely an instruction to apply a computer to the judicial exception. The claim only recites the idea of a solution or outcome, i.e. that the building information model is “generated” and the building object is “integrated” without reciting how this is actually accomplished. Further, the computer elements claimed are cited as merely generic tools to perform the operations.
Step 2B: Claim provides an Inventive Concept?
No, as discussed with respect to Step 2A, the additional limitations are Insignificant Extra-Solution Activity or Mere Instructions to Apply and do not impose any meaningful limits on practicing the abstract idea and therefore the claim does not provide an inventive concept in Step 2B.
Insignificant Extra-Solution Activity (MPEP 2106.05(g)) has found mere data gathering and
post solution activity to be insignificant extra-solution activity.
Data gathering:
a) reading in a two-dimensional, digitized building plan of the building, wherein the building plan comprises building objects and building object data assigned to the building objects,
Reading in this data merely gathers said data in a generic manner, and therefore amounts to no more than mere data gathering.
A claim element that amounts to merely gathering data is not indicative of integration into a
practical solution nor evidence that the claim provides an inventive concept or significantly more, as exemplified by ((MPEP 2106.05)(g)(Mere Data Gathering) i. Performing clinical tests on individuals to obtain input for an equation, In re Grams, 888 F.2d 835, 839-40; 12 USPQ2d 1824, 1827-28 (Fed. Cir. 1989); iv. Obtaining information about transactions using the Internet to verify credit card transactions, CyberSource v. Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011);
b) acquiring three-dimensional building data of at least a part of the building, wherein the three- dimensional building data is acquired by means of a scanning unit, wherein the three-dimensional building data are acquired by scanning the building along a trajectory, wherein a starting point of the trajectory is determined based on the two-dimensional, digitized building plan and the three- dimensional building data are acquired proceeding from the starting point,
Scanning in this data in a generic manner merely acts to gather this data, and therefore amounts to no more than mere data gathering. Specifying that it is gathered along a certain trajectory merely further clarifies the context in which the data is gathered.
A claim element that amounts to merely gathering data is not indicative of integration into a
practical solution nor evidence that the claim provides an inventive concept or significantly more, as exemplified by ((MPEP 2106.05)(g)(Mere Data Gathering) i. Performing clinical tests on individuals to obtain input for an equation, In re Grams, 888 F.2d 835, 839-40; 12 USPQ2d 1824, 1827-28 (Fed. Cir. 1989); iv. Obtaining information about transactions using the Internet to verify credit card transactions, CyberSource v. Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011);
e) generating the building information model from the three-dimensional building data, f) integrating the detected building object and the assigned building object data into the building information model
“Generating” this model and “integrating” the building object into the model, when recited at such a high level of generality, amounts to no more than gathering data representative of a building information model with certain elements contained therein.
A claim element that amounts to merely gathering data is not indicative of integration into a
practical solution nor evidence that the claim provides an inventive concept or significantly more, as exemplified by ((MPEP 2106.05)(g)(Mere Data Gathering) i. Performing clinical tests on individuals to obtain input for an equation, In re Grams, 888 F.2d 835, 839-40; 12 USPQ2d 1824, 1827-28 (Fed. Cir. 1989); iv. Obtaining information about transactions using the Internet to verify credit card transactions, CyberSource v. Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011);
Should it be found that this is not an example of mere data gathering, it is also an example of mere instructions to apply.
Post-Solution Activity:
outputting comparison data …g) outputting the generated building information model in order to work on the building.
Outputting the data and model in such a generic manner amounts to no more than presenting the results of the mental process, and therefore amounts to no more than insignificant post-solution activity.
This element merely acts on the results of the previous abstract steps. A claim element that merely acts on a series of previous abstract steps is not indicative of integration into a practical solution nor evidence that the claim provides an inventive concept, as exemplified by ((MPEP 2106.05)(g)(Insignificant application) i. Cutting hair after first determining the hair style, In re Brown, 645 Fed. App'x 1014, 1016-1017 (Fed. Cir. 2016) and ii. Printing or downloading generated menus, Ameranth, 842 F.3d at 1241-42, 120 USPQ2d at 1854-55.)
Mere Instructions to Apply (MPEP 2106.05(f)) has found that merely applying a judicial exception such as an abstract idea, as by performing it on a computer, does not integrate the claim into a practical solution.
Mere Instructions to Apply:
e) generating the building information model from the three-dimensional building data, f) integrating the detected building object and the assigned building object data into the building information model
Applying a computer to generically generate a model and integrate elements into that model at a high level of generality is simply the act of instructing a computer to perform generic functions to perform that generation and integration, which is merely an instruction to apply a computer to the judicial exception. The claim only recites the idea of a solution or outcome, i.e. that the building information model is “generated” and the building object is “integrated” without reciting how this is actually accomplished. Further, the computer elements claimed are cited as merely generic tools to perform the operations.
The courts have found that such mere instructions to apply are not indicative of integration into a practical application nor recitation of significantly more than the judicial exception (MPEP 2106.05(f) “Another consideration when determining whether a claim integrates a judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. As explained by the Supreme Court, in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do "‘more than simply stat[e] the [judicial exception] while adding the words ‘apply it’". Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983”)
Moreover, Mere Instructions To Apply An Exception (MPEP 2106.05(f)) has found that simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. In light of this, the additional generic computer component elements of “a building information model; a two-dimensional, digitized building plan; three-dimensional building data; a scanning unit” are not sufficient to integrate a judicial exception into a practical application nor provide evidence of an inventive concept.
Well-Understood, Routine, Conventional Activity (WURC) has found that claim elements that are understood to be Well-Understood, Routine, Conventional Activity are not indicative of Integration into a Practical Solution nor evidence of an Inventive Concept (MPEP 2106.05(d))
WURC:
acquiring three-dimensional building data of at least a part of the building, wherein the three- dimensional building data is acquired by means of a scanning unit
Using a scanning unit to scan 3D building data is an example of a well-understood, routine, and conventional activity, as evidenced by:
Scan-To-Bim Procedure for an Old Industrial Plant ([Page 1023 Par 1 -Page 1024 Par 2])
From BIM to Scan Planning and Optimization for Construction Control ([Abstract, Page 1 Par 1 – Page 4 Par 6])
A Survey of Applications With Combined BIM and 3D Laser Scanning in the Life Cycle of Buildings ([Abstract, Page 5627 Col 2 Par 1 – Page 5628 Col 2 Par 3])
A Survey of Mobile Laser Scanning Applications and Key Techniques over Urban Areas ([Abstract, Page 1 Par 1 -Page 2 Par 1])
The additional elements have been considered both individually and as an ordered combination in the consideration of whether they constitute significantly more, and have been determined not to constitute such.
The claim is ineligible.
Claim 14 The elements of claim 14 are substantially the same as those of claim 1. Therefore, the elements of claim 14 are rejected due to the same reasons as outlined above for claim 1.
Moreover, Mere Instructions To Apply An Exception (MPEP 2106.05(f)) has found that simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. In light of this, the additional generic computer component elements of “A system for generating a building information model for working on a building, comprising: an interface that is set up in such a way as to…; a two-dimensional building plan; a scanning unit; three-dimensional building data; a comparison unit; a detection unit; a model generator; an integration unit; an output unit” are not sufficient to integrate a judicial exception into a practical application nor provide evidence of an inventive concept.
Claim 4 recites “wherein method steps b) to d) are repeated after predefined time steps.”
This merely clarifies when certain operations are performed and is therefore merely an extension of the mental process, mere data gathering, post-solution activity, and mere instructions to apply.
Claim 5 recites “wherein method steps b) to d) are repeated after detection of a building object.”
This merely clarifies when certain operations are performed and is therefore merely an extension of the mental process, mere data gathering, post-solution activity, and mere instructions to apply.
Claim 6 recites “wherein a deviation of the three-dimensional building data from the building plan is detected based on the comparison data.”
Detecting a deviation between these two representation is a mental process equivalent to observing both representations and judging their differences.
Claim 7 recites “wherein, if a deviation is identified, a building object and corresponding building object data are generated depending on the three-dimensional building data and integrated into the building information model.”
Generating these objects and placing them in the model based on the scan data is a mental process equivalent to observing the scan data, recognizing the objects represented in that data, and drawing representations of them in the model.
Claim 8 recites “wherein the method step of comparing the three- dimensional building data with the building plan comprises the following substeps: determining a horizontal structure based on the three-dimensional building data, generating a first line based on the horizontal structure, comparing the first line with lines of the building plan, wherein the first line is assigned to a second line of the building plan, and correcting the spatial coordinates of the three-dimensional building data, assigned to the first line, depending on the position of the second line” Determining a “horizontal structure” based on the scan data is a mental process equivalent to recognizing structures in the scan that are horizon. Generating lines based on the scan data is equivalent to drawing line representations of these features, for example a line to represent the bottom of a wall. If a similar line exists in the building plan and it is known that the lines should be aligned between the representations, the scan data representation can be altered to change the relative position of these features.
Claim 9 recites “wherein the at least one building object is detected based on the second line in the building plan.”
Detecting building objects based on these lines is a mental process equivalent to observing said lines in the plan and judging what they represent. For example, if a line exists at the extremities of the interior, it could be judged that this line represents a wall.
Claim 10 recites “wherein the horizontal structure is detected based on the three-dimensional building data by virtue of data points lying one above the other in the vertical direction being counted and being connected horizontally to adjacent data points along the respectively topmost data point provided that the adjacent data points in the vertical direction have a substantially corresponding number.”
This limitation describes interpreting a point cloud, particularly observing collections of points on a plane and making judgments about it. This is a mental process. If a person observed a vertical, rectangular collection of data points in a point cloud as described, that person could reasonably judge that those points represent a wall.
Claim 11 recites “wherein the method step of comparing the three- dimensional building data with the building plan is carried out by means of a machine learning model trained for this purpose.”
Using a machine learning model to perform this mental process amounts to no more than mere instructions to apply.
Applying a computer to perform generic machine learning operations at a high level of generality is simply the act of instructing a computer to perform generic functions to perform those operations, which is merely an instruction to apply a computer to the judicial exception. The claim only recites the idea of a solution or outcome, i.e. that the data and plan are “compared” without reciting how this is actually accomplished. Further, the computer elements claimed are cited as merely generic tools to perform the operations.
The courts have found that such mere instructions to apply are not indicative of integration into a practical application nor recitation of significantly more than the judicial exception (MPEP 2106.05(f) “Another consideration when determining whether a claim integrates a judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. As explained by the Supreme Court, in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do "‘more than simply stat[e] the [judicial exception] while adding the words ‘apply it’". Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983”)
Claim 12 recites “wherein the building plan is a floor plan or an electrical installation plan.”
This merely clarifies the form of the building plan and is therefore merely an extension of the mental process and mere data gathering.
Claim 13 recites “wherein the building information model is a BIM model.”
This merely clarifies the form of the building information model and is therefore merely an extension of the mental process and mere instructions to apply.
Claim 15 recites “A non-transitory computer readable hardware storage device having computer readable program code stored therein, said program code executable by a processor of a computer system to implement the method as claimed in claim 1.”
This claim merely defines a general purpose computer to carry out the abstract process of claim 1, and therefore amounts to no more than mere instructions to apply.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-7, and 12-15 are rejected under 35 U.S.C. 103 as being unpatentable over Automated progress control using laser scanning technology (Hereinafter Zhang) in view of Scan-To-Bim Procedure for an Old Industrial Plant (Hereinafter Guida) in further view of Floorplan-based Localization and Map Update Using LiDAR Sensor (Hereinafter Song) as well as From BIM to Scan Planning and Optimization for Construction Control (Hereinafter Frias)
Claim 1. Zhang teaches A method for generating a building information model for working on a building, comprising the following method steps: reading in a ([Page 109 Col 1 Par 4] “This section describes the steps adopted in this study to capture, process, model, and integrate 3D laser scanner data about a construction process. The flowchart of the steps is presented in Fig. 1.” [Fig. 1] Shows the system flowchart, including a first step of creating a digital model of the building and defining objects within it [Page 109 Col 1 Par 5] “First, a 3D model of the structure is developed and the activities of the project are defined, using a 3D modeling software, such as MicroStation, Revit, or ArchiCad. For example, as was done in this study, the volume/surface area of the objects involved in each activity can be calculated by MicroStation V8i using the object's 3D coordinates.” [Page 112 Col 2 Par 1-2] “The dimensions of the completed structure are shown in Fig. 3. 3.1. 3D model and work schedule The project consisted of five activities, namely Column A, Column B, Column C, Column D, and Slab. A 3D model of this simple structure was produced by MicroStation V8i using the State plane coordinate system.”)
PNG
media_image1.png
804
537
media_image1.png
Greyscale
PNG
media_image6.png
707
1199
media_image6.png
Greyscale
b) acquiring three-dimensional building data of at least a part of the building, wherein the three- dimensional building data is acquired by means of a scanning unit, ([Page 109 Col 2 Par 1] “3D point cloud data is captured using a laser scanner that comes typically with its own software”) wherein the three-dimensional building data ([Page 109 Col 2 Par 1] “3D point cloud data is captured using a laser scanner that comes typically with its own software”) ([Page 109 Col 2 Par 6 – Page 110 Col 1 Par 1] “In the scan data, there are numerous points, including many points that are not related to the object under consideration. So, if the objective is to monitor the percentage of completion of an object, the points associated with the object need to be extracted from the original data. A java script was developed to count the number of points in the related portions of the point clouds. To account for construction and laser scanner tolerances, the true coordinates of the object in the 3D model are augmented by 0.7% increasing the volume of the object by 2% following [36] observation in their laser scanner experiments. The points (N1) in the point cloud that fall within the augmented object are counted. If no points are detected on any of the faces of the object, it is concluded that the percentage of completion is 0% (Fig. 2a). If points are detected within the boundaries of the object, a decision needs to be made about whether these points belong to the object being considered or to other objects or trash that are situated in the location where the 3D model indicates the object being considered should be. If the points represent the object considered, the points should appear on all the faces of the object. If the points represent trash or another object, whose shape is not the exact same shape as the object considered, the points do not appear on all faces of the object considered.”) ([Page 109 Col 2 Par 1] “3D point cloud data is captured using a laser scanner that comes typically with its own software”) ([Page 109 Col 2 Par 6 – Page 110 Col 2 Par 2] “In the scan data, there are numerous points, including many points that are not related to the object under consideration. So, if the objective is to monitor the percentage of completion of an object, the points associated with the object need to be extracted from the original data. A java script was developed to count the number of points in the related portions of the point clouds. To account for construction and laser scanner tolerances, the true coordinates of the object in the 3D model are augmented by 0.7% increasing the volume of the object by 2% following [36] observation in their laser scanner experiments. The points (N1) in the point cloud that fall within the augmented object are counted. If no points are detected on any of the faces of the object, it is concluded that the percentage of completion is 0% (Fig. 2a). If points are detected within the boundaries of the object, a decision needs to be made about whether these points belong to the object being considered or to other objects or trash that are situated in the location where the 3D model indicates the object being considered should be. If the points represent the object considered, the points should appear on all the faces of the object. If the points represent trash or another object, whose shape is not the exact same shape as the object considered, the points do not appear on all faces of the object considered… After shrinking the 3D model of an object by 1.5%, if no points can be counted on any face of the object, this signifies that some parts of the object are in place since the scanner does not record points inside the object. Therefore, one can use the N2/N1 ratio to measure progress. If the ratio N2/N1 ≤ 0.1%, the object is considered to be completed 100% (see Fig. 2b). The N2/N1 ratio is set as 0.1% rather than a flat 0% in order to account for the few points that may appear on any face of the object as a result of noisy data. If N2/N1 ≥ α, a threshold value, it is concluded that the points have nothing to do with the object considered, i.e., a percentage of completion of 0% (Fig. 2c). ” [Fig.2] Details the method of determining if objects in the point cloud match those in the plan, as well as their completion amount)
PNG
media_image7.png
725
587
media_image7.png
Greyscale
e) ([Page 109 Col 2 Par 1] “3D point cloud data is captured using a laser scanner that comes typically with its own software”) f) ([Fig.2] Details the method of determining if objects in the point cloud match those in the plan, as well as their completion amount) and the assigned building object data ([Page 109 Col 1 Par 5] For example, as was done in this study, the volume/surface area of the objects involved in each activity can be calculated by MicroStation V8i using the object's 3D coordinates.” [Page 112 Col 2 Par 1-2] “The dimensions of the completed structure are shown in Fig. 3. 3.1. 3D model and work schedule The project consisted of five activities, namely Column A, Column B, Column C, Column D, and Slab. A 3D model of this simple structure was produced by MicroStation V8i using the State plane coordinate system.”)
Zhang does not explicitly teach A method for generating a building information model for working on a building, comprising the following method steps: reading in two-dimensional building plan data; wherein the three-dimensional building data are acquired by scanning the building along a trajectory, wherein a starting point of the trajectory is determined based on the two-dimensional, digitized building plan and the three- dimensional building data are acquired proceeding from the starting point, comparing the acquired three-dimensional building data with the two-dimensional, digitized building plan wherein comparing is performed at least partly during the acquiring such that the three-dimensional building data are compared with the two-dimensional, digitized building plan at least partly while the scanning unit acquires the three-dimensional building data; generating the building information model from scan data, integrating object data into the building information model, and outputting the generated building information model in order to work on the building.
Guida teaches A method for generating a building information model for working on a building, comprising the following method steps: ([Abstract] “The work presents an efficient solution, based on integrating different data sources, for the digitization and modelling of an old structure with the HBIM methodology. … A procedural pipeline is formalized for data acquisition, processing, and generating a complete model for a disused industrial plant, formerly used for tobacco processing and located in the city of Battipaglia, Italy.” [Page 1020 Par 1] “In this paper we propose some results of a SCAN-to-BIM process for getting a digital model of the industrial plant by means a synergic approach of SLAM and aero-photogrammetry methods.” [Page 1023 Par 1] “Once the application of the model has been established, i.e. to conserve the heritage through re-functionalisation and maintenance, the level of development (LOD) to be achieved is determined equal to LOD G…”) and the three- dimensional building data are acquired proceeding from the starting point, ([Page 1021 Par 2-3] “The quality of the data collected with the SLAM approach depends largely on how the acquisition is performed. For the case study, a few simple rules were observed. The area of interest is previously inspected to remove any obstacles and to identify critical sectors not identified in the design phase. The detection is performed walking slowly, with a speed of about 0.5 m/s, to have a good coverage and a high-resolution data. The artificial targets, used to materialise the photogrammetric GCPs, are scanned with the special accessory to store their coordinates, which are indispensable during the integration phase and to check for any drift. Since the surface is small, the acquisition is resolved with a single path, taking care to create intersections of the same (Fig. 2) Regarding drone acquisitions, two flight are prepared, both automatic and with double grid: a first one for the acquisition of nadir photogrammetric images and a second one, with the optical axis tilted about 45°, to survey the vertical walls and any shadow cones. The flight lines are designed using the DJI Ground- Station software package. The height is calculated in the DJI Ground- Station software using elevation data derived from Google Earth. Parallel flights lines are programmed to have an image overlap of 60% and sidelap of 60%, setting the proper camera parameters (dimensions of the sensor, focal length and flight height). In the nadir flights, 93 and 94 images are acquired for the first (from North to South) and second (from West to East) grid, respectively. For the oblique frames, 45 images are collected for the first grid and 50 for the second. The image acquisition is planned bearing in mind the project requirements - a Ground Sampling Distance (GSD) of about 1 cm - and, at the same time, with the aim of guaranteeing a high level of automation in the following phases.” [Page 1022 Par 1] “smoothing approaches estimate the full trajectory of the instrument from the full set of measurements. They address the so-called full SLAM problem and typically rely on least-square error minimization techniques. GeoSLAM algorithm can perform both an open-loop incremental solution for online SLAM and a closed-loop global registration for full SLAM (as in the case study). However, it is appropriate to introduce the general characteristics of the algorithm to understand its performance. For GeoSLAM formulation, the trajectory can describe the position of the sensor during data acquisition and can project raw laser measurements (2D laser profiles or segments) into a registered 3D point cloud when necessary. Data processing is an incremental and iterative procedure following a framework like the iterative closest point (ICP) algorithm.” [Page 4 Par 4] “Regarding photogrammetry, data treatment is performed by AgisoftMetashape, 1.6.5 version. Its workflow is based on two steps: “Align Photos” and “Build Dense Cloud”. At the first step an algorithm evaluates the camera internal parameters (focal length, position of the principal point, radial and tangential distortions), the camera positions for each photo and the “Sparse Cloud”. In the next phase, a greater pixel number is re-projected for each aligned camera, creating the “Dense Cloud”. The extracted point cloud has more than 48 million points, with average GCPs errors of about 2.8 cm.” [Fig. 2] Shows the scanning trajectory as well as a horizontal slice of the generated point cloud. The start/end point can be seen at the top left of the map; see annotated arrow pointing to the start of the scanning trajectory line)
PNG
media_image2.png
690
1266
media_image2.png
Greyscale
([Page 1023 Par 1 - 2] “Once the application of the model has been established, i.e. to conserve the heritage through re-functionalisation and maintenance, the level of development (LOD) to be achieved is determined equal to LOD G, corresponding to the degree of detail of an upgraded object, and a level of accuracy (LOA) necessary LOA20, according to the USIBD classification, which allows geometric deviations between the model and the point cloud between 15 and 50 mm. The software used for the BIM modelling is Autodesk Revit which, thanks to the compatibility with Autodesk Recap PRO, allows the direct import of the point cloud in .rcp format… through the recognition of the intersections between the surfaces detected based on the arrangement of points in the cloud, it is possible to represent the lines that interpolate these points, resulting in the basis for the construction of other elements (Fig. 3)… It is therefore proposed to use these shape elements as the basis for modelling the walls as system families, using the “wall from surfaces” property, thus obtaining intelligent objects, to which the specific semantics of wall is associated, that faithfully reproduce the irregular course of the surfaces. Although HBIM is assuming an important role for the study and quantification of degradation phenomena by guaranteeing a detailed geometric representation of the artefacts [10], there is a lack of parameters related to the state of preservation. A solution to the problem has been obtained with the creation of parameters with which to populate the model with reliable and updatable information. The proposed experimentation foresees the representation of each degradation phenomenon through a mapping obtained by positioning the various adaptive points of the element, on the basis of the point cloud. In accordance with the indications of the UNI 11182 standard, the screens are imported into the software, associating graphic and textual information to them in the form of shared parameters. For a complete characterisation of the project parameters, global parameters are created. In this way, a hybrid graphical-informative database is configured, which comprehensively describes the alteration processes affecting the building and can be continuously updated. The development of proprietary libraries has made it possible to codify an approach that could become standardised and usable for future applications.” [Fig. 3] Shows the BIM model generated using the point-cloud with the raw point-cloud data overlaid)
PNG
media_image3.png
830
1298
media_image3.png
Greyscale
Guida is analogous art because it is within the field of scan-based BIM model development. It would have been obvious to one of ordinary skill in the art to combine Guida with Zhang before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to extend the system to not only be useful for new constructions, but to also assist in renovations, repair, and maintenance of existing structures. Guida notes how, while BIM systems are incredibly useful, their application to the renovation of existing structures is marred by limitations present in most BIM software, which are largely designed with only entirely new constructions in mind ([Page 1024 Par 1] “Almost all BIM software is designed for new constructions, so it may happen that some elements, not present in the software libraries, will have to be created by generating new parametric families [9] which took a lot of time due to the large number of elements and the amount of information, where the type and quality of the information input is proportional to the expected LOD. The most interesting challenge is the generation of custom parametric masonry families. The software does not allow the creation of walls that are not perfectly vertical. The geometries are modelled using Revit’s “conceptual masses” tool, families specific to the project.”) To this end, Guida introduces features that allow the easy integration of BIM systems with existing structures, allowing preservation and maintenance to be tracked alongside traditional BIM capabilities ([Page 102 Par 1-2] “It is therefore proposed to use these shape elements as the basis for modelling the walls as system families, using the “wall from surfaces” property, thus obtaining intelligent objects, to which the specific semantics of wall is associated, that faithfully reproduce the irregular course of the surfaces. Although HBIM is assuming an important role for the study and quantification of degradation phenomena by guaranteeing a detailed geometric representation of the artefacts [10], there is a lack of parameters related to the state of preservation. A solution to the problem has been obtained with the creation of parameters with which to populate the model with reliable and updatable information. The proposed experimentation foresees the representation of each degradation phenomenon through a mapping obtained by positioning the various adaptive points of the element, on the basis of the point cloud. In accordance with the indications of the UNI 11182 standard, the screens are imported into the software, associating graphic and textual information to them in the form of shared parameters. For a complete characterisation of the project parameters, global parameters are created. In this way, a hybrid graphical-informative database is configured, which comprehensively describes the alteration processes affecting the building and can be continuously updated.”) Overall, one of ordinary skill in the art would have recognized that combining Guida with Zhang would enable the construction tracking system of Zhang to be extended to be useful both in the construction of entirely new structures and the maintenance and renovation of existing structures, ultimately significantly increasing the applicability and utility of the system to a variety of scenarios in which it would not have been otherwise been particularly well suited.
The combination of Zhang and Guida does not explicitly teach reading in two-dimensional building plan data; wherein a starting point of the scan trajectory is determined based on the two-dimensional, digitized building plan; comparing the acquired three-dimensional building data with the two-dimensional, digitized building plan wherein comparing is performed at least partly during the acquiring such that the three-dimensional building data are compared with the two-dimensional, digitized building plan at least partly while the scanning unit acquires the three-dimensional building data;
Song makes obvious reading in two-dimensional building plan data; ([Page 31 Col 1 Par 1] “To solve the problem mentioned above, in this paper, we proposed a matching algorithm that matches 3D line information extracted from 3D LiDAR with lines from the floorplan that are essential to exist for every building and MCL algorithm using results of matching. Line information with accurate metrics was extracted using a LiDAR sensor. Especially, the line was extracted only from the ceiling where floor information was removed for using only lines that were robust in the movement of furniture and other dynamic objects. Moreover, by using already existing floor plans, it is not necessary to make maps in advance. It also has the advantage of accurately matching the location with the actual map. As a result, based on the existing floorplan map shown in Fig. 1, MCL was possible by matching with the current line. Moreover, it was also possible to accumulate currently extracted lines on the map.” [Fig. 1] Shows imported floorplan data [Fig. 2] Shows the framework of the system, which clearly involves importing floorplan data)
PNG
media_image4.png
447
651
media_image4.png
Greyscale
PNG
media_image5.png
486
1392
media_image5.png
Greyscale
([Abstract] “In this paper, we propose a novel localization and map update method for indoor using the LIDAR sensor and floorplan. Existing indoor localization algorithms need a previously generated 3D map and match those maps to the actual structure to get the precise location because there is no position reference like GPS. To solve this problem, the localization and map update method based on the floorplan, which generally exists, is proposed. For this, 3D LiDAR point clouds are accumulated, and ceiling parts are extracted, which is less sensitive to environmental changes such as furniture. Thereafter, the lines are extracted from the border of the ceiling parts, and the position is estimated through the Monte Carlo Localization algorithm using the comparison with floorplan lines.” [Page 31 Col 1 Par 1 – Col 2 Par 1] “To solve the problem mentioned above, in this paper, we proposed a matching algorithm that matches 3D line information extracted from 3D LiDAR with lines from the floorplan that are essential to exist for every building and MCL algorithm using results of matching. Line information with accurate metrics was extracted using a LiDAR sensor. Especially, the line was extracted only from the ceiling where floor information was removed for using only lines that were robust in the movement of furniture and other dynamic objects. Moreover, by using already existing floor plans, it is not necessary to make maps in advance. It also has the advantage of accurately matching the location with the actual map. As a result, based on the existing floorplan map shown in Fig. 1, MCL was possible by matching with the current line. Moreover, it was also possible to accumulate currently extracted lines on the map. … Therefore, it is efficient to use the floorplan for reference of an indoor environment and use it directly for localization.” [Page 32 Col 1 Par 1-2] “When the pointcloud accumulated until the current time t is called aPt, the origin of the pointcloud can be calculated from the previous timestep pose gTt−1 and the relative pose Tt−1 t , which is the result of NDT matching. Therefore, it can be said that gTt = Tt−1 t gTt−1. At this time, the pointcloud to be used in the next time step, t + 1, can be said to be ((Tt−1 t )−1 aPt) ∪ Pt and prevent the increase in computation cost due to an excessively dense pointcloud, a voxelized sparse point cloud is used. As shown in Fig. 4, the initial accumulated point cloud has only a single point cloud, and as time passes, the point clouds are gradually accumulated. The exact position of the next timestep can be estimated through this, and the accumulated pointcloud can be obtained. Since using NDT in this paper is to accumulate local point clouds, there is no need to accumulate all point clouds. Therefore, the accumulation of pointcloud targets only the pointcloud within time τ, the oldest element among the point clouds accumulated up to time t is the element included in Pt−τ” [Page 32 Col 2 Par 4 – Page 33 Col 1 Par 1] “In order to estimate the location by matching the line extracted through the above process with lines of the map, the map lines with the highest correspondence with the line currently extracted from the current location T of the robot should be searched. Since the previously extracted lines are expressed based on the robot’s local frame, they have to be expressed based on the robot’s current position gTt(gxs =g Tt·xs, gxe =g Tt·xe and gθ = θ(gTt)+θ).Then, to determine the similarity with the map lines, all map lines and the currently extracted lines are compared. Calculate the angle difference between the currently extracted line and the map lines, and calculate the center point distance between the map lines whose difference is less than θthres. Assuming that the currently extracted i’th line and the j’th line of the map are compared, if the distance is dij, the final distance to the line is determined through the dot product dij · ni with the normal vector ni of the currently extracted i’th line. … The particle with the highest similarity compared to the map line among the particles was used as the current position.” [Fig. 2] Shows the framework of the system, which clearly involves importing floorplan data and comparing it to the 3D lidar data. [Examiner’s note: As explained in the paragraphs cited above, the comparison process between the currently accumulated 3D lidar pointcloud data and the map/floorplan relies on the robot’s current position and the current time while generating the pointcloud; in other words, the comparison happens in real time during the acquisition of 3D building data.])
Song is analogous art because it is within the field of building scanning and model generation. It would have been obvious to one of ordinary skill in the art to combine Song with Zhang and Guida before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to better localize the scanning unit. As noted by Song, the ability to accurately determine the position of a scanning unit, an essential component for building scanning, is severely impacted by the unique conditions of indoor scanning, particularly in visually regular locations or locations without GPS or other signals, with Song noting the significant drawbacks of previous attempts to overcome these accuracy issues ([Page 30 Col 1 Par 1 – Col 2 Par 4] “Recently, simultaneous localization and mapping (SLAM) technology, which uses LiDAR technology to localize and mapping indoors and outdoors, has been used for various mobile robots. However, there is no GPS signal in indoor, mobile robots cannot have an exact location reference, which leads to inaccurate location recognition results [1]. … In order to perform indoor localization, information that can be obtained at indoors is used, usually radio signals. A typical approach is to measure the strength of existing wifi networks and use them to estimate locations [3] [4]. Although wifi is typically installed, it is not a common approach since multiple APs can exist, and wifi does not exist in all locations. There is also a disadvantage that wifi signals should be mapped on the entire building in advance. Similarly, there is a method that utilizes UWB sensors that use a wide radio bandwidth [2]. Those methods estimate the location of the mobile robot by obtaining distance information from multiple pre-installed UWB sensors. This allows for approximate indoor location estimation using less energy, but the UWB sensor must always be connected with an energy source, and sensor must be installed in advance, and the exact location of the installed UWB must be known. In order to compensate for the preinstallation, which is a disadvantage of the above method, another method [8] has been proposed to recognize and utilize changes in the earth’s magnetic field that occur in rebars of indoor structures. The magnetic method can correct the position by recognizing the change in magnetic values, but the disadvantage is that the change in the magnetic field can be caused by changes in the location of other iron objects such as tables, chairs, and computers. Localization using conventional RGB features has the disadvantage of not working well in plain indoor structures. To compensate for this disadvantage, a line visual SLAM algorithm [5] has also been proposed that utilizes line information that exists a lot in artificial structures. However, due to RGB image characteristics, even if calibrated through IMU, the exact metric is unknown. Also, additional works are inevitable to match localization and mapping results with the actual structure. Using Monte Carlo localization, a traditional indoor position recognition technology using LiDAR, [10] uses an existing map and 2D LiDAR to perform localization. Because it performs two-dimensional comparisons at robot height, it can produce inaccurate results and has a disadvantage that existing maps are needed”) To this end, Song presents a method for accurately determining the location of a sensor within a space using 3D lidar and existing building floorplans ([Page 31 Col 1 Par 1 -Col 2 Par 1] “To solve the problem mentioned above, in this paper, we proposed a matching algorithm that matches 3D line information extracted from 3D LiDAR with lines from the floorplan that are essential to exist for every building and MCL algorithm using results of matching. Line information with accurate metrics was extracted using a LiDAR sensor. Especially, the line was extracted only from the ceiling where floor information was removed for using only lines that were robust in the movement of furniture and other dynamic objects. Moreover, by using already existing floor plans, it is not necessary to make maps in advance. It also has the advantage of accurately matching the location with the actual map. As a result, based on the existing floorplan map shown in Fig. 1, MCL was possible by matching with the current line. Moreover, it was also possible to accumulate currently extracted lines on the map. … most buildings have floorplans for each floor, and in general, As shown in Fig. 3, it exists as a computer-aided design (CAD) file, so it is easy to manage and use. Therefore, it is efficient to use the floorplan for reference of an indoor environment and use it directly for localization.”) Overall, one of ordinary skill in the art would have recognized that combing Song with Zhang and Guida would result in much more accurate location determination for the unit used for 3D scanning, this more accurate location determination naturally leading to more accurate 3D scans which rely on this location as a basis for geometry calculation.
The combination of Zhang, Guida, and Song does not explicitly teach wherein a starting point of the scan trajectory is determined based on the two-dimensional, digitized building plan;
Frias makes obvious wherein a starting point of the scan trajectory is determined based on the two-dimensional, digitized building plan; ([Abstract] “The method starts by extracting floor plans from the BIM model according to the planned construction status, and including geometry and semantics of the building elements considered for construction control. The navigable space is defined from a binary map considering a security distance to building elements. After a grid-based and a triangulation-based distribution are implemented for generating scan position candidates, a visibility analysis is carried out to determine the optimal number and position of scans. The optimal route to visit all scan positions is addressed by using a probabilistic ant colony optimization algorithm.”)
Frias is analogous art because it is within the field of building scanning and model development. It would have been obvious to one of ordinary skill in the art to combine Frias with Zhang, Guida, and Song before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to make accurate scanning a much easier, less time-consuming task; this is noted by Frias as a significant issue with previous systems ([Page 2 Par 4] “Generally, scanning is a time-consuming task, so minimizing the number of scanning operations is essential for efficient scanning planning. In addition, data acquisition must be successful in terms of integrity.”) To this end, Frias presents a method for optimizing the scanning route, resulting in a scanning path that balances sensor visibility and total route length to minimizes scan time while maintaining high accuracy ([Page 2 Par 5] “In this work, a method to determinate the optimal scan positions and the optimal route followed by a stop &go system based on the use of BIM models, and considering data completeness as stopping criteria, is presented. … floor plans according to the planned construction status considered for construction control. The well-known DXF standard containing geometric information of the building elements is used to calculate candidates to scan positions, which are subsequently submitted to a visibility analysis using a ray-tracing algorithm. Next, scan positions are optimized based on visibility and data completeness as stopping criteria, and a probabilistic ant colony optimization algorithm is implemented to obtain a suboptimal route in a reasonable time.”) Overall, one of ordinary skill in the art would have recognized that combining Frias with Zhang, Guida, and Song would result in a scanning system that took significantly less time to complete the scanning process.
Claim 14. The elements of claim 14 are substantially the same as those of claim 1. Therefore, the elements of claim 14 are rejected due to the same reasons as outlined above for claim 1.
Claim 4. Zhang teaches wherein method steps b) to d) are repeated after predefined time steps. ([Page 113 Col 1 Par 2] “The position of the scanner in the first scan was set as the origin of the space coordinate system. Because the data are analyzed in the same way every day, only Day 3 is selected as an example to describe the process.” [Examiner’s note: steps b) to d) themselves are taught by the combination of Zhang, Guida, Song, and Frias. See the rejection of claim 1.])
Claim 5. Zhang teaches wherein method steps b) to d) are repeated after detection of a building object. ([Page 113 Col 1 Par 2- Col 2 Par 1] “The position of the scanner in the first scan was set as the origin of the space coordinate system. Because the data are analyzed in the same way every day, only Day 3 is selected as an example to describe the process… The coordinates of the critical corners of the object were acquired from the 3D model which had also been set using the State plane coordinate system, and which had been stored in a MicroStation .dgn file.” [Examiner’s note: detection of the objects is part of the daily schedule, i.e. the steps are repeated after the detection of the objects. steps b) to d) themselves are taught by the combination of Zhang, Guida, Song, and Frias. See the rejection of claim 1.])
Claim 6. Zhang teaches wherein a deviation of the three-dimensional building data from the building plan is detected based on the comparison data. ([Page 109 Col 2 Par 6 – Page 110 Col 2 Par 2] “In the scan data, there are numerous points, including many points that are not related to the object under consideration. So, if the objective is to monitor the percentage of completion of an object, the points associated with the object need to be extracted from the original data. A java script was developed to count the number of points in the related portions of the point clouds. To account for construction and laser scanner tolerances, the true coordinates of the object in the 3D model are augmented by 0.7% increasing the volume of the object by 2% following [36] observation in their laser scanner experiments. The points (N1) in the point cloud that fall within the augmented object are counted. If no points are detected on any of the faces of the object, it is concluded that the percentage of completion is 0% (Fig. 2a). If points are detected within the boundaries of the object, a decision needs to be made about whether these points belong to the object being considered or to other objects or trash that are situated in the location where the 3D model indicates the object being considered should be. If the points represent the object considered, the points should appear on all the faces of the object. If the points represent trash or another object, whose shape is not the exact same shape as the object considered, the points do not appear on all faces of the object considered… After shrinking the 3D model of an object by 1.5%, if no points can be counted on any face of the object, this signifies that some parts of the object are in place since the scanner does not record points inside the object. Therefore, one can use the N2/N1 ratio to measure progress. If the ratio N2/N1 ≤ 0.1%, the object is considered to be completed 100% (see Fig. 2b). The N2/N1 ratio is set as 0.1% rather than a flat 0% in order to account for the few points that may appear on any face of the object as a result of noisy data. If N2/N1 ≥ α, a threshold value, it is concluded that the points have nothing to do with the object considered, i.e., a percentage of completion of 0% (Fig. 2c). ” [Fig.2] Details the method of determining if objects in the point cloud match those in the plan, as well as their completion amount)
PNG
media_image7.png
725
587
media_image7.png
Greyscale
Claim 7. Zhang teaches wherein, if a deviation is identified, ([Page 109 Col 2 Par 6 – Page 110 Col 2 Par 2] “In the scan data, there are numerous points, including many points that are not related to the object under consideration. So, if the objective is to monitor the percentage of completion of an object, the points associated with the object need to be extracted from the original data. A java script was developed to count the number of points in the related portions of the point clouds. To account for construction and laser scanner tolerances, the true coordinates of the object in the 3D model are augmented by 0.7% increasing the volume of the object by 2% following [36] observation in their laser scanner experiments. The points (N1) in the point cloud that fall within the augmented object are counted. If no points are detected on any of the faces of the object, it is concluded that the percentage of completion is 0% (Fig. 2a). If points are detected within the boundaries of the object, a decision needs to be made about whether these points belong to the object being considered or to other objects or trash that are situated in the location where the 3D model indicates the object being considered should be. If the points represent the object considered, the points should appear on all the faces of the object. If the points represent trash or another object, whose shape is not the exact same shape as the object considered, the points do not appear on all faces of the object considered… After shrinking the 3D model of an object by 1.5%, if no points can be counted on any face of the object, this signifies that some parts of the object are in place since the scanner does not record points inside the object. Therefore, one can use the N2/N1 ratio to measure progress. If the ratio N2/N1 ≤ 0.1%, the object is considered to be completed 100% (see Fig. 2b). The N2/N1 ratio is set as 0.1% rather than a flat 0% in order to account for the few points that may appear on any face of the object as a result of noisy data. If N2/N1 ≥ α, a threshold value, it is concluded that the points have nothing to do with the object considered, i.e., a percentage of completion of 0% (Fig. 2c). ” [Fig.2] Details the method of determining if objects in the point cloud match those in the plan, as well as their completion amount) data ([Page 109 Col 2 Par 1] “3D point cloud data is captured using a laser scanner that comes typically with its own software”)
Guida makes obvious a building object and corresponding building object data are generated depending on the scan data and integrated into the building information model. ([Page 1023 Par 1 - 2] “Once the application of the model has been established, i.e. to conserve the heritage through re-functionalisation and maintenance, the level of development (LOD) to be achieved is determined equal to LOD G, corresponding to the degree of detail of an upgraded object, and a level of accuracy (LOA) necessary LOA20, according to the USIBD classification, which allows geometric deviations between the model and the point cloud between 15 and 50 mm. The software used for the BIM modelling is Autodesk Revit which, thanks to the compatibility with Autodesk Recap PRO, allows the direct import of the point cloud in .rcp format… through the recognition of the intersections between the surfaces detected based on the arrangement of points in the cloud, it is possible to represent the lines that interpolate these points, resulting in the basis for the construction of other elements (Fig. 3)… It is therefore proposed to use these shape elements as the basis for modelling the walls as system families, using the “wall from surfaces” property, thus obtaining intelligent objects, to which the specific semantics of wall is associated, that faithfully reproduce the irregular course of the surfaces. Although HBIM is assuming an important role for the study and quantification of degradation phenomena by guaranteeing a detailed geometric representation of the artefacts [10], there is a lack of parameters related to the state of preservation. A solution to the problem has been obtained with the creation of parameters with which to populate the model with reliable and updatable information. The proposed experimentation foresees the representation of each degradation phenomenon through a mapping obtained by positioning the various adaptive points of the element, on the basis of the point cloud. In accordance with the indications of the UNI 11182 standard, the screens are imported into the software, associating graphic and textual information to them in the form of shared parameters. For a complete characterisation of the project parameters, global parameters are created. In this way, a hybrid graphical-informative database is configured, which comprehensively describes the alteration processes affecting the building and can be continuously updated. The development of proprietary libraries has made it possible to codify an approach that could become standardised and usable for future applications.” [Fig. 3] Shows the BIM model generated using the point-cloud with the raw point-cloud data overlaid)
PNG
media_image3.png
830
1298
media_image3.png
Greyscale
Claim 12. Song makes obvious wherein the building plan is a floor plan or an electrical installation plan. ([Page 31 Col 1 Par 1] “To solve the problem mentioned above, in this paper, we proposed a matching algorithm that matches 3D line information extracted from 3D LiDAR with lines from the floorplan that are essential to exist for every building and MCL algorithm using results of matching. … Moreover, by using already existing floor plans, it is not necessary to make maps in advance. It also has the advantage of accurately matching the location with the actual map. As a result, based on the existing floorplan map shown in Fig. 1, MCL was possible by matching with the current line. Moreover, it was also possible to accumulate currently extracted lines on the map.” [Fig. 1] Shows imported floorplan data [Fig. 2] Shows the framework of the system, which clearly involves importing floorplan data)
Claim 13. Guida makes obvious wherein the building information model is a BIM model. [Abstract] “The work presents an efficient solution, based on integrating different data sources, for the digitization and modelling of an old structure with the HBIM methodology. … A procedural pipeline is formalized for data acquisition, processing, and generating a complete model for a disused industrial plant, formerly used for tobacco processing and located in the city of Battipaglia, Italy.” [Page 1020 Par 1] “In this paper we propose some results of a SCAN-to-BIM process for getting a digital model of the industrial plant by means a synergic approach of SLAM and aero-photogrammetry methods.”)
Claim 15. Zhang teaches A non-transitory computer readable hardware storage device having computer readable program code stored therein, said program code executable by a processor of a computer system to implement the method as claimed in claim 1. ([Page 109 Col 1 Par 5] “First, a 3D model of the structure is developed and the activities of the project are defined, using a 3D modeling software, such as MicroStation, Revit, or ArchiCad.” [Page 109 Col 2 Par 3] “Registration can be performed by commercially available independent software or by software provided by the manufacturer of the laser scanner, such as a program called Cyclone provided alongside the Leica HDS 6000 laser scanner that was used in this study. After registration, the coordinates of all points are stored in a file that includes four columns of information that involve a point's ID number and its x, y, z coordinates”)
(2) Claims 8-10 are rejected under 35 U.S.C. 103 as being unpatentable over Automated progress control using laser scanning technology (Hereinafter Zhang) in view of Scan-To-Bim Procedure for an Old Industrial Plant (Hereinafter Guida) in further view of Floorplan-based Localization and Map Update Using LiDAR Sensor (Hereinafter Song) as well as From BIM to Scan Planning and Optimization for Construction Control (Hereinafter Frias) in addition to Line-Based Registration of Photogrammetric Point Clouds with 3D City Models by means of Mixed Integer Linear Programming (Hereinafter Goebbels)
Claim 8. Zhang teaches wherein the method step of comparing the three- dimensional building data with the building plan ([Page 109 Col 2 Par 6 – Page 110 Col 2 Par 2] “In the scan data, there are numerous points, including many points that are not related to the object under consideration. So, if the objective is to monitor the percentage of completion of an object, the points associated with the object need to be extracted from the original data. A java script was developed to count the number of points in the related portions of the point clouds. To account for construction and laser scanner tolerances, the true coordinates of the object in the 3D model are augmented by 0.7% increasing the volume of the object by 2% following [36] observation in their laser scanner experiments. The points (N1) in the point cloud that fall within the augmented object are counted. If no points are detected on any of the faces of the object, it is concluded that the percentage of completion is 0% (Fig. 2a). If points are detected within the boundaries of the object, a decision needs to be made about whether these points belong to the object being considered or to other objects or trash that are situated in the location where the 3D model indicates the object being considered should be. If the points represent the object considered, the points should appear on all the faces of the object. If the points represent trash or another object, whose shape is not the exact same shape as the object considered, the points do not appear on all faces of the object considered… After shrinking the 3D model of an object by 1.5%, if no points can be counted on any face of the object, this signifies that some parts of the object are in place since the scanner does not record points inside the object. Therefore, one can use the N2/N1 ratio to measure progress. If the ratio N2/N1 ≤ 0.1%, the object is considered to be completed 100% (see Fig. 2b). The N2/N1 ratio is set as 0.1% rather than a flat 0% in order to account for the few points that may appear on any face of the object as a result of noisy data. If N2/N1 ≥ α, a threshold value, it is concluded that the points have nothing to do with the object considered, i.e., a percentage of completion of 0% (Fig. 2c). ”)comprises the following substeps: the three-dimensional building data, ([Page 109 Col 2 Par 1] “3D point cloud data is captured using a laser scanner that comes typically with its own software”)([Page 109 Col 1 Par 5] “First, a 3D model of the structure is developed and the activities of the project are defined, using a 3D modeling software, such as MicroStation, Revit, or ArchiCad. For example, as was done in this study, the volume/surface area of the objects involved in each activity can be calculated by MicroStation V8i using the object's 3D coordinates.”) ([Page 109 Col 1 Par 5] “First, a 3D model of the structure is developed and the activities of the project are defined, using a 3D modeling software, such as MicroStation, Revit, or ArchiCad. For example, as was done in this study, the volume/surface area of the objects involved in each activity can be calculated by MicroStation V8i using the object's 3D coordinates.”) ([Page 109 Col 2 Par 1] “3D point cloud data is captured using a laser scanner that comes typically with its own software”)
Zhang does not explicitly teach wherein the process comprises the following substeps: determining a horizontal structure based on the scan data; generating a first line based on the horizontal structure, comparing the first line with lines of the model wherein the first line is assigned to a second line of the model and correcting the spatial coordinates of the scan data assigned to the first line, depending on the position of the second line.
Song makes obvious wherein the process comprises the following substeps: determining a horizontal structure based on the scan data; generating a first line based on the horizontal structure, comparing the first line with lines of the building plan, wherein the first line is assigned to a second line of the building plan and correcting the spatial coordinates ([Fig. 2] Shows the process framework which involves extracting the ceiling from the 3D point cloud (determining a horizontal structure based on the scan data) extracting lines based on the ceiling (generating a first line based on the horizontal structure) and matching the extracted ceiling lines to the lines of the building plan to generate correction data([Page 31 Col 1 Par 1] “To solve the problem mentioned above, in this paper, we proposed a matching algorithm that matches 3D line information extracted from 3D LiDAR with lines from the floorplan that are essential to exist for every building and MCL algorithm using results of matching. Line information with accurate metrics was extracted using a LiDAR sensor. Especially, the line was extracted only from the ceiling where floor information was removed for using only lines that were robust in the movement of furniture and other dynamic objects. Moreover, by using already existing floor plans, it is not necessary to make maps in advance. It also has the advantage of accurately matching the location with the actual map. As a result, based on the existing floorplan map shown in Fig. 1, MCL was possible by matching with the current line. Moreover, it was also possible to accumulate currently extracted lines on the map.”)
PNG
media_image5.png
486
1392
media_image5.png
Greyscale
The combination of Zhang, Guida, Song, and Frias does not explicitly teach correcting the spatial coordinates of the scan data assigned to the first line, depending on the position of the second line.
Goebbels makes obvious correcting the spatial coordinates of the scan data assigned to the first line, depending on the position of the second line. ([Page 4 Col 2 Par 1 – Page 5 Col 1 Par 2] “Our goal is to match the footprint image with a similar image that we obtain from the city model. To this end, we draw a single picture of filled footprints of all CityGML buildings, only considering the area of the photogrammetric point cloud. Then we detect edges with the Canny operator. … Both on the footprint image and on the edge picture, we apply a probabilistic Hough transform to detect line segments, see Figure 4. In the following section we use the sets P and Q that contain line segments of the footprint image and the model’s edge picture, respectively. … we have to select from a candidate set of line pairs before we can compute a linear transformation to align line segments of P with line segments of Q. Let P = {(p1,1, p1,2),...,(pm,1, pm,2)} and Q = {(q1,1,q1,2),...,(qn,1,qn,2)}. Each line segment is defined by its two endpoints that are given in homogeneous coordinates, for example pi,k = (pi,k.x, pi,k.y,1) ⊤. Now we have to determine a linear transform L that aligns the largest possible subset of P with a corresponding subset of Q by using translation, scaling and rotation as feasible operations. A common method to find large corresponding sets of line features is to use RANSAC in Hough space. For example, (Colleu et al., 2008) use this approach to match video frames with city model data. In contrast to this we match bounded line segments with a MIP that automatically also computes an initial version of the transformation matrix. First, we have to find matching candidate pairs between P and Q. … We have to maximize an objective function like m ∑ i=1 n ∑ j=1 xi, j (1) subject to the restriction that there is a linear mapping L = s1 cos(α) −s1 sin(α) d1 s2 sin(α) s2 cos(α) d2 0 0 1 ∈ R 3×3 that approximately maps line of segment (pi,1, pi,2) onto line of (qj,1,qj,2) if xi, j = 1.” [Page 6 Col 1 Par 4 – Col 2 Par 2] “Based on L, we can align the point cloud in the x-y-plane using matrix … Then we finally align the cloud with the city model by multiplying its points with [Matrix] After aligning the point cloud with the city model, we can match model walls with areas in video frames” [Figure 4] describes matching line segments between scan and model data] [Figure 6] Shows a street during initial scan, combination of scan and model, and final scan-model alignment) [Examiner’s note: P and Q refer to sets that contain detected lines from the scan data and model data, respectively])
PNG
media_image8.png
510
754
media_image8.png
Greyscale
PNG
media_image9.png
265
671
media_image9.png
Greyscale
Goebbels is analogous art because it is within the field of model generation from point cloud data. It would have been obvious to one of ordinary skill in the art to combine it with Zhang, Guida, Song, and Frias before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to better align the building plans with the scan data. As noted by Goebbels doing the alignment process (known as registration) manually can be extremely difficult, especially to the degree of precision required to be used with highly detailed models ([Page 1 Col 2 Par 2] “It is difficult to do a manual registration of photogrammetric point clouds with sufficient precision to cleanly project points or textured meshes to CityGML walls”) To this end, Goebbels presents a method to automatically align/register building plans with scan data. ([Page 1 Col 2 Par 2] “It is difficult to do a manual registration of photogrammetric point clouds with sufficient precision to cleanly project points or textured meshes to CityGML walls. Therefore, we apply an automatic precise registration of the point cloud with the city model.”) Overall, one of ordinary skill in the art would have recognized that combining Goebbels with Zhang, Guida, Song, and Frias would result in a system that makes combining and comparing the scan data and building plan significantly easier.
Claim 9. Goebbels teaches wherein the at least one building object is detected ([Page 4 Col 2 Par 1] “Our goal is to match the footprint image with a similar image that we obtain from the city model. To this end, we draw a single picture of filled footprints of all CityGML buildings, only considering the area of the photogrammetric point cloud. Then we detect edges with the Canny operator. These edges correspond with facades…”) based on the second line in the building plan. ([Page 4 Col 2 Par 2] “In the following section we use the sets P and Q that contain line segments of the footprint image and the model’s edge picture, respectively.”)
Claim 10. Zhang teaches ([Page 109 Col 2 Par 1] “3D point cloud data is captured using a laser scanner that comes typically with its own software”)
Song makes obvious wherein the horizontal structure is detected based on the three-dimensional building data by virtue of data points lying one above the other in the vertical direction ([Page 32 Col 1 Par 3 – Col 2 Par 1] “C. Ceiling Detection First, filter the pointcloud based on a specific height to extract the ceiling part from to local pointcloud accumulated in Subsection. II.B as shown in Fig. 5(a). Since this paper targets a mobile robot, it is based on the height of the 3D LiDAR sensor mounted on the mobile robot. After filtering out the floor and wall parts of the pointcloud, the pointcloud includes ceiling part remains as shown in Fig. 5(b). After that, planes are extracted using the RANSAC algorithm for the pointcloud, and only the plane whose normal vector coincides with the vertical direction is remained. Then the pointcloud corresponding to that plane will be extracted. Through this, as shown in Fig. 5(c), only the pointcloud corresponding to the ceiling part can be extracted, and it becomes the base for extracting lines for the following subsection.
Goebbels makes obvious wherein the horizontal structure is detected by virtue of data points lying one above the other in the vertical direction being counted and being connected horizontally to adjacent data points along the respectively topmost data point provided that the adjacent data points in the vertical direction have a substantially corresponding number. ([Abstract] “We detect wall planes to rotate these clouds so that walls become vertical. This allows us to find buildings’ footprints by accumulating points that are orthogonally projected to the ground. … To this end, we match detected footprints with corresponding footprints of CityGML models in a x-y-plane based on line segments” [Page 4 Col 1 Par 1-2] “To generate a preliminary binary image of likely wall footprints, a resolution of 9 pixels per square meter is sufficient for our data, see Figure 3. We compute minimum and maximum z-coordinates (height values) of all points with x-and y-coordinates within the pixel’s area. If these values at least differ 3.5m in height (one building level) and if there exist at least eight points with z-coordinates pairwise belonging to disjoint intervals of width 0.5m then we classify the pixel as being part of a wall footprint. One could also generate a density image by counting the points above the pixel’s area… Walls might not be exactly vertical. Before we reduce the cloud to wall and ground points, we have to rotate it with a matrix D to make walls upright. To this end, we divide the ground into 10m×10m sections. For each section we iteratively apply a RANSAC algorithm to the section’s subset of the cloud that also corresponds roughly with previously computed pixels of wall footprints. With RANSAC we estimate nearly vertical planes for each section.”)
(3) Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Automated progress control using laser scanning technology (Hereinafter Zhang) in view of Scan-To-Bim Procedure for an Old Industrial Plant (Hereinafter Guida) in further view of Floorplan-based Localization and Map Update Using LiDAR Sensor (Hereinafter Song) as well as From BIM to Scan Planning and Optimization for Construction Control (Hereinafter Frias) in addition to Building Change Detection for Remote Sensing Images Using a Dual-Task Constrained Deep Siamese Convolutional Network Model (Hereinafter Liu)
Claim 11. Zhang teaches wherein the method step of comparing the three- dimensional building data with the building plan ([Page 109 Col 2 Par 6 – Page 110 Col 2 Par 2] “In the scan data, there are numerous points, including many points that are not related to the object under consideration. So, if the objective is to monitor the percentage of completion of an object, the points associated with the object need to be extracted from the original data. A java script was developed to count the number of points in the related portions of the point clouds. To account for construction and laser scanner tolerances, the true coordinates of the object in the 3D model are augmented by 0.7% increasing the volume of the object by 2% following [36] observation in their laser scanner experiments. The points (N1) in the point cloud that fall within the augmented object are counted. If no points are detected on any of the faces of the object, it is concluded that the percentage of completion is 0% (Fig. 2a). If points are detected within the boundaries of the object, a decision needs to be made about whether these points belong to the object being considered or to other objects or trash that are situated in the location where the 3D model indicates the object being considered should be. If the points represent the object considered, the points should appear on all the faces of the object. If the points represent trash or another object, whose shape is not the exact same shape as the object considered, the points do not appear on all faces of the object considered… After shrinking the 3D model of an object by 1.5%, if no points can be counted on any face of the object, this signifies that some parts of the object are in place since the scanner does not record points inside the object. Therefore, one can use the N2/N1 ratio to measure progress. If the ratio N2/N1 ≤ 0.1%, the object is considered to be completed 100% (see Fig. 2b). The N2/N1 ratio is set as 0.1% rather than a flat 0% in order to account for the few points that may appear on any face of the object as a result of noisy data. If N2/N1 ≥ α, a threshold value, it is concluded that the points have nothing to do with the object considered, i.e., a percentage of completion of 0% (Fig. 2c). ” [Fig.2] Details the method of determining if objects in the point cloud match those in the plan, as well as their completion amount)
PNG
media_image7.png
725
587
media_image7.png
Greyscale
The combination of Zhang, Guida, Song, and Frias does not explicitly teach wherein structure comparison is carried out by means of a machine learning model trained for this purpose.
Liu makes obvious wherein structure comparison is carried out by means of a machine learning model trained for this purpose. ([Abstract] “In recent years, building change detection methods have made great progress by introducing deep learning, but they still suffer from the problem of the extracted features not being discriminative enough, resulting in incomplete regions and irregular boundaries. To tackle this problem, we propose a dual-task constrained deep Siamese convolutional network (DTCDSCN) model” [Page 812 Col 1 Par 1-2] “in this letter, we propose a dual-task constrained deep Siamese convolutional network (DTCDSCN) model to jointly optimize the building change detection and semantic segmentation tasks. In summary, this letter makes three main contributions. 1) We propose a novel building change detection network model, which can simultaneously carry out the main task of building change detection and the auxiliary task of building extraction. The two tasks share the same feature extraction layer, so the auxiliary task enables the model to learn discriminative object-level features that contribute to more precise building change detection results.”)
Liu is analogous art because it is within the field of discrepancy detection between two representations of a building. It would have been obvious to one of ordinary skill in the art to combine it with Zhang, Guida, Song, and Frias before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to better detect the differences between the two building representations. As noted by Liu, manual change detection is can be extremely time consuming and difficult, while automatic methods are known to suffer from inaccuracy. ([Page 811 Col 1 Par 1] “Building change detection is extremely important in the fields of land-use planning, city management, and emergency response. However, manual change detection is time-consuming and labor-intensive, so there is a need for automatic and efficient change detection” [Abstract] “In recent years, building change detection methods have made great progress by introducing deep learning, but they still suffer from the problem of the extracted features not being discriminative enough, resulting in incomplete regions and irregular boundaries.”) To this end, Liu presents a system for building change detection that is automatic and extremely accurate by leveraging machine learning models ([Abstract] “To tackle this problem, we propose a dual-task constrained deep Siamese convolutional network (DTCDSCN) model, which contains three subnetworks: a change detection network and two semantic segmentation networks. DTCDSCN can accomplish both change detection and semantic segmentation at the same time, which can help to learn more discriminative object-level features and obtain a complete change detection map”) Overall, one of ordinary skill in the art would have recognized that combining Liu with Zhang, Guida, Song, and Frias would result in a system that is capable of quicker, more accurate comparison and detection of differences between building representations.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael P Mirabito whose telephone number is (703)756-1494. The examiner can normally be reached M-F 10:30 am - 6:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emerson Puente can be reached at (571) 272-3652. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.P.M./ Examiner, Art Unit 2187
/EMERSON C PUENTE/ Supervisory Patent Examiner, Art Unit 2187
Prosecution Insights