DETAILED ACTION
Claims 1-8 and 10-20 are presented for examination. Claims 1, 4, 10, and 11 stand currently amended.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's remarks filed 27 April 2026 have been fully considered and Examiner’s response is as follows:
Applicant remarks page 8 argues:
The Office Action appears to confuse a GNSS restricted environment with a GNSS environment. The two are opposite. A GNSS restricted environment is a location or situation where Global Navigation Satellite System (GNSS) signals are either denied, degraded, or too weak to provide reliable positioning.
This argument has been fully considered and is persuasive. Therefore, the rejection has been withdrawn. However after further consideration, a new ground of rejection is made over an updated interpretation of Pettersson and further in view of US patent 9,846,242 B2 Gil, et al. [herein “Gil”]. Because this new ground of rejection was not necessitated by a change in claim scope, this rejection is non-final.
Applicant remarks page 8 further argues:
Accordingly, the assertion that Pettersson uses GPS for positioning renders Pettersson inoperable in a GNSS restricted environment. A person of ordinary skill in the art wouldn't look to a satellite-dependent system, such as GPS or RTK, to solve the problem of achieving precision layout in an environment where those very satellite signals are restricted or unavailable.
This argument is unpersuasive.
Pettersson also discloses an internal construction site coordinate system as an alternative to GPS.
Pettersson column 13 lines 13-17 disclose “the mobile support system 5,5',7 is provided with specific location and orientation information stored on the construction database. For example, this might include GPS coordinates, 3D coordinates of an internal construction site coordinate system, 3D building information.” An internal construction site coordinate system, as contrasted with the GPS coordinates, is not a GNSS environment. The internal construction site coordinate system corresponds with a GNSS restricted environment.
Applicant remarks page 9 further argues:
Claim 1 recites the element above. The Office Action relies on Zollmann's teaching of using RTK, a satellite-based system, to achieve the claimed precision.
Like missing element A above, Zollmann cannot teach or suggest marking points in a GNSS restricted environment using a method or system (e.g., RTK) that relies on GNSS signals since those signals are restricted or unavailable.
This argument has been fully considered and is persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made over US 10,713,607 B2 Pettersson, et al. [herein “Pettersson”] in view of US patent 10,788,323 B2 Singer [herein “Singer”], US patent 9,846,242 B2 Gil, et al. [herein “Gil”], and Degani, A., et al. “An Automated System for Projection of Interior Construction Layouts” IEEE Transactions on Automation Science & Engineering, vol. 16, no. 4 (2019) [herein “Degani”].
Pettersson column 13 lines 13-17 disclose “an internal construction site coordinate system.”
But Pettersson and Singer does not explicitly disclose accuracy better than 5 centimeters; however, in analogous art of construction site surveying, Gil column 4 lines 41-50 teach:
The fixed location antennas 20 may be placed at previously known landmarks or coordinates, via conventional surveying methods ….
The onboard GPS devices may be capable of determining location within an accuracy range fifteen feet to less than one inch.
Gil column 5 lines 3-5 teach “A portable antenna 26 may be utilized to report the location data 14 to the computing device 12 from the restricted antenna system 18.”
Gil column 7 lines 55-58 teach “Measurements from the RAS 18 are understood to be highly accurate, or at least more accurate than the measurements from the GPS module 16. In some embodiments, accuracy within ~7 cm is understood to be possible.”
Accuracies of approximately 7 cm is within the scope of similar and overlapping ranges to a 5cm accuracy. Accuracies of less than one inch correspond with accuracies less than 2.54 cm which is less than 5 cm. The RAS being more accurate than the measurements from the GPS module correspond with an accuracy range less than 2.54 cm of the GPS device taught at Gil column 5 line 50.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pettersson, Singer, and Gil. One having ordinary skill in the art would have found motivation to use location determination of less than one inch into the system of construction site management with marking robot because “RAS 18 is understood to be the most suitable for navigating the interior of buildings, as GPS signals typically cannot penetrate ceilings and walls.” Gil column 7 lines 59-61. This is an art recognized suitability for the intended purpose of providing location measurements at a construction site building interior. See MPEP §2144.07.
Applicant remarks page 9 further argues:
C. Third missing element - "aligning an internal map of the augmented-reality device to the construction site by comparing locations of marks of points in the internal map to data from the building plan."
To expedite prosecution, claim 1 is amended to recite the above. The marks of points in claim 1 are physical markings and specifically, "permanent marker or spray paint."
The Office Action relies on Singer's reference marker 200. ….
This argument is unpersuasive.
Pettersson column 12 lines 43-48 disclose:
The mobile support system might be a ground based robotic vehicle or a flying robotic vehicle (UAV) and the markings might be based on at least one of a 2D marking, e.g. a painted marking, a laser burned marking, or written information, a 3D structure, e.g. by a surface spattering device.
A robotic vehicle is robotic. Painting markings is marking out points using paint.
Furthermore, light is a physical thing. Thus, the projected markings of Singer using light also remain with the amended claim scope of “physical markings.”
Applicant remarks page 9 further argues:
D. Fourth missing element - "a number of points marked at the construction site is equal to or greater than 500 .... "
….
Further, the Office Action relies on Degani teaching using robotic marking systems where "the quantity of layout work is large enough to justify their setup costs," and "Examiner considers 'large enough' to denote an amount which is at least similar to 500 or more."
Examiner is relying on personal knowledge by using the phrase "Examiner considers." To rely on personal knowledge or experience, the Examiner must provide an affidavit or declaration "setting forth the specific factual statements and explanations to support the finding." MPEP 2144.03(C). This has not been done.
This argument is unpersuasive.
Examiner has not relied upon personal knowledge. Examiner cites MPEP 2144.05(I) regarding similar and overlapping ranges and amounts. Applicant may rebut Examiner’s assertion of obviousness regarding this amount (“a number of points marked … greater than 500”) by showing unexpected results or criticality of the claimed amount. See MPEP §2144.05(I).
Examiner’s consideration is in making a legal finding that MPEP §2144.05(I) applies, not a factual finding based on personal knowledge.
Drawings
The drawings were received on 27 April 2026. These drawings are accepted.
Claim Rejections - 35 USC § 112
Claim 1 has been appropriately corrected. Accordingly, Examiner's rejection of claims 1-3 under § 112 is withdrawn.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3 and 7
Claims 1-3 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over US 10,713,607 B2 Pettersson, et al. [herein “Pettersson”] in view of US patent 10,788,323 B2 Singer [herein “Singer”], US patent 9,846,242 B2 Gil, et al. [herein “Gil”], and Degani, A., et al. “An Automated System for Projection of Interior Construction Layouts” IEEE Transactions on Automation Science & Engineering, vol. 16, no. 4 (2019) [herein “Degani”].
Claim 1 recites “1. A method for construction layout using robotic marking and augmented-reality metadata.” Pettersson column 12 lines 43-48 disclose:
The mobile support system might be a ground based robotic vehicle or a flying robotic vehicle (UAV) and the markings might be based on at least one of a 2D marking, e.g. a painted marking, a laser burned marking, or written information, a 3D structure, e.g. by a surface spattering device.
A robotic vehicle is robotic.
Pettersson does not explicitly disclose augmented reality; however, in analogous art of building construction surveying information, Singer column 9 lines 20-21 teaches “the [Augmented-Reality (AR)]-data may be overlaid onto the real view of the scene which the user has when he is wearing the AR-device.”
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pettersson and Singer. One having ordinary skill in the art would have found motivation to use augmented reality into the system of construction site management with marking robot for the advantageous purpose of “to display virtual objects with a precise spatial link to the coordinate system, i.e. to the natural environment” and “improving the positioning accuracy of displayed AR-data on the display of an AR-device.” See Singer column 2 lines 10-12 and 41-43.
Claim 1 further recites “the method comprising: receiving data from a building plan, wherein the building plan includes a plurality of elements.” Pettersson column 12 lines 2-10 teach:
storing a general construction database 2, the database comprising structured datasets of a plurality of object entities of physical construction components, in particular wherein the datasets comprise digital construction plan information (e.g. 2D or 3D information, or CAD information) and attribute information of the object entities, a hierarchical structure of the object entities representing a desired construction result to be established by the physical construction components,
Claim 1 further recites “marking out points at a construction site related to the plurality of elements in the building plan, wherein: marking out points is performed by a robotic device.” Pettersson column 12 lines 43-48 disclose:
The mobile support system might be a ground based robotic vehicle or a flying robotic vehicle (UAV) and the markings might be based on at least one of a 2D marking, e.g. a painted marking, a laser burned marking, or written information, a 3D structure, e.g. by a surface spattering device.
A robotic vehicle is robotic. Painting markings is marking out points.
Claim 1 further recites “and points are marked at the construction site within a pre-established accuracy using a robotic total station with the robotic device.” Pettersson column 13 lines 9-11 disclose "the mobile support system might also be adapted both for taking control measurements and for tagging markings at the same time, i.e. in one go.” Pettersson column 14 lines 25-26 disclose “regarding special requirements for the accuracy and small tolerance levels.” A tolerance level corresponds with a pre-established required accuracy.
Pettersson column 15 lines 5-9 disclose “the mobile support system 7' comprises a temperature and humidity detector 19' and a generic laser based surveying device 20 for measuring spatial points on an arbitrarily shaped surface and for measuring edges and corners of adjacent surfaces.” A person of ordinary skill in the art would understand a laser-based surveying device corresponds with a total station. The mobile support system corresponds with the robotic device.
Claim 1 further recites “marks are made using a permanent marker or spray paint.” Pettersson column 12 lines 43-48 disclose:
The mobile support system might be a ground based robotic vehicle or a flying robotic vehicle (UAV) and the markings might be based on at least one of a 2D marking, e.g. a painted marking, a laser burned marking, or written information, a 3D structure, e.g. by a surface spattering device.
Painted marking are marks made using a spray paint.
Claim 1 further recites “metadata of points are not marked using the robotic device.” Pettersson column 13 lines 51-57 disclose “Additional information about the construction and the work package is given as human readable instructions 10 or as code instructions, particularly by pictograms 11, e.g. indicating painting of the floor with a corresponding color, or by machine readable codes 12, e.g. representing guiding points or specific settings for an automated or semi-automated executing device.” The additional information in the form of human readable instructions or code instructions correspond with metadata of various points which are not marked by the tags.
Claim 1 further recites “the robotic device operates in a global navigation satellite system (GNSS) restricted environment.” Pettersson column 13 lines 13-17 disclose “the mobile support system 5,5',7 is provided with specific location and orientation information stored on the construction database. For example, this might include GPS coordinates, 3D coordinates of an internal construction site coordinate system, 3D building information.” An internal construction site coordinate system, as contrasted with the GPS coordinates, is not a GNSS environment. The internal construction site coordinate system corresponds with a GNSS restricted environment.
Claim 1 further recites “accuracy of marking points at the construction site, compared to the building plan, is equal to or better than 5 centimeters.” Pettersson column 13 lines 13-17 disclose “an internal construction site coordinate system.”
But Pettersson and Singer does not explicitly disclose accuracy better than 5 centimeters; however, in analogous art of construction site surveying, Gil column 4 lines 41-50 teach:
The fixed location antennas 20 may be placed at previously known landmarks or coordinates, via conventional surveying methods ….
The onboard GPS devices may be capable of determining location within an accuracy range fifteen feet to less than one inch.
Gil column 5 lines 3-5 teach “A portable antenna 26 may be utilized to report the location data 14 to the computing device 12 from the restricted antenna system 18.”
Gil column 7 lines 55-58 teach “Measurements from the RAS 18 are understood to be highly accurate, or at least more accurate than the measurements from the GPS module 16. In some embodiments, accuracy within ~7 cm is understood to be possible.”
Accuracies of approximately 7 cm is within the scope of similar and overlapping ranges to a 5cm accuracy. Accuracies of less than one inch correspond with accuracies less than 2.54 cm which is less than 5 cm. The RAS being more accurate than the measurements from the GPS module correspond with an accuracy range less than 2.54 cm of the GPS device taught at Gil column 5 line 50.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pettersson, Singer, and Gil. One having ordinary skill in the art would have found motivation to use location determination of less than one inch into the system of construction site management with marking robot because “RAS 18 is understood to be the most suitable for navigating the interior of buildings, as GPS signals typically cannot penetrate ceilings and walls.” Gil column 7 lines 59-61. This is an art recognized suitability for the intended purpose of providing location measurements at a construction site building interior. See MPEP §2144.07.
Claim 1 further recites “and a number of points marked at the construction site is equal to or greater than 500.” In analogous art of managing construction layout information, Degani page 1826 left column second paragraph teaches “Robotic marking systems are restricted to environments where the floors are clean and clear, for marking and travel, and where the quantity of layout work is large enough to justify their setup costs.” Examiner considers “large enough” to render obvious an amount which is at least similar to 500 or more. MPEP §2144.05(I) states “a prima facie case of obviousness exists where the claimed ranges or amounts do not overlap with the prior art but are merely close.”
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pettersson, Singer, Gil and Degani. One having ordinary skill in the art would have found motivation to use marking at least 500 points into the system of construction site management with marking robot for the advantageous purpose “to justify their setup costs.” See Degani page 1826 left column second paragraph.
Claim 1 further recites “transmitting data from the building plan to an augmented-reality device, wherein data from the building plan includes metadata about the points.” Pettersson column 13 lines 51-57 disclose “Additional information about the construction and the work package is given as human readable instructions 10 or as code instructions, particularly by pictograms 11, e.g. indicating painting of the floor with a corresponding color, or by machine readable codes 12, e.g. representing guiding points or specific settings for an automated or semi-automated executing device.” The additional information in the form of human readable instructions or code instructions correspond with metadata of various points.
Pettersson does not explicitly disclose augmented reality; however, in analogous art of building construction surveying information, Singer column 9 lines 20-21 teaches “the [Augmented-Reality (AR)]-data may be overlaid onto the real view of the scene which the user has when he is wearing the AR-device.”
Singer column 8 lines 57-62 teach:
Accordingly, the AR-glasses may store, or receive the AR-data from a server, i.e. the augmented information displayed for assisting the user. These AR-data are spatially related to the reference system, in particular to the BIM model. The BIM-model may optionally also be stored on the AR-device or retrieved from the server.
The Building Information Model (BIM) corresponds with building plan data.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pettersson and Singer. One having ordinary skill in the art would have found motivation to use augmented reality into the system of construction site management with marking robot for the advantageous purpose of “to display virtual objects with a precise spatial link to the coordinate system, i.e. to the natural environment” and “improving the positioning accuracy of displayed AR-data on the display of an AR-device.” See Singer column 2 lines 10-12 and 41-43.
Claim 1 further recites “aligning an internal map of the augmented-reality device to the construction site by comparing locations of marks of points in the internal map to data from the building plan.” Singer column 9 lines 24-30 teach:
Alternatively to the act of referencing the AR-device relative to the reference system by means of the referencing marker, the process can also be understood in such a way that the AR-device does not really "lock" in into the reference system but merely determines a spatial relationship between the referencing marker and AR-data assigned to the referencing marker.
Determining a spatial relationship between a reference marker and AR data is a comparison between points in the internal map and data from the building plan.
Singer column 10 lines 4-8 teach “the projector 20 to project the marker accordingly on a surface adjacent to where the AR-data are linked, in particular wherein the AR-data and the projection of the referencing marker have the same position within the reference system or BIM-model.” The AR data and reference system or BIM model having linked same positions corresponds with the AR data being aligned with the building plan information. The reference marker and/or AR data correspond with the interna map.
Claim 1 further recites “and presenting metadata about the points to a user of the augmented-reality device as the user looks at the points at the construction site.” Pettersson does not explicitly disclose augmented reality; however, in analogous art of building construction surveying information, Singer column 9 lines 20-21 teaches “the [Augmented-Reality (AR)]-data may be overlaid onto the real view of the scene which the user has when he is wearing the AR-device.” The AR data being overlaid onto the real view of the scene is a presentation of the metadata to a user as the user looks at the construction site.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pettersson and Singer. One having ordinary skill in the art would have found motivation to use augmented reality into the system of construction site management with marking robot for the advantageous purpose of “to display virtual objects with a precise spatial link to the coordinate system, i.e. to the natural environment” and “improving the positioning accuracy of displayed AR-data on the display of an AR-device.” See Singer column 2 lines 10-12 and 41-43.
Claim 2 further recites “2. The method of claim 1, wherein the augmented-reality device is a first augmented-reality device, and the method further comprises transmitting data from the building plan to a second augmented-reality device.” Singer column 6 lines 50-51 disclose “Augmented Reality (AR)-device according to the invention, i.e. AR-glasses 10 and an AR-helmet 11.” Singer column 6 lines 59-67 teach:
The display shown in the two examples of FIGS. 1a and 1b may comprise a projector (not shown) for projecting the AR-data onto the display 120/121. Other embodiments of the AR-device according to the invention are handheld devices such as smart phones or tablet computers. Such handhelds usually also comprise a visual sensor (camera), a computer (processor) and a display (screen) and are configured to display referenced AR-data.
Glasses, helmet, projector, smart phones, and tablet computers are each respective augmented-reality devices. Singer column 2 lines 1-3 teach “Augmented Reality (AR) systems are often utilised for supporting users on a site by referenced visualisations.” Plural users correspond with a recognition there may be more than one user and accordingly a second or more device.
Claim 3 further recites “3. The method of claim 2, wherein the first augmented-reality device receives different metadata than the second augmented-reality device.” Singer column 9 lines 17-23 teach:
Whatever data distribution option is chosen, once the AR-device 10 is provided with a selection of AR-data 1200, 1201 and with a selection of at least one referencing marker 200, the AR-data may be overlaid onto the real view of the scene which the user has when he is wearing the AR-device in case the referencing marker 200 is situated within the field of view of the visual sensor 100.
Selected AR-data corresponds with respective received metadata. The overlaying being contextual on the referencing marker being situated within a field of view corresponds with different metadata being displayed for different fields of view and different reference markers.
Claim 7 further recites “7. The system of claim 4, wherein a number of points marked is equal to or greater than 200.” In analogous art of managing construction layout information, Degani page 1826 left column second paragraph teaches “Robotic marking systems are restricted to environments where the floors are clean and clear, for marking and travel, and where the quantity of layout work is large enough to justify their setup costs.” Examiner considers “large enough” to render obvious an amount which is at least similar to 200 or more. MPEP §2144.05(I) states “a prima facie case of obviousness exists where the claimed ranges or amounts do not overlap with the prior art but are merely close.”
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pettersson, Singer, Gil and Degani. One having ordinary skill in the art would have found motivation to use marking at least 200 points into the system of construction site management with marking robot for the advantageous purpose “to justify their setup costs.” See Degani page 1826 left column second paragraph.
Claims 4-6, 10-14, 19, and 20
Claims 4-6, 10-14, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over US 10,713,607 B2 Pettersson, et al. [herein “Pettersson”] in view of US patent 10,788,323 B2 Singer [herein “Singer”].
Claim 4 recites “4. A system for construction layout using robotic marking and augmented-reality metadata.” Pettersson column 12 lines 43-48 disclose:
The mobile support system might be a ground based robotic vehicle or a flying robotic vehicle (UAV) and the markings might be based on at least one of a 2D marking, e.g. a painted marking, a laser burned marking, or written information, a 3D structure, e.g. by a surface spattering device.
A robotic vehicle is robotic.
Pettersson does not explicitly disclose augmented reality; however, in analogous art of building construction surveying information, Singer column 9 lines 20-21 teaches “the [Augmented-Reality (AR)]-data may be overlaid onto the real view of the scene which the user has when he is wearing the AR-device.”
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pettersson and Singer. One having ordinary skill in the art would have found motivation to use augmented reality into the system of construction site management with marking robot for the advantageous purpose of “to display virtual objects with a precise spatial link to the coordinate system, i.e. to the natural environment” and “improving the positioning accuracy of displayed AR-data on the display of an AR-device.” See Singer column 2 lines 10-12 and 41-43.
Claim 4 further recites “the system comprising: a robotic device comprising instructions to mark out points at a construction site using physical markings, wherein the points are marked within a pre-established accuracy related to a plurality of elements in a building plan.” Pettersson column 12 lines 43-48 disclose:
The mobile support system might be a ground based robotic vehicle or a flying robotic vehicle (UAV) and the markings might be based on at least one of a 2D marking, e.g. a painted marking, a laser burned marking, or written information, a 3D structure, e.g. by a surface spattering device.
A robotic vehicle is robotic. Painting markings is marking out points using physical markings.
Pettersson column 13 lines 9-11 disclose "the mobile support system might also be adapted both for taking control measurements and for tagging markings at the same time, i.e. in one go.” Pettersson column 14 lines 25-26 disclose “regarding special requirements for the accuracy and small tolerance levels.” A tolerance level corresponds with a pre-established required accuracy.
Pettersson column 15 lines 5-9 disclose “the mobile support system 7' comprises a temperature and humidity detector 19' and a generic laser based surveying device 20 for measuring spatial points on an arbitrarily shaped surface and for measuring edges and corners of adjacent surfaces.” A person of ordinary skill in the art would understand a laser-based surveying device corresponds with a total station. The mobile support system corresponds with the robotic device.
Claim 4 further recites “and an augmented-reality device comprising instructions to: receive data of the building plan, wherein data of the building plan includes metadata about the points.” Pettersson column 12 lines 2-10 teach:
storing a general construction database 2, the database comprising structured datasets of a plurality of object entities of physical construction components, in particular wherein the datasets comprise digital construction plan information (e.g. 2D or 3D information, or CAD information) and attribute information of the object entities, a hierarchical structure of the object entities representing a desired construction result to be established by the physical construction components,
Claim 4 further recites “align an internal map of the augmented-reality device to the construction site based on the physical markings and the building plan.” Singer column 9 lines 24-30 teach:
Alternatively to the act of referencing the AR-device relative to the reference system by means of the referencing marker, the process can also be understood in such a way that the AR-device does not really "lock" in into the reference system but merely determines a spatial relationship between the referencing marker and AR-data assigned to the referencing marker.
Determining a spatial relationship between a reference marker and AR data is a comparison between points in the internal map and data from the building plan.
Singer column 10 lines 4-8 teach “the projector 20 to project the marker accordingly on a surface adjacent to where the AR-data are linked, in particular wherein the AR-data and the projection of the referencing marker have the same position within the reference system or BIM-model.” The AR data and reference system or BIM model having linked same positions corresponds with the AR data being aligned with the building plan information. The reference marker and/or AR data correspond with the interna map.
Claim 4 further recites “and present metadata about the points to a user of the augmented-reality device as the user looks at points at the construction site.” Pettersson does not explicitly disclose augmented reality; however, in analogous art of building construction surveying information, Singer column 9 lines 20-21 teaches “the [Augmented-Reality (AR)]-data may be overlaid onto the real view of the scene which the user has when he is wearing the AR-device.” The AR data being overlaid onto the real view of the scene is a presentation of the metadata to a user as the user looks at the construction site.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pettersson and Singer. One having ordinary skill in the art would have found motivation to use augmented reality into the system of construction site management with marking robot for the advantageous purpose of “to display virtual objects with a precise spatial link to the coordinate system, i.e. to the natural environment” and “improving the positioning accuracy of displayed AR-data on the display of an AR-device.” See Singer column 2 lines 10-12 and 41-43.
Claim 5 further recites “5. The system of claim 4, wherein aligning the internal map of the augmented-reality device to the construction site comprises comparing locations of points in the internal map to data from the building plan.” Singer column 9 lines 24-30 teach:
Alternatively to the act of referencing the AR-device relative to the reference system by means of the referencing marker, the process can also be understood in such a way that the AR-device does not really "lock" in into the reference system but merely determines a spatial relationship between the referencing marker and AR-data assigned to the referencing marker.
Determining a spatial relationship between a reference marker and AR data is a comparison between points in the internal map and data from the building plan.
Singer column 10 lines 4-8 teach “the projector 20 to project the marker accordingly on a surface adjacent to where the AR-data are linked, in particular wherein the AR-data and the projection of the referencing marker have the same position within the reference system or BIM-model.” The AR data and reference system or BIM model having linked same positions corresponds with the AR data being aligned with the building plan information. The reference marker and/or AR data correspond with the interna map.
Claim 6 further recites “6. The system of claim 4, wherein instructions of the robotic device do not have the robotic device mark metadata near points.” Pettersson column 13 lines 51-57 disclose “Additional information about the construction and the work package is given as human readable instructions 10 or as code instructions, particularly by pictograms 11, e.g. indicating painting of the floor with a corresponding color, or by machine readable codes 12, e.g. representing guiding points or specific settings for an automated or semi-automated executing device.” The additional information in the form of human readable instructions or code instructions correspond with metadata of various points which are not marked by the tags.
Claim 10 further recites “10. The system of claim 4, wherein the augmented-reality device is a first augmented-reality device, and the system further comprises a second augmented-reality device configured to receive data of the building plan.” Singer column 6 lines 50-51 disclose “Augmented Reality (AR)-device according to the invention, i.e. AR-glasses 10 and an AR-helmet 11.” Singer column 6 lines 59-67 teach:
The display shown in the two examples of FIGS. 1a and 1b may comprise a projector (not shown) for projecting the AR-data onto the display 120/121. Other embodiments of the AR-device according to the invention are handheld devices such as smart phones or tablet computers. Such handhelds usually also comprise a visual sensor (camera), a computer (processor) and a display (screen) and are configured to display referenced AR-data.
Glasses, helmet, projector, smart phones, and tablet computers are each respective augmented-reality devices. Singer column 2 lines 1-3 teach “Augmented Reality (AR) systems are often utilised for supporting users on a site by referenced visualisations.” Plural users correspond with a recognition there may be more than one user and accordingly a second or more device.
Claim 11 recites “11. A method for construction layout using robotic marking and augmented-reality metadata.” Pettersson column 12 lines 43-48 disclose:
The mobile support system might be a ground based robotic vehicle or a flying robotic vehicle (UAV) and the markings might be based on at least one of a 2D marking, e.g. a painted marking, a laser burned marking, or written information, a 3D structure, e.g. by a surface spattering device.
A robotic vehicle is robotic.
Pettersson does not explicitly disclose augmented reality; however, in analogous art of building construction surveying information, Singer column 9 lines 20-21 teaches “the [Augmented-Reality (AR)]-data may be overlaid onto the real view of the scene which the user has when he is wearing the AR-device.”
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pettersson and Singer. One having ordinary skill in the art would have found motivation to use augmented reality into the system of construction site management with marking robot for the advantageous purpose of “to display virtual objects with a precise spatial link to the coordinate system, i.e. to the natural environment” and “improving the positioning accuracy of displayed AR-data on the display of an AR-device.” See Singer column 2 lines 10-12 and 41-43.
Claim 11 further recites “the method comprising: receiving data from a building plan, wherein the building plan includes a plurality of elements.” Pettersson column 12 lines 2-10 teach:
storing a general construction database 2, the database comprising structured datasets of a plurality of object entities of physical construction components, in particular wherein the datasets comprise digital construction plan information (e.g. 2D or 3D information, or CAD information) and attribute information of the object entities, a hierarchical structure of the object entities representing a desired construction result to be established by the physical construction components,
Claim 11 further recites “marking out points at a construction site related to the plurality of elements in the building plan using physical markings, wherein: marking out points is performed by a robotic device.” Pettersson column 12 lines 43-48 disclose:
The mobile support system might be a ground based robotic vehicle or a flying robotic vehicle (UAV) and the markings might be based on at least one of a 2D marking, e.g. a painted marking, a laser burned marking, or written information, a 3D structure, e.g. by a surface spattering device.
A robotic vehicle is robotic. Painting markings is marking out points using physical markings.
Claim 11 further recites “and points are marked at the construction site within a pre-established accuracy.” Pettersson column 13 lines 9-11 disclose "the mobile support system might also be adapted both for taking control measurements and for tagging markings at the same time, i.e. in one go.” Pettersson column 14 lines 25-26 disclose “regarding special requirements for the accuracy and small tolerance levels.” A tolerance level corresponds with a pre-established required accuracy.
Claim 11 further recites “transmitting data from the building plan to an augmented-reality device, wherein data from the building plan includes metadata about the points.” Pettersson column 13 lines 51-57 disclose “Additional information about the construction and the work package is given as human readable instructions 10 or as code instructions, particularly by pictograms 11, e.g. indicating painting of the floor with a corresponding color, or by machine readable codes 12, e.g. representing guiding points or specific settings for an automated or semi-automated executing device.” The additional information in the form of human readable instructions or code instructions correspond with metadata of various points.
Pettersson does not explicitly disclose augmented reality; however, in analogous art of building construction surveying information, Singer column 9 lines 20-21 teaches “the [Augmented-Reality (AR)]-data may be overlaid onto the real view of the scene which the user has when he is wearing the AR-device.”
Singer column 8 lines 57-62 teach:
Accordingly, the AR-glasses may store, or receive the AR-data from a server, i.e. the augmented information displayed for assisting the user. These AR-data are spatially related to the reference system, in particular to the BIM model. The BIM-model may optionally also be stored on the AR-device or retrieved from the server.
The Building Information Model (BIM) corresponds with building plan data.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pettersson and Singer. One having ordinary skill in the art would have found motivation to use augmented reality into the system of construction site management with marking robot for the advantageous purpose of “to display virtual objects with a precise spatial link to the coordinate system, i.e. to the natural environment” and “improving the positioning accuracy of displayed AR-data on the display of an AR-device.” See Singer column 2 lines 10-12 and 41-43.
Claim 11 further recites “aligning an internal map of the augmented-reality device to the construction site based on the physical markings and the building plan.” Singer column 9 lines 24-30 teach:
Alternatively to the act of referencing the AR-device relative to the reference system by means of the referencing marker, the process can also be understood in such a way that the AR-device does not really "lock" in into the reference system but merely determines a spatial relationship between the referencing marker and AR-data assigned to the referencing marker.
Determining a spatial relationship between a reference marker and AR data is a comparison between points in the internal map and data from the building plan.
Singer column 10 lines 4-8 teach “the projector 20 to project the marker accordingly on a surface adjacent to where the AR-data are linked, in particular wherein the AR-data and the projection of the referencing marker have the same position within the reference system or BIM-model.” The AR data and reference system or BIM model having linked same positions corresponds with the AR data being aligned with the building plan information. The reference marker and/or AR data correspond with the interna map.
Claim 11 further recites “and presenting metadata about the points to a user of the augmented-reality device as the user looks at the points at the construction site.” Pettersson does not explicitly disclose augmented reality; however, in analogous art of building construction surveying information, Singer column 9 lines 20-21 teaches “the [Augmented-Reality (AR)]-data may be overlaid onto the real view of the scene which the user has when he is wearing the AR-device.” The AR data being overlaid onto the real view of the scene is a presentation of the metadata to a user as the user looks at the construction site.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pettersson and Singer. One having ordinary skill in the art would have found motivation to use augmented reality into the system of construction site management with marking robot for the advantageous purpose of “to display virtual objects with a precise spatial link to the coordinate system, i.e. to the natural environment” and “improving the positioning accuracy of displayed AR-data on the display of an AR-device.” See Singer column 2 lines 10-12 and 41-43.
Claim 12 further recites “12. The method of claim 11, wherein aligning the internal map of the augmented-reality device to the construction site comprises comparing locations of points in the internal map to data from the building plan.” Singer column 9 lines 24-30 teach:
Alternatively to the act of referencing the AR-device relative to the reference system by means of the referencing marker, the process can also be understood in such a way that the AR-device does not really "lock" in into the reference system but merely determines a spatial relationship between the referencing marker and AR-data assigned to the referencing marker.
Determining a spatial relationship between a reference marker and AR data is a comparison between points in the internal map and data from the building plan.
Singer column 10 lines 4-8 teach “the projector 20 to project the marker accordingly on a surface adjacent to where the AR-data are linked, in particular wherein the AR-data and the projection of the referencing marker have the same position within the reference system or BIM-model.” The AR data and reference system or BIM model having linked same positions corresponds with the AR data being aligned with the building plan information. The reference marker and/or AR data correspond with the interna map.
Claim 13 further recites “13. The method of claim 11, wherein points are marked without marking metadata near points.” Pettersson column 13 lines 51-57 disclose “Additional information about the construction and the work package is given as human readable instructions 10 or as code instructions, particularly by pictograms 11, e.g. indicating painting of the floor with a corresponding color, or by machine readable codes 12, e.g. representing guiding points or specific settings for an automated or semi-automated executing device.” The additional information in the form of human readable instructions or code instructions correspond with metadata of various points which are not marked by the tags.
Claim 14 further recites “14. The method of claim 11, wherein transmitting data from the building plan to the augmented-reality device is performed using one or more wireless connections.” Singer column 7 lines 4-7 teach “the AR-device may comprise a wireless communication unit (using e.g. WiFi, Bluetooth, radio link, etc.) for at least one of: connecting to, communicating with, and transferring data from/to a server.”
Dependent claims 19 and 20 are substantially similar to claims 2 and 3 above and are rejected for the same reasons.
Dependent Claims 8 and 16
Claims 8 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Pattersson and Singer as applied to claims 4 and 11 above, and further in view of US patent 9,846,242 B2 Gil, et al. [herein “Gil”].
Claim 8 further recites “8. The system of claim 4, wherein the pre-established accuracy is equal to or less than 5 centimeters.” Pettersson column 13 lines 13-17 disclose “an internal construction site coordinate system.”
But Pettersson and Singer does not explicitly disclose accuracy better than 5 centimeters; however, in analogous art of construction site surveying, Gil column 4 lines 41-50 teach:
The fixed location antennas 20 may be placed at previously known landmarks or coordinates, via conventional surveying methods ….
The onboard GPS devices may be capable of determining location within an accuracy range fifteen feet to less than one inch.
Gil column 5 lines 3-5 teach “A portable antenna 26 may be utilized to report the location data 14 to the computing device 12 from the restricted antenna system 18.”
Gil column 7 lines 55-58 teach “Measurements from the RAS 18 are understood to be highly accurate, or at least more accurate than the measurements from the GPS module 16. In some embodiments, accuracy within ~7 cm is understood to be possible.”
Accuracies of approximately 7 cm is within the scope of similar and overlapping ranges to a 5cm accuracy. Accuracies of less than one inch correspond with accuracies less than 2.54 cm which is less than 5 cm. The RAS being more accurate than the measurements from the GPS module correspond with an accuracy range less than 2.54 cm of the GPS device taught at Gil column 5 line 50.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pettersson, Singer, and Gil. One having ordinary skill in the art would have found motivation to use location determination of less than one inch into the system of construction site management with marking robot because “RAS 18 is understood to be the most suitable for navigating the interior of buildings, as GPS signals typically cannot penetrate ceilings and walls.” Gil column 7 lines 59-61. This is an art recognized suitability for the intended purpose of providing location measurements at a construction site building interior. See MPEP §2144.07.
Dependent claim 16 is substantially similar to claim 8 above and is rejected for the same reasons.
Dependent Claims 15 and 18
Claims 15 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Pattersson and Singer as applied to claim 11 above, and further in view of Degani, A., et al. “An Automated System for Projection of Interior Construction Layouts” IEEE Transactions on Automation Science & Engineering, vol. 16, no. 4 (2019) [herein “Degani”].
Dependent claim 15 is substantially similar to claim 7 above and is rejected for the same reasons.
Claim 18 further recites “18. The method of claim 11, wherein at least one of the points is marked on a ceiling.” Pettersson does not explicitly teach marking points on a ceiling; however, in analogous art of managing construction layout information, Degani abstract teaches:
The system projects any desired information—drawings and images—onto the work surface (floor, walls, or ceiling) in the correct location, scale, and orientation. The prototype apparatus consists of a laser range scanner, a projector, and a camera. Projection of the work instructions directly onto the work surface is accurate and immediate.
Projecting information onto a ceiling work surface is marking points on a ceiling.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pettersson, Singer, and Degani. One having ordinary skill in the art would have found motivation to use marking at least 200 points into the system of construction site management with marking robot for the advantageous purpose because “[p]rojection of the work instructions directly onto the work surface is accurate and immediate.” See Degani abstract.
Dependent Claim 17
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Pattersson and Singer as applied to claim 11 above, and further in view of US patent 12,222,723 B2 Yamauchi [herein “Yamauchi”].
Claim 17 further recites “17. The method of claim 11, wherein the robotic device is a quadruped.” Pattersson does not explicitly disclose a quadruped robot; however, in analogous art of construction site robots, Yamauchi column 6 line 67 teaches “FIG. 1A depicts a quadruped robot with four legs.”
Yamauchi column 5 lines 62-66 teach “For instance, if the environment for the robot is a construction site, a construction site is a changing environment often including temporary or non-permanent objects, such as tools, tool storage, machinery, material, etc.” Yamauchi column 7 lines 4-6 teach “that provide a means to traverse the terrain within the environment 10.”
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Pattersson, Singer, and Yamauchi. One having ordinary skill in the art would have found motivation to use a quadruped robot with four legs into the system of construction site management with marking robot because quadruped robots have art recognized suitability for the intended purpose of navigating a construction site. See MPEP §2144.07 and Yamauchi column 5 lines 62-66 and column 7 lines 4-6.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jay B Hann whose telephone number is (571)272-3330. The examiner can normally be reached M-F 10am-7pm EDT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Renee Chavez can be reached at (571) 270-1104. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Jay Hann/Primary Examiner, Art Unit 2186 13 May 2026
Prosecution Insights