DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status
Claims 1-17 are pending. Claims 1, 7, and 17 are amended.
Claims 1-2, 5-6, 8-11, and 13-17 are rejected under 35 USC 102, and claims 3-4, 7, and 12 are rejected under 35 USC 103.
Response to Amendment
The rejections under 35 USC 112(b) are withdrawn.
The amendments are addressed with respect to prior art in the new grounds of rejection.
Response to Arguments
Applicant's arguments on pg. 6-8 of the Remarks filed 7/8/2026 have been fully considered but they are not persuasive.
On pg. 6-8 ¶ 1, the applicant argues that Zollmann does not teach retrieval and presentation of construction task status as in the instant specification; therefore, the claims should be allowed. The examiner respectfully disagrees. Although the instant specification gives examples of construction task status data in [0043], the description is not a narrowing “special definition” that is different from the plain meaning of the term (MPEP 2173.01). Also, instant specification paragraph [0211] provides a disclaimer that the embodiments disclosed are not limiting: “The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.” The term “construction task status” in the claim language is given the broadest reasonable interpretation in light of the specification (MPEP 2111). Based on that claim interpretation standard, the term “construction task status” includes the construction task information as shown in previously cited Zollmann.
On pg. 8 ¶ 2, the applicant argues that Zollmann does not disclose claim 6 which further describes the construction task status. The examiner respectfully disagrees. Under the broadest reasonable interpretation, only one of the listed alternatives must be taught. Therefore, the “construction task status of each element” may only include “construction status” as shown in the previous rejection. Previously cited Zollmann discloses a construction status by illustrating that an element requires replacement or repainting. For the sake of compact prosecution, Zollmann also discloses “at least one image relating to the respective element” or “element details” even if it is interpreted that Zollmann does not discloses “construction status”, which the examiner does not concede.
On pg. 8 ¶ 3, the applicant argues that the previous rejection of claim 1 under 35 USC 102 using Zollmann is improper because the elements of Zollmann are not arranged as claimed in the instant application. The examiner respectfully disagrees. The applicant correctly describes the standard for a rejection under 35 USC 102; see MPEP 2131“The elements must be arranged as required by the claim”. However, the cited elements of Zollmann disclose the arranged elements are required. In general, Zollmann pg. 140-141 describe the data gathering including the physical hardware and the conceptual data flow. Then Zollmann pg. 142-145 describe combining data sources for analysis, and Zollmann pg. 146-152 describe user interface and data display techniques. Aspects of each section are relevant to respective portions of the claim.
The arguments directed to the unamended claim limitations are not persuasive; therefore, the rejection under 35 USC 102 is unchanged for the unamended portions of the claims.
Applicant's arguments on pg. 9-10 of the Remarks filed 7/8/2026 have been fully considered but they are moot as they are directed to the claim amendments which have not been previously considered. The claim amendments are addressed in the new grounds of rejection.
The dependent claims are argued to be allowable because the independent claims are supposedly allowable. However, the independent claims are not allowable; therefore, the dependent claims are not allowable since they do not add any further allowable limitations.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 5-6, 8-11, and 13-17 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zollmann et al. “Augmented Reality for Construction Site Monitoring and Documentation” 2014.
Regarding claim 1, Zollmann discloses a method of generating an interactive graphical user interface (GUI), comprising: (Abstract “In this paper, we will describe how to use AR to support monitoring and documentation of construction site progress.”)
receiving at least one image captured by an image sensor of a mobile device in a construction site (pg. 140 Col. 1 ¶ 1 “For this purpose, we use an aerial client that flies over the area of interest on a regular basis. During such flight sessions, the aerial vehicle captures a set of aerial images from meaningful positions.”), the at least one image is associated with at least one positioning parameter indicative of a position of the mobile device when the at least one image is captured (pg. 141 Col. 1 ¶ 3 “Additionally, we include available GPS information from the MAV into the reconstruction workflow to reduce computation time and to obtain a georeferenced 3-D model at a metric scale, as proposed by Irschara et al. [17].”);
rendering the at least one image on a display (pg. 141 Col. 2 ¶ 1 “For overlaying the captured progress information onto the user’s view of the physical outdoor environment, we need a mobile AR client that is appropriate for working in outdoor environments and integrates all sensors that are required for video capturing and for achieving an adequate registration.”);
receiving user input indicating selection of an area in the at least one image depicting a corresponding area in the construction site (pg. 148 Col. 1 ¶ 3 “Such a 3-D filtering is particularly interesting for the visualization of 3-D data that were reconstructed with aerial vision, since the users may want to inspect one specific element that is occluded by other previous structures in their view. By defining a focus area in 3-D, it is possible to exclusively visualize information for this selected region.”);
accessing a 3D model documenting a respective construction task status relating to each of a plurality of elements in the construction site (pg. 142 Col. 2 ¶ 3 “When inspecting the progress of a construction site, it is often helpful to not only visualize the as-built status, as is provided by the 3-D reconstruction, but also to render information about existing surrounding structures, as given by GIS data and to compare the current or previous situations with information provided by BIMs. BIMs describe systems that combine different information about the lifecycle of a building such as construction plans, but also plans for building management.”);
registering the at least one image to the 3D model according to the at least one positioning parameter to identify, in the selected area, at least one element documented in the 3D model (pg. 142 Col. 2 ¶ 3 “While the concept of BIMs aims to provide 3-D as well as 4-D information for construction sites that could be directly used for visualization in AR, in reality, many companies still work with 2-D CAD plans. In this case, we have to apply a data conversion step that transcodes the 2-D information into 3-D models that can be used for visualization purposes.”);
retrieving a respective construction task status relating to the at least one identified element (Fig. 22 “Replace” “Repaint”, Section VII(B) Annotations, users may create, store, or retrieve status messages such as notes that are annotated onto the model.); and
adapting a GUI presented in association with the selected area (Fig. 22 The user view is based on the perspective of the user. pg. 148 Col. 1 ¶ 3, The user can define a focus area.) by presenting at least one or more textual items (Fig. 22 “Replace” “Repaint”, The model view includes the annotations of the incomplete steps based on the user selected perspective area,). and one or more visual interface features (Fig. 22 “4.32998” The model view includes a visual interface indication of a measurement. Both the left and right images also include visual points to indicate a position in the model.) generated based on the retrieved construction task status so as to visually represent the retrieved construction task status (Fig. 22 “Replace” “Repaint”, The model view includes the annotations of the incomplete steps. To display the overlay, the data must be retrieved and sent to the display, for example as generally shown in Fig. 2.).
Regarding claim 2, Zollmann discloses the method of claim 1, and Zollmann discloses further comprising updating the respective construction task status in the 3D model according to user input received via the GUI (Section VII(B) Annotations, The user may add annotations to reflect the current status.).
Regarding claim 5, Zollmann discloses the method of claim 1, and Zollmann discloses wherein each of the plurality of elements is a member of a group consisting of: a decorative element (Under the broadest reasonable interpretation, only one of the listed alternatives must be taught. Fig. 22 The annotations describe decorative elements.).
Regarding claim 6, Zollmann discloses the method of claim 1, and Zollmann discloses wherein the construction task status of each element comprises at least one member of a group consisting of: construction status (Under the broadest reasonable interpretation, only one of the listed alternatives must be taught. Fig. 22 The annotations describe the status of the elements.).
Regarding claim 8, Zollmann discloses the method of claim 1, and Zollmann discloses wherein the 3D model comprises a building information model (BIM) (pg. 142 Col. 2 ¶ 3 “While the concept of BIMs aims to provide 3-D as well as 4-D information for construction sites that could be directly used for visualization in AR, in reality, many companies still work with 2-D CAD plans. In this case, we have to apply a data conversion step that transcodes the 2-D information into 3-D models that can be used for visualization purposes.” Although the original BIM model is 2D, it is converted to 3D.).
Regarding claim 9, Zollmann discloses the method of claim 1, and Zollmann discloses further comprising rendering at least one augmented reality (AR) image on the display (Abstract “Augmented reality (AR) allows for an on-site presentation of information that is registered to the physical environment.”), the at least one AR image comprises at least one computer generated object merged into the at least one image (Fig. 2 The computer generated objects are overlaid on the image.).
Regarding claim 10, Zollmann discloses the method of claim 1, and Zollmann discloses wherein the at least one positioning parameter is derived from at least one intrinsic parameter of the image sensor (Under the broadest reasonable interpretation, only one of the listed alternatives must be taught. pg. 141 Col. 1 ¶ 1 “During one flight session the aerial client is able to create image sets of approximately 200–300 high resolution and highly overlapping images. These images are published over the network and used by the reconstruction client as input for 3-D reconstruction.” The overlapping of the images is intrinsic to the data, and the position of the camara poses is derived from the images.).
Regarding claim 11, Zollmann discloses the method of claim 1, and Zollmann discloses wherein the at least one positioning parameter is computed by at least one device deployed in the construction site which is configured to compute the at least one positioning parameter based on the position of the mobile device (pg. 144 Col 1 ¶ 3 “In order to achieve such a highly accurate position and orientation estimate of the AR system in outdoor environments, we combine the measurements from different sensors: • L1/L2 RTK GPS; • IMU; • vision-based panoramic tracker.” The position is computed using various sensors on the mobile device.).
Regarding claim 13, Zollmann discloses the method of claim 1, and Zollmann discloses wherein the registering is based on matching at least one common feature identified in the at least one image and in the 3D model oriented with respect to each other according to the at least one positioning parameter (pg. 144 Col. 2 ¶ 1 “The panorama tracker is based on feature detection and matching.” “1) determine the pose of a new frame in relation to already mapped data; and 2) add features from this newly localized frame to the existing map that is then used again for further pose estimation” The already mapped data is an existing 3D model, and the feature detection attempts to match the new data with the existing model.).
Regarding claim 14, Zollmann discloses the method of claim 1, and Zollmann discloses wherein the at least one image comprises at least one frame extracted from a video stream captured by the image sensor (pg. 145 Col. 1 ¶ 2 “III. On the AR client-site, either a visual panorama tracker or a model-based 6-DOF tracker receives absolute pose measurements and uses the video stream as tracking input.” Pg. 145 Col. 1 ¶ 3 “We compute the georeferenced absolute pose of this panoramic representation by sending the initial keyframe to the reconstruction client (Fig. 9).”).
Regarding claim 15, Zollmann discloses the method of claim 1, and Zollmann discloses wherein the display is a member of a group consisting of: a 2D display (Under the broadest reasonable interpretation, only one of the listed alternatives must be taught. pg. 141 Col. 1 ¶ 3 “This tablet PC provides a screen that is specially built to be viewable outdoors, even under sunlight conditions.”).
Regarding claim 16, Zollmann discloses the method of claim 1, and Zollmann discloses wherein the image sensor is a member of a group consisting of: a camera (Under the broadest reasonable interpretation, only one of the listed alternatives must be taught. pg. 140 Col. 1 ¶ 3 – Col. 2 ¶ 1 “For our system, we equipped an AscTec Faclon 8 Octocopter with a standard consumer digital camera.”)
Regarding claim 17, Zollmann discloses a system for generating an interactive graphical user interface (GUI), comprising: (Abstract “In this paper, we will describe how to use AR to support monitoring and documentation of construction site progress.”) at least one processor configured to execute a code, the code comprising: (pg. 140 Col. 1 ¶ 2 “For a high level of flexibility and reliable communication between the clients, we use the robot operating system (ROS1) for exchanging data (Fig. 2) over the network. Each client has to registered itself within the system to be able to receive the relevant data.”). The remainder of the claim is rejected in the same way as claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over Zollmann et al. “Augmented Reality for Construction Site Monitoring and Documentation” 2014 in view of Fosburgh et al. (US 2014/0316837 A1).
Regarding claim 3, Zollmann discloses the method of claim 1, but Zollmann does not disclose further comprising
adapting a plurality of GUIs presented in association with the selected area depicted in a plurality of images captured by a plurality of image sensors of a plurality of mobile devices,
each of the plurality of GUIs is adapted according to the respective task status retrieved from the 3D model for the at least one identified element.
Fosburgh teaches adapting a plurality of GUIs presented in association with the selected area depicted in a plurality of images captured by a plurality of image sensors of a plurality of mobile devices ([0016] “This data can then be sent in the form of reports 150 to various assets in the field in real-time to change the parameters of the task to reflect current conditions. It is noted that reports 150 can be sent to various devices such as laptop computers, tablet computers, personal digital assistants, or other display devices to keep personnel at the construction site appraised as to the progress of the project.” Fig. 4 “Image Capture Device” The multiple devices receive a report based on the asset devices data which may include image capture device data.),
each of the plurality of GUIs is adapted according to the respective task status retrieved from the 3D model for the at least one identified element ([0016] “This data can then be sent in the form of reports 150 to various assets in the field in real-time to change the parameters of the task to reflect current conditions.” [0043] “In operation 740 of FIG. 7, progress on the construction site design is dynamically updated in real-time by the construction management computing system 101 based on the geospatial information to create a real-time as constructed model of the construction project.” “created as a two-dimensional model of the construction site, a three dimensional model, or a multi-dimensional model of the construction project which is modeled in more than three dimensions.” A 3D model reflects the real-time status of the construction site elements.).
Zollmann and Fosburgh are analogous because they are from the “same field of endeavor” construction management.
Before the effective filing date of the claimed invention, it would have been obvious to one of the ordinary skill in the art, having the teachings of Zollmann and Fosburgh before him or her, to modify Zollmann to include multiple devices as taught by Fosburgh.
The suggestion/motivation for doing so would have been Fosburgh [0019] “Thus, as motorized assets 120 move around construction site 200, they are constantly generating data providing an instantaneous view of the current terrain conformation which can be used to continuously update real-time as-constructed model 108.”
Regarding claim 4, Zollmann discloses the method of claim 1, but Zollmann does not disclose further comprising adapting a plurality of GUIs presented in association with a plurality of selected areas in the construction site depicted in a plurality of images captured by a plurality of image sensors of a plurality of mobile devices,
each of the plurality of GUIs is adapted according to a respective task status retrieved from the 3D model for at least one of the plurality of elements identified in a respective selected area.
Fosburgh teaches further comprising adapting a plurality of GUIs presented in association with a plurality of selected areas in the construction site depicted in a plurality of images captured by a plurality of image sensors of a plurality of mobile devices ([0016] “This data can then be sent in the form of reports 150 to various assets in the field in real-time to change the parameters of the task to reflect current conditions. It is noted that reports 150 can be sent to various devices such as laptop computers, tablet computers, personal digital assistants, or other display devices to keep personnel at the construction site appraised as to the progress of the project.” Fig. 4 “Image Capture Device” The multiple devices receive a report based on the asset devices data which may include image capture device data.),
each of the plurality of GUIs is adapted according to a respective task status retrieved from the 3D model for at least one of the plurality of elements identified in a respective selected area ([0016] “This data can then be sent in the form of reports 150 to various assets in the field in real-time to change the parameters of the task to reflect current conditions.” [0043] “created as a two-dimensional model of the construction site, a three-dimensional model, or a multi-dimensional model of the construction project which is modeled in more than three dimensions.”).
Zollmann and Fosburgh are analogous because they are from the “same field of endeavor” construction management.
Before the effective filing date of the claimed invention, it would have been obvious to one of the ordinary skill in the art, having the teachings of Zollmann and Fosburgh before him or her, to modify Zollmann to include multiple devices as taught by Fosburgh.
The suggestion/motivation for doing so would have been Fosburgh [0019] “Thus, as motorized assets 120 move around construction site 200, they are constantly generating data providing an instantaneous view of the current terrain conformation which can be used to continuously update real-time as-constructed model 108.”
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Zollmann et al. “Augmented Reality for Construction Site Monitoring and Documentation” 2014 in view of Gong et al. (CN 113762782 A).
Regarding claim 7, Zollmann discloses the method of claim 1, but Zollmann does not disclose wherein the construction task status of at least one relating to at least one of the plurality of elements is created according to at least one template.
Gong teaches wherein the construction task status of at least one relating to at least one of the plurality of elements is created according to at least one template (Specific Implementation Examples ¶ 4 “the director can guide the task template information through the main control device. wherein the task template can be a fixed format, and comprises specific checking content and operation guidance basic content, such as " template name ", " task checking item ", " operation instruction ", " picture example ", " video guide ", " voice guiding " and other content.” Specific implementation examples ¶ 17 “task template unit: for defining task decomposition step, checking item, example picture and so on, supporting batch introduction;” The inspection status is based on a template.).
Zollmann and Gong are analogous because they are from the “same field of endeavor” construction management.
Before the effective filing date of the claimed invention, it would have been obvious to one of the ordinary skill in the art, having the teachings of Zollmann and Gong before him or her, to modify Zollmann to include templates as taught by Gong.
The suggestion/motivation for doing so would have been Gong Contents of the Invention “Based on this, in order to solve the above technical problem, providing a construction engineering supervision field efficiency improving method, device and mobile terminal and storage medium, which can improve the efficiency of the construction engineering field supervision work”
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Zollmann et al. “Augmented Reality for Construction Site Monitoring and Documentation” 2014 in view of Yuan et al. (CN 110619258 A).
Regarding claim 12, Zollmann discloses the method of claim 1, but Zollmann does not wherein the registering is based on a translation vector computed based on the at least one positioning parameter.
Yuan wherein the registering is based on a translation vector computed based on the at least one positioning parameter (Abstract “utilizing an equal interval resampling and translation vector method,” Disclosure of Invention ¶ 7 “the similarity comparison of the road curve utilizes an equidistant resampling and translation vector method to carry out similarity comparison on the extracted road center line and the road network vector unit to be checked;”).
Zollmann and Yuan are analogous because they are from the “same field of endeavor” inspection modelling.
Before the effective filing date of the claimed invention, it would have been obvious to one of the ordinary skill in the art, having the teachings of Zollmann and Yuan before him or her, to modify Zollmann to include a translation vector as taught by Yuan.
The suggestion/motivation for doing so would have been Yuan Abstract “In the image blocks, road area extraction is carried out by using spectrum and shape features, a road center line set is obtained through vectorization, spatial analysis, simplification and other methods after refinement, and similarity comparison is performed on the extracted road center line and the to-be-checked road track by utilizing an equal interval resampling and translation vector method, and marking different route positions of the track, thereby obtaining a road track checking result.“ “Compared with a traditional method adopting manual field investigation and remote sensing manual eye interpretation, the workload is greatly reduced, and the efficiency is improved.”
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TROY A MAUST whose telephone number is (571)272-1931. The examiner can normally be reached on Monday-Friday from 8AM to 4PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rehana Perveen, can be reached at telephone number (571) 272-3676. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/T.A.M./Examiner, Art Unit 2189
/REHANA PERVEEN/Supervisory Patent Examiner, Art Unit 2189
Prosecution Insights