Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-12, 17-23 are rejected under 35 U.S.C. 103 as being unpatentable over Ahmed et al. (US Pub 2025/0086830 A1) in view of Murphy et al. (US Pub 2022/0292421 A1).
As to claim 1, Ahmed discloses a method for accessing a 2D drawing in a document retrieval system from a work site (Ahmed, Fig. 1B, ¶0002, “The BIM model is then sent to a construction site, most commonly in the form of two-dimensional (2D) drawings” ¶003, “Often the crew at a construction site need to correctly interpret and work from a set of 2D drawings created from the BIM”), comprising the steps of:
configuring a reference device at the work site to provide a geographical reference to a mobile device at the remote work site (Ahmed, Fig 1A, ¶0044-0045, “a positioning system 100 that is set up at the location 1.”);
receiving an annotated document from the mobile device by a document retrieval system in communication with a document repository configured to store a plurality of two dimensional (2D) drawings (Ahmed, ¶0021, “use a camera coupled to the headset to capture an image of one or more two-dimensional (2D) markers.” ¶0063, “FIG. 3B shows how the example 300 may be located within a construction site. In FIG. 3B the construction site is an internal room 305, but in other examples the construction site may be an exterior area. In general, the 2D markers may be placed within any area of the construction site.” ¶0083, “FIG. 5A shows at least a portion of a BIM 510 that represents a physical area of the construction site.”” The BIM may have multiple layers, e.g. representing different parts of construction or services, as is known in the art. A dashed line is used to indicate that what is shown is a modelled representation of the physical area, e.g. a Computer Aided Design-CAD-representation, rather than the physical area itself. The BIM 510 is defined with reference to a coordinate system for the BIM 512.” ¶0107, “a captured view of one or more of the surveying equipment markers and the two-dimensional marker using the camera, e.g. camera 520 in FIG. 5B, may be used to “bind” the measured locations of the surveying equipment markers, as obtained using the surveying equipment, to the two-dimensional marker such that the measured locations and the detected two-dimensional marker are usable to derive a BIM-to-positioning-system transformation as described herein.”¶0109, “The present method adds the placing and measurement of 2D markers as part of this existing surveyance. For example, once a total station is set up in a space, making multiple measurements of additional control marker is relatively quick (e.g., on the order of seconds). These 2D markers are then usable for rapid configuration of a headset for displaying the BIM during subsequent construction, such as interior construction where accurate placement of finishes is desired.”);
accessing the 2D drawing in the document repository by the document retrieval system based on the annotated document (Ahmed, ¶0003, “Often the crew at a construction site need to correctly interpret and work from a set of 2D drawings created from the BIM”. ¶0082, “Objects within the BIM, or more precisely within the coordinate system used by the BIM, such as the locations of structures, points and other geometric features, need to be mapped to the coordinate system of the headset positioning system such that these objects may be correctly aligned within that latter coordinate system and then projected onto the display devices 255a, 255b of the augmented reality glasses 250.” ; and
providing a Ahmed, ¶0001, “allows a user wearing the headset to view a virtual image of the BIM aligned with the view of the construction site from the headset.” ¶0002, “A builder then uses the drawings and/or BIM model in conjunction with the marks (“Set Out marks”) made by the engineer to erect the structure according to the drawings or model in the correct place” ¶0082, “Objects within the BIM, or more precisely within the coordinate system used by the BIM, such as the locations of structures, points and other geometric features, need to be mapped to the coordinate system of the headset positioning system such that these objects may be correctly aligned within that latter coordinate system and then projected onto the display devices 255a, 255b of the augmented reality glasses 250.” ¶0083, “The BIM may have multiple layers, e.g. representing different parts of construction or services”),
Ahmed does not disclose “a rasterized version of the 2D drawing” and “wherein the work site is remotely located from the document retrieval system”.
Murphy teaches “a rasterized version of the 2D drawing” (Murphy, ¶0013, “representing the portion of a two dimensional representation as a raster image” ¶0040, “The rasterized file will be rendered suitable for a controller hosting an artificial intelligence engine (“AI engine”) to process, the AI engine may function best with a particular image size or range of image size and may include steps to scale input images to a pixel count range in order to achieve a desired result.” ¶0045, “a combination of a vector graphic two-dimensional references such as floor plans and associated raster graphic version of the two-dimensional references; raster graphic patterns associated with features; and a determination of boundaries may be automatically or manually derived. An exemplary AI-processed two-dimensional reference that includes a floorplan 210, with boundaries 211 predicted, is shown in FIG. 2B, based on the floorplan of FIG. 2A.” ¶0047, “the AI engine may process a raster file that is converted for output as an image file of a floorplan 210 as illustrated in FIG. 2B.” ¶0177, “determining whether one or both of the first and the second two-dimensional representation received into the controller includes a vector image, and if one of the first and the second two-dimensional representation received into the controller includes a vector image converting at least a portion of the vector image into a raster image.”)
and “wherein the work site is remotely located from the document retrieval system” (Murphy, ¶0060, “The two dimensional reference 121 may be electronically provided to a controller 123 running an AI engine. The controller 123 may include, for example, one or more of: a cloud server, an onsite server, a network server or other computing device, capable of running executable software and thereby activate the AI engine.” ¶0129, ¶0131, “The controller 800 includes a processor unit 802, such as one or more semiconductor-based processors, coupled to a communication device 801 configured to communicate via a communication network (not shown in FIG. 8). The communication device 801 may be used to communicate, for example, with one or more online devices, such as a personal computer, laptop, or a handheld device.” ¶0140.).
Ahmed and Murphy are considered to be analogous art because all pertain to building information. It would have been obvious before the effective filing date of the claimed invention to have modified Ahmed with the features of “a rasterized version of the 2D drawing” and “wherein the work site is remotely located from the document retrieval system” as taught by Murphy. The claim would have been obvious because the substitution of one known element for another would have yielded predictable results to one of ordinary skill in the art at the time of the invention.
As to claim 2, claim 1 is incorporated and the combination of Ahmed and Murphy discloses the step of the document retrieval system receiving a current task completion status from the mobile device (Ahmed, ¶0003,” once a task has been completed at a construction site, it is generally necessary to validate the task or check it has been done at the correct location.” Murphy, ¶0012, “identify aspects of a building described by a two dimensional representation and specify quantities for variables used to generate a bid or other proposal for completion of a project (or some subset of a project) represented by the two dimensional representation.” ¶0027, “The interactive interface is operative to generate values of variables useful for completion of a project and/or task.” ¶0063, “Each of the items in the continuum of work may be stored and subsequently used for an accounting.”).
As to claim 3, claim 1 is incorporated and the combination of Ahmed and Murphy discloses a first group of a planning, management, renovation, audit, building, maintenance, and demolition project from the second group of architecture, engineering, and construction is located at the work site (Murphy, ¶0063, “a second user interface 125 rendition, and submitted to AI analysis to track change orders and calculate costs, expenses, man hours or other variable associated with a change order. Change order renditions may also be time and/or date stamped to create a continuum of work, change orders, bid submissions related to original projects and change order and other related functionalities. Each of the items in the continuum of work may be stored and subsequently used for an accounting.” ¶0083, “a controller may be provided with two dimensional references that include a series of architectural drawings with disparate drawings representing different elevations within a structure. A three dimensional model may be effectively built based upon a sequenced stacking of the disparate drawings representing different levels of elevations. In other examples, the series of drawings may include cross sectional representation as well as elevation representation. A cross section drawing, for example, may be used to infer a common three-dimensional nature that can be attributed to the features, boundaries and areas that are extracted by the processes discussed herein. Elevation drawings may also present a structure in a three-dimensional perspective.” ¶0083, “a preliminary, or a final model output results may be displayed as a user interface in an integrated fashion in relation to a replication of the two-dimensional reference (such as an input architectural floor plan or technical drawing). The user interface may also be shown in a form that includes user modifiable elements, such as, but not limited to: polylines, polygons, arcs, circles, ellipses, splines, line segments, icons, points and other drawing features or combinations of lines and other elements.”).
As to claim 4, claim 1 is incorporated and the combination of Ahmed and Murphy discloses the annotated document comprises annotation metadata comprising location data received from the reference device (Ahmed, Fig 1A, ¶0044-0045, “a positioning system 100 that is set up at the location 1.” Fig. 3A-3D, ¶0077-0078, “the set of sensor devices may relate to a positioning system and the camera may be used for calibration of that positioning system.”).
As to claim 5, claim 1 is incorporated and the combination of Ahmed and Murphy discloses the annotated document comprises a photograph (Ahmed, ¶0079, “the two-dimensional marker may be detected in image data such as a captured image and/or one or more frames of captured video, and then undistorted based on known camera or image capture device parameters, such as intrinsic camera parameters.”).
As to claim 6, claim 4 is incorporated and the combination of Ahmed and Murphy discloses the step of annotating the 2D drawing according to the annotated document (Ahmed, ¶0008, “a surveyor, engineer or construction worker, wants to quickly activate a BIM augmented reality overlay to check the construction.” ¶0109, “The present method adds the placing and measurement of 2D markers as part of this existing surveyance. For example, once a total station is set up in a space, making multiple measurements of additional control marker is relatively quick (e.g., on the order of seconds). These 2D markers are then usable for rapid configuration of a headset for displaying the BIM during subsequent construction, such as interior construction where accurate placement of finishes is desired.” Murphy, ¶0083, “a preliminary, or a final model output results may be displayed as a user interface in an integrated fashion in relation to a replication of the two-dimensional reference (such as an input architectural floor plan or technical drawing). The user interface may also be shown in a form that includes user modifiable elements, such as, but not limited to: polylines, polygons, arcs, circles, ellipses, splines, line segments, icons, points and other drawing features or combinations of lines and other elements.”).
As to claim 7, claim 6 is incorporated and the combination of Ahmed and Murphy discloses the step of storing the annotated 2D drawing and/or the annotated submitted drawing by the document retrieval system (Murphy, ¶0133, “The storage device 803 can additionally store related data in a database 805. The processor and storage devices may access an AI training component 806 and database, as needed which may also include storage of machine learned models 807.”).
As to claim 8, claim 1 is incorporated and the combination of Ahmed and Murphy discloses the annotated document indicates a discrepancy between the accessed 2D drawing and a corresponding feature represented by the accessed 2D drawing at the work site (Ahmed, ¶0002, “The BIM model is then sent to a construction site, most commonly in the form of two-dimensional (2D) drawings or, in some cases, as a 3D model on a computing device. An engineer, using a conventional stake out/set out device, establishes control points at known locations in the real-world coordinates on the site and uses the control points as a reference to mark out the location where each structure in the 2D drawings or BIM model is to be constructed. A builder then uses the drawings and/or BIM model in conjunction with the marks (“Set Out marks”) made by the engineer to erect the structure according to the drawings or model in the correct place. Finally, an engineer must validate the structure or task carried out. This can be performed using a 3D laser scanner to capture a point-cloud from which a 3D model of the “as built” structure can be derived automatically. The “as built” model is then manually compared to the original BIM model.” ¶0008, “a user, such as a surveyor, engineer or construction worker, wants to quickly activate a BIM augmented reality overlay to check the construction.” Murphy, ¶0003, “contractors and sub-contractors must rely upon two dimensional documents, such as blueprints or floorplans to assess what is involved in a change order. The two-dimensional documents include details of what a building should include upon completion of construction of the building. Sometimes a change order is to address a deficiency in a design plan and other times a change order reflects a proposed modification by a party involved, such as a building owner, a designer, an occupant or other person with an interest in how the building is constructed.”).
As to claim 9, claim 1 is incorporated and the combination of Ahmed and Murphy discloses the 2D drawing comprises identifying design and trade metadata (Murphy, ¶0013, “generating a user interface including at least some of the multiple components; arranging the components in the user interface to form boundaries; generating at least one of: an area of a feature based upon the formed boundaries, and a length of a feature based upon a formed boundaries; based upon one or more of: the components included in the two dimensional representation, the area of a feature, and the formed boundaries, designating a quantity of an item to be included in the construction of the building” ¶0058, “generating a take-off, generating a bid, tracking change orders, and the like.”).
As to claim 10, claim 9 is incorporated and the combination of Ahmed and Murphy discloses the identifying design and trade metadata comprises a field selected from the group of a trade field, a sheet number, a sheet name, and a computer assisted drafting (CAD) stage (Murphy, ¶0058, “generating a take-off, generating a bid, tracking change orders, and the like.” ¶0034, “design firms and others who generate engineering designs for buildings may be reluctant to share raw CAD files with others, the present invention provides a solution that does not require raw CAD files.”).
As to claim 11, claim 1 is incorporated and the combination of Ahmed and Murphy discloses the 2D drawing is a vector drawing (Murphy, ¶0033, “rendering an image as a vector diagram, such as via a scan of paper-based initial drawings” ¶0035, “a drawing or other two dimensional representation may be stored in paper format or digital version or may not exist or may never have existed. The input may also be in any raster graphics image or vector image format.”).
As to claim 12, claim 1 is incorporated and the combination of Ahmed and Murphy discloses the reference device comprises an indoor positioning system (IPS) device (Ahmed, Fig. 1A, ¶0044-0046.).
As to claim 17, the combination of Ahmed and Murphy discloses a system for accessing to a remotely located two dimensional (2D) drawing via a mobile device at a work site, comprising: a reference device configured to provide a geographical reference to the mobile device; a document repository configured to store a plurality of 2D drawings; a document retrieval system configured to access the plurality of 2D drawings in the document repository, wherein the mobile device is configured to provide an annotated document to the document retrieval system, the document retrieval system is configured to access the 2D drawing in the document repository based on the annotated document, and the document retrieval system is configured to provide a rasterized version of the 2D drawing to the mobile device (See claim 1 for detailed analysis.).
As to claim 18, claim 17 is incorporated and the combination of Ahmed and Murphy discloses the annotated document comprises annotation metadata comprising location data received from the reference device (See claim 4 for detailed analysis.).
As to claim 19, claim 17 is incorporated and the combination of Ahmed and Murphy discloses the annotated document comprises a photograph (See claim 5 for detailed analysis.).
As to claim 20, claim 17 is incorporated and the combination of Ahmed and Murphy discloses the 2D drawing comprises identifying design and trade metadata (See claim 9 for detailed analysis.).
As to claim 21, claim 17 is incorporated and the combination of Ahmed and Murphy discloses the identifying design and trade metadata comprises a field selected from the group of a trade field, a sheet number, a sheet name, and a computer assisted drafting (CAD) stage (See claim 10 for detailed analysis.).
As to claim 22, claim 17 is incorporated and the combination of Ahmed and Murphy discloses the 2D drawing is a vector drawing (See claim 11 for detailed analysis.).
As to claim 23, claim 17 is incorporated and the combination of Ahmed and Murphy discloses the reference device comprises an indoor positioning system (IPS) device (See claim 12 for detailed analysis.).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Sasson et al. (US Pub 2019/0180433 A1) teaches annotation of construction site images.
Kincart et al. (US Pub 2024/0420432 A1) teaches an Agent is located in relation to an item of equipment or a sensor and presents digital content appropriate for a given set of circumstances and purpose to be served related to the item of equipment or sensor.
Gustafson (US Patent 11,386,364 B1) teaches enabling the performance of work tasks conducted at a remote project site like a construction site.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YU CHEN whose telephone number is (571)270-7951. The examiner can normally be reached on M-F 8-5 PST Mid-day flex.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached on 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YU CHEN/Primary Examiner, Art Unit 2613
Prosecution Insights