DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Responsive to the communication dated 9/22/2026.
Claims 1 – 20 are presented for examination.
Priority
ADS dated 8/31/2023 does not claim any domestic or international priority.
The effective filing date is 8/31/2023.
Information Disclosure Statement
IDS dated 9/22/2026 has been reviewed. See attached.
Drawings
The drawings dated 8/31/2023 have been reviewed. They are accepted.
Specification
The abstract dated 8/31/2023 has 99 words, 7 lines, and no legal phraseology. The abstract is accepted.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 7, 17 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “short” in claim 7 is a relative term which renders the claim indefinite. The term “short” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 11, 4,14, 5, 15, 6, 16, 8, 18, 10, 20, 9, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Glunz_2017 (US 2017/0132568 A1) in view of Gausebeck_2019 (US 2019/0026956 A1) in view of Waichberg_2022 (US 2022/0301305 A1).
Claim 1. Glunz_2017 makes obvious “A system for capture of physical object data for environment modeling (abstract: “a method and system for… model and site interaction for Building Information Modeling (BIM)… collaboration information for actual physical objects at physical locations is automatically collected and associated with virtual objects in virtual object models… physically added to at a project site… XD (e.g., 3D, or lower or higher dimensional)… the actual physical objects at the actual physical locations…”; FIG. 1, FIG. 2), comprising: A display of a physical space for which a [model/architectural drawings] is to be generated (par 2: “… generation and management of digital representation of physical and functional characteristics of physical spaces…”;par 63: “FIG. 1 is a block diagram illustrating an exemplary electronic information display system…”; par 85: “FIG. 2 is a block diagram illustrating an exemplary electronic information display system… with application 30 presents a graphical user interface (GUI)…”; par 246 – 247: “A 4D BIM is a term widely used in the CAD industry… i.s., 4D is 3D + schedule… the construction of the 4D models enables… display the process of construction activities…”; par 326: “… creates a new set of architectural drawings…”; par 396: “… creates a new set of architectural drawings…”; par 410: “… collaboration message including the new set of architectural drawings…”; FIG. 24); A mobile computing device configured to perform the following (FIG. 1: mobile tablet 12 and mobile phone 14 both ablet to execute program 30; FIG. 2: mobile tablet 12’ executing program 30; par 64: “… network devices 12, 14, 16 include, but are not limited to… laptop computers, tablet computers… mobile phones… smart phones… portable game consoles… wearable network devices…”): Receive instructions to make an object (par 85: “… application 30 presents a graphical user interface (GUI)…”; par 224: “… if a user desires a 3D model of a pre-hung door… such a 3D model could be created…”; par 250: “… software for architects, structural engineers, MEP engineers, designers and contractors… allows users to design a building and structure and its components in 3D, annotate the model with 2D drafting elements…”),and annotate the [model/architectural drawings] with a marker based on the instructions by placing the marker at a location at which an object is to be identified (FIG. 17, 20, 24, 26; Par 250: “… allows users to design a building and structure and its components in 3D, annotate the model…”); A server comprising a central processing unit, memory, an input port configured to receive thedata and annotations from the mobile computing device, and an output port, wherein the central processing unit is configured to: Generate a [model/architectural drawings] of the space based on the data ( FIG. 2 cloud computing network 18 with application 30’; Par 329: “… server network device 24 sends via the communications network 18 to the target application 30, 30’, 30” on the target network device 12, 14, 16, 20, 22, 26 the created set architectural drawings 43…”; Par 245: “… application 31 converts the created preliminary 3D object model into a final type of 3D modeling object…”; Par 299: “… the method and system allow new, 2D, 3D and higher dimensional models to be created for existing 3D modeling programs…”; par 326: “… creates a new set of architectural drawings…”; par 396: “… creates a new set of architectural drawings…”; par 410: “… collaboration message including the new set of architectural drawings…”); and place the marker in the [model/architectural drawings] (par 350: “… dashed lined 250 to give a visual indication that collaboration information 252 is available… electronic link 250’… for a selected portion 214, 248 of a desired 3D object model…”; par 392: “… one or more visual display characteristics such as color, font, type of line from solid to dashed, from non-blinking to blinking, to a new shape view, etc.)… is available for the selected portion 248 of the desired 3D object model 246… thereby providing visual indication…”; FIG. 20, 24, 26 ).
While Glunz_2017 explicitly recites “architectural drawings” and models used by architects illustrates floorplans in FIG. 24 and 27 and while the teaching of architectural drawings along with illustrations of floorplans may properly be found to make obvious the claimed “floorplan”, Glunz_2017 does not explicitly recite the word “floorplan.”
While Glunz_2017 teaches wearable device (FIG. 7, par 206 – 208) and while this teaching may properly be found to make obvious to use wearable devices to command software by voice of gesture, Glunz_2017 does not explicitly recite “wherein the instructions comprise one or more of a voice command an eye gaze or gesture.”
Therefore, Glunz_2017 does not explicitly recite “floorplan” for a physical space nor “wherein the instructions comprise one or more of a voice command an eye gaze or gesture” nor “mesh data over an image of” nor “mesh.”
Gausebeck_2019, however, makes obvious “floorplan” (par 77: “… 3D model generation component 118 can generate a 3D model or representation of the 3D model of an environment corresponding to a floorplan model of the environment… the environment comprises an interior of an architectural space…”; par 78: “… a floorplan model can be a simplified representation of surface (e.g., walls, floors, ceilings, etc.), portals (e.g., door openings) and/or window openings associated with the interior environment… a floorplan model can also include one or more objects…”; par 79: “… 3D model generation component 118 can be an schematic floorplan of an architectural structure… a floorplan model can comprise a series of lines…”; par 80: “… displays of individual items (e.g., dimensions) and/or classes of items can be toggled in a floorplan via a viewer on a remote device… a 2D floorplan model can include surfaces (e.g., walls, floors, ceilings, etc.)… with derived 3D data 116 used to generate a 3D model and projected to a flat 2D surface…”; par 81: “… a floorplan in 2D or 3D, etc.), and/or associated 2D and 3D data can be rendered at a user device 130 via a display 132. For example,.. a graphical user interface (GUI)…”)
Gausebeck_2019, makes obvious “wherein the instructions comprise one or more of a voice command an eye gaze or gesture” (par 87: “… the user device 130 can further receive user input (e.g., via a mouse, tough-screen, keyboard, gesture direction, gaze detection, etc.)…”; par 219: “… AR data objects 3002 can include data objects that are not visually displayed (or initially visually displayed) but can be interacted with and/or are activated in response to a trigger (e.g., a user pointing at, viewing in line of sight of the user, a gesture etc.)… based on viewing or pointing to an actual object appearing in the environment… auxiliary data can be rendered that is associated with the building… video data, sound data, graphical image data corresponding to an object or thing… the AR data object 3022 can include various types of auxiliary datasets. For example, the AR data object 3022 can include marks or labels… locations… video and/or snapshots… these marks may be manually or automatically…” EXAMINER NOTE: These citations teach to Receive instructions to make an object wherein the instructions comprise one or more of a voice command an eye gaze or gesture; and annotate the floorplan with a marker.)
Gausebeck_2019, makes obvious “mesh data [associated with] an image of” and “mesh” (Par 70: “in various embodiments, the 3D model generation component… for example, a mesh comprising a series of triangles… for example, a 3D model of an interior environment of building can comprise mesh data (e.g., a triangle mesh, a quad mesh, a parametric mesh, etc.”; par 71: “… portions of the 3D model geometric data (e.g., the mesh) can include image data… in various embodiments, received 2D image data 102 (or portions thereof) can be associated with portions of the mesh…”; par 248: “… 2D image data can be associated with portions of a 3D model mesh to associate visual data from the 2D image data… with the mesh…”).
Glunz_2017 and Gausebeck_2019 are analogous art because they are from the same field of endeavor called architectural models. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine Glunz_2017 and Gausebeck_2019. The rationale for doing so would have been that Glunz_2017 teaches to build XD models of architecture and teaches to use mobile devices that include graphical user interfaces (GUI) and include wearable devices. Gausebeck_2019
Teaches that XD models of architecture can be projected into a 2D floorplan and displayed on a GUI and further that interaction with devices such as wearable ones can be accomplished by gestures and gazes.
Therefore, it would have been obvious to combine Glunz_2017 and Gausebeck_2019 for the benefit of viewing floorplans and interacting with devices to obtain the invention as specified in the claims.
While Gausebeck_2019 teaches that data objects can be toggled and overlayed onto images (par 80: “… display of individual items… or classes of items can be toggled…”; par 218: “… digital data objects can be rendered as overlay data… rendered on the device display…”; par 219: “… data objects 3022 can include data objects that are not visually displayed (or initially visually displayed) but can be… activated…”; par 220: “data objects as overlays… cameras 1404 can capture live image data of an environment…”), Gausebeck_2019 does not teach that the overlayed and toggled data objects include a mesh.
Therefore, Glunz_2017 and Gausebeck_2019 does not explicitly teach that the mesh data is “over” an image.
Waichberg_2022 makes obvious to display a mesh “over” an image (FIG. 5: “… layering mesh over real world images captured by local user’s device…”; page 8 item 1: “… receive an image… form the user device… provide a 3D mesh for overlay on the image… and receive annotation at a user-selected position on the image… receive the annotation and the user-selected position from the support device…”
Glunz_2017 and Waichberg_2022 are analogous art because they are from the same field of endeavor called annotating image. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine Glunz_2017 and Waichberg_2022 The rationale for doing so would have been Glunz_2017 teaches a support platform for architects that includes annotating image with things such a punch list items (par 362: “punch list”) in order ensure repairs or modifications are performed. Waichberg_2022 teaches a service platform that include overlaying a mesh on images for the purpose of guidance (par 40) for improved annotations. The annotation is anchored on the spatial mesh (par 56). Therefore, it would have been obvious to combine the mobile device that takes images and also the wearable devices taught by Gluz_2017 and used for annotating architectural drawings with the overlayed mesh taught by Waichberg_2022 for the benefit of guidance and anchoring of annotations to selected portions of the mesh to obtain the invention as specified in the claims.
Claim 11. The limitations of claim 11 are substantially the same as those of claim 1 and are rejected due to the same reasons as outlined above.
Claim 4, 14. Glunz_2017 makes obvious “wherein the mobile computing device generates a reference marker and assigns the reference marker to the object and at least one other object in the space to be linked” (FIG. 20, 26, 27).
Gausebeck_2019 makes obvious “wherein the mobile computing device generates a reference marker and assigns the reference marker to the object and at least one other object in the space to be linked” (par 88: “… selecting a location on the representation of the 3D model for viewing… including a way point… selecting a tag icon…”; par 90: “… walking mode can also consist of moving between waypoints, where the waypoints are associated with known positions of captured 2D imagery associated with the 3D model… a waypoint that is associated with a capture position of a 2D image associated with the point or area…” EXAMINER NOTE: a waypoint is a location with a marker/tag that is simultaneously linked to a previous location and a next location and also a current location and also linked to images, audio, text, etc.)
Claim 5, 15. Gausebeck_2019 makes obvious “wherein the mobile computing device generates a link marker indicating a location between entrances and egresses between rooms in the space, associates the link marker with the reference markers, and associates the link marker with other reference markers in another space associated with a same link marker” (FIG.3 illustrates a walking path between two rooms, a hallway, and stairs. The walking path includes entrances and egresses between rooms.”; par 88: “… selecting a location on the representation of the 3D model for viewing… including a way point… selecting a tag icon…”; par 90: “… walking mode can also consist of moving between waypoints, where the waypoints are associated with known positions of captured 2D imagery associated with the 3D model… a waypoint that is associated with a capture position of a 2D image associated with the point or area…” EXAMINER NOTE: a waypoint is a location with a marker/tag that is simultaneously linked to a previous location and a next location and also a current location and also linked to images, audio, text. Also, a waypoint may be, for example, entrances and egresses).
Claim 6, 16. Glunz_2017 makes obvious “wherein the mobile computing device captures media comprising one or more of audio, images, and video at points of interest in the physical space and associates the captured media with the marker” (par 446: “in a preferred embodiment the mobile target network device 12, 14, 16, 106-110, 27, 29 provides, electronic text, audio, digital pictures, video, etc. electronic collaborative information 254’ including for example a notation… the electronic collaborative information is also used to verify there is a one-to-one correlation between all actual physical objects and all virtual objects…; FIG. 20, FIG. 26, FIG. 27; par 419 – 420: “… an actual mobile target device 12, 14, 16, 106-110, 27, 29 is used to follow a path 330 through an actual physical location obtaining and sending actual physical collaborative information… the actual physical collaborative information includes GPS and/or location coordinates… actual mobile target device… includes an actual camera 331 or a network device with a camera component (e.g., 12, etc.) and collects actual electronic collaborative information 254’ with a camera point-of-view cone 333 including GPS 3332’ and/or other location coordinates at the actual physical locations…”).
Claim 8, 18. Glunz_2017 makes obvious “wherein the mobile computing device adds a note representing by a box filled with text and associates the notice with the marker” (FIG. 20; par 352: “… message including electronic instructions for the 3D modeling program to display a dialog box on the 3D modeling program being accessed…”; par 269: “… display a dialog box 254 on the 3D modeling program…”; par 371 - 372: “… the dialog box 254 (FIG. 20) is created… and allows, text, audio, video, etc. information to be inserted and/or attached to the target device… Sally selects portion 248 and dialog box 254 pops up. Sally enters audio information 255, text information 256, etc. into the dialog box… jane is aware in real-time that Sally made this request…”).
Claim 10, 20. Glunz_2017 makes obvious “wherein the object comprises one or more of a thermostat, sensor, furniture, corner, door, window, pipe, wire, and water tank” (par 223: “… 3D object model template is a template for a specific type of 3D modeling object… may be for door, window, beam, truss… may be pipeline components… electronic device components… may be furniture… the present invention is not limited to such embodiments…” EXAMINER NOTE: Glunz_2017 does not explicitly recites “thermostat” or “sensor” or “wire”, however, electronic device components are recited.)
Claim 9, 19. Glunz_2017 makes obvious “wherein the floor plan is a drafting-style floor plan” (FIG. 27). Also, Gausebeck_2019 makes obvious “wherein the floor plan is a drafting-style floor plan” (par 78: “… a floorplan model can be a simplified representation of surface (e.g., walls… doors… window openings associated with an interior environment…”; par 80: “… a 3D model and projected to a flat 2D surface…” EXAMINER NOTE: instant specification par 31: “… a second style of floorplan can resemble a drafting diagram that removes clutter by recognizing wall in a 3D space before projection to 2D…”).
Claims 3, 13 are rejected under 35 U.S.C. 103 as being unpatentable over Glunz_2017 in view of Gausebeck_2019 in view of Waichberg_2022 in view of Marker_2022 (Drawing camera frustrum in Rviz using tf and Visualization Marker, 11/30/2022).
Claim 3, 13 Marker_2022 makes obvious “wherein the mobile computing device lays trace lines as the marker, wherein the trace lines each comprise a short line extending from a surface of the object” (page 1: “… draw the camera frustrum… using visualization markers… indicating my camera’s frustrum, something like this (red lines indicate camera frustrum)…”
PNG
media_image1.png
348
603
media_image1.png
Greyscale
EXAMINER NOTE: the above image shows a visualization marker which has trace lines extending from a surface of an object where the object is on a floorplan.)
Glunz_2017 and Marker_2022 are analogous art because they are from the same field of endeavor called markers on a floorplan. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine Glunz_2017 and Marker_2022. The rationale for doing so would have been that Glunz_2017 teaches to include information in architectural drawings that include a camera point-of-view of actual physical locations in the modeling program (FIG. 30, FIG. 27; par 420: “… mobile target device 12… collect… a camera point of view… including GPS…at the actual physical location. This helps ensure a person does not have to guess or estimate the precise physical location… or estimate the precise point-of-view… in such an embodiment, another person can return to the physical location… and precisely find the exact physical location… to re-view the exact same camera point-of-view…”). Marker_2022 teaches to have a marker called a visualization marker that uses lines extending from a surface to indicate the precise point-of-view. Therefore, it would have been obvious to combine Glunz_2017 and Marker_2022 for the benefit of having information that will allow someone to re-visualize with the precise field-of-view to obtain the invention as specified in the claims.
Claims 2, 12 are rejected under 35 U.S.C. 103 as being unpatentable over Glunz_2017 in view of Gausebeck_2019 in view of Waichberg_2022 in view of Stob_2019 (US 2019/0057395 A1).
Claim 2, 12. Stob_2019 makes obvious “wherein the mobile computing device adjusts a location of the marker in the floorplan based on further instructions” (par 37: “… a pin location 54… the user can specify the location of the pin (and geocoordinates) for different locations associated with the organization building by moving the pin… ability to insert or move pins…”).
Glunz_2017 and Stob_2019 are analogous art because they are from the same field of endeavor called marking thing on a drawing of an environment. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine Glunz_2017 and Stob_2019.
The rationale for doing so would have been Glunz_2017 teaches to physically go to a location and mark the location on a drawing of an environment/building to identify, for example, needed repairs (e.g., punch lists, etc.). Stob_2019 teaches to mark the location associated with a location of an environment/building on a drawing and that the inserting or moving the marking (i.e., pin) can be enabled or disabled based on the location of a device.
Therefore, it would have been obvious to combine Glunz_2017 and Stob_2019 for the benefit of preventing a marked/tagged/pinned location in a building from being moved/modified if the person is not physical at the location because a punch list item can’t be repaired if you don’t go to the physical location to obtain the invention as specified in the claims.
Claims 7, 17 are rejected under 35 U.S.C. 103 as being unpatentable over Glunz_2017 in view of Gausebeck_2019 in view of Waichberg_2022 in view of Iverson_2017 (US 2017/0358133 A1).
Claim 7, 17. Iverson_2017 makes obvious “wherein the mesh data can be temporarily removed from the image” (par 39: “… in some cases, the UI 500 may also provide a toggle control item 595 or the like for switching between a wire-frame view… and a normal or textured mode… visualization of mesh data… for more intuitive editing, an undo or redo option may be enabled…”)
Glunz_2017 and Waichberg_2022 and Iverson_2017 are analogous art because they are from the same field of endeavor called display of 3D images. Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to combine Glunz_2017 and Waicherg_2022 and Iverson_2017. The rationale for doing so would have been Glunz_2017 teaches to display textured 3D images on a mobile device while Waichberg_2022 teaches to include a mesh overlayed on top . Iverson_2017 teaches that display of high resolution mesh, such as 3D images and video data are ubiquitous, but presenting such images might not be suitable due to limited bandwidth and further teaches the perform simplification techniques such as displaying a wire frame mesh or the normal textured model. Therefore, it would have been obvious to combine Glunz_2017 and Waichberg_2022 and Iverson_2017 for the benefit of reducing the amount of data transmitted to the mobile device if there is limited bandwidth to obtain the invention as specified in the claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIAN S COOK whose telephone number is (571)272-4276. The examiner can normally be reached 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emerson Puente can be reached at 571-272-3652. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRIAN S COOK/Primary Examiner, Art Unit 2187
