DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
IDS filed 11/17/2025 is being considered by the examiner
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 14 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 14 recites, "determine a travel path to service each asset of the multiple assets within the building, the travel path being determined based on a minimum travel distance required [line 4]." Because claim 8 has been amended to include "determine … a travel path through the building to the virtual asset, the travel path being determined based at least I part of a travel distance to the virtual asset [line 8]," it is unclear whether the travel path recited in claim 14 refers to the travel path recited in claim 8, or if the travel path recited in claim 14 is a newly determined travel path. As such, claim 14 is indefinite.
Response to Amendment
Applicant's amendment filed 2/2/2026 has been received and entered into the record. As a result, claims 1, 8, and 15 have been amended. Therefore, claims 1-20 are presented for examination.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-16 are rejected under 35 U.S.C. 103 as being unpatentable over Sridharan et al. [U.S. Pub. 2021/0200713] ("Sridharan") in view of Lovett et al. [U.S. Pub. 2016/0140868] ("Lovett")
With regard to claim 1, Sridharan teaches a building system comprising one or more storage devices storing instructions thereon that, when executed by one or more processors, cause the one or more processors to ("Memory 520 can be communicably connected to processor 512 via processing circuit 510 and can include computer code for executing (e.g., by processor 512 one or more processes described herein [par. 0157]"):
ingest information associated with a physical asset of a building ("BIM data may be ingested by a space graph and stored in a graph … Objects in the BIM may be stored as digital twins of physical building assets, for example. The space graph can be a dynamic data structure allowing entities (e.g., building assets), relationships, and/or information to be stored for operating a building. The space graph can be a data structure including nodes and edges, the nodes representing particular entities and the edges between the nodes representing a relationship between the entities [par. 0165]"), the physical asset being one of a heating system, a ventilation system, or an air conditioning system ("the objects represent spaces or equipment within a building associated with the BIM. For example, a BIM may include a plurality objects representing spaces such as rooms, wings, floors, levels, zones, etc., and a plurality of objects representing equipment such as HVAC equipment (e.g., ducts, VAVs, AHUs, etc.) [par. 0158]");
cross-correlate a first identifier associated with the information with a second identifier associated with a virtual asset of a graphical model of the building to determine a location of the virtual asset within the graphical model, the virtual asset corresponding to the physical asset ("BAS points of the facility (e.g., site, campus, building, etc.) associated with the BIM are mapped to corresponding objects in the BIM. Step 1106 may be performed by BAS-BIM integrator 528, for example. Mapping BAS points to corresponding BIM objects may include associating a point (e.g., a sensor) of the BAS to an object within the BIM. For example, a BAS point for an output air temperature sensor of an AHU may be mapped to a BIM object representing the AHU [par. 0204]" and "a BAS point associated with a temperature sensor may be mapped to a corresponding object within the BIM that provides a digital representation of a temperature sensor. In this way, temperature data from the temperature sensor may be viewed within a 3D model, generated from the BIM [par. 0226]");
cause the graphical model of the building to include a representation of the information using the location of the virtual asset ("At step 1114, a three-dimensional (3D) model is generated from the configured BIM. Step 1114 may be performed by BIM viewer 534, for example. Generating the 3D model may include processing the BIM data via a computer-aided drawing program or another type of 3D modelling engine. The 3D model generally includes a 3D graphical representation of a building or site represented by the BIM. The 3D may also include 3D graphical representations of building assets such as spaces and/or equipment [par. 0206]" and "temperature data from the temperature sensor may be viewed within a 3D model, generated from the BIM [par. 0226]"); and
cause a display device of a user device to display the graphical model within a user interface ("the 3D model is a 3D building model generated by BIM viewer 534 and presented to a user of user device 546 via UI generator 536 [par. 0247]").
Although Sridharan teaches identifying and navigating to virtual assets within the graphical model [par. 0269], Sridharan does not explicitly teach to determine, using the location of the virtual asset within the graphical model, a travel path through the building to the virtual asset, the travel path being determined based at least in part on a travel distance to the virtual asset.
In the same field of endeavor (augmented reality for system maintenance), Lovett teaches to determine, using a location of a virtual asset within a graphical model, a travel path through a building to the virtual asset ("The augmented reality creates and calculates a directional path 320 for guiding a user to one or more computer devices 302 or components 306 (e.g., a port on a server, a cable, etc.) for performing maintenance or service (such as those computer devices 302 requiring or scheduled for maintenance or service) [par. 0050]"),
the travel path being determined based at least in part on a travel distance to the virtual asset ("the augmented reality module 214 illustrates all of the mapping 300 and/or a portion of the mapping 300 in either two-dimensional (2D) or three-dimensional (3D) and overlaid on a real-time video image of the data center 100. The mapping 300 showing a current location of a user 350 relative to the target location while correctly orienting a user 350 for easy navigation to the target location [par. 0049]" and "Attention cues may be activated by the management module 220 and provide for alerts, or guides such as 'you have turned down aisle 3 and are 30 feet away from the target location.' [par. 0053]").
Lovett further teaches, "Identifying each specific computing device or component requiring maintenance or repair services is a challenge … a need exists for identifying the exact computing device requiring maintenance and/or repair without requiring manually installed service required tags or indicators [par. 0002]" and "Temperature, elevations, safety codes, building codes, fire alarms, exits, hazardous areas, and other data relating to the data center 100 may be integrated and displayed with the mapping in the augmented reality view [par. 0035]."
It would have been obvious to one having ordinary skill in the art at the time of filing the invention to have modified Sridharan's interactive model, to include Lovett's teachings of the augmented reality device and associated functions to perform maintenance, for the benefit providing a user greater convenience when viewing and performing maintenance on the building.
With regard to claim 2, the combination above teaches the building system of claim 1. Sridharan in the combination further teaches wherein the information associated with the physical asset is indicative of at least one of a space utilization metric, a temperature metric ("temperature data from the temperature sensor may be viewed within a 3D model, generated from the BIM [par. 0226]"), a humidity level metric, or an internal air quality metric associated with an area in the building and
the representation of the information is a heat map overlaid onto a corresponding virtual area within the graphical model of the building ("Referring now to FIG. 23, an example interface 2300 of a zone heat map that presents thermal data for particular spaces or zones within the BIM viewer is shown [par. 0260]").
Note: claim is presented in the alternative.
With regard to claim 3, the combination above teaches the building system of claim 2. Sridharan in the combination further teaches wherein the user interface is configured to allow a user to modify one or more set points associated with the physical asset ("A user may select a space (e.g., space 28) from space tree 2406, for example, and configure temperature data for the space [par. 0262]").
With regard to claim 4, the combination above teaches the building system of claim 1. Sridharan in the combination further teaches wherein the instructions further cause the one or more processors to:
determine that multiple sensors within an area of the building are above or below associated set points ("providing a user with an easy to understand overview of temperatures within a plurality of zones of a building or site. For example, user may identify, at a glance, what zones are above or below desired temperature setpoints [par. 0261]");
determine, based on the multiple sensors being above or below the associated set points that there is a problem within the area of the building ("Such information may allow the user to quickly identify problems with spaces or equipment (e.g., air condition equipment that is not working correctly) or identify other information to aid in building or site management [par. 0261]"); and
provide, via the user interface, a notification regarding the problem within the area of the building, the notification represented within a corresponding virtual area of the graphical model of the building ("3D model 2310 includes a number of zones 2312-2320 that have been configured to present thermal data corresponding to each zone [par. 0260]" and "a shading, color, style, or other identifying feature of each zone may change based on the temperature of the zone, an alarm status for the zone, a temperature setpoint for the zone, etc., thereby providing a user with an easy to understand overview of temperatures within a plurality of zones of a building or site. For example, user may identify, at a glance, what zones are above or below desired temperature setpoints [par. 0261]").
With regard to claim 5, the combination above teaches the building system of claim 1. Sridharan in the combination further teaches wherein the user interface is configured to allow a user to selectively view specific spaces or areas within the building that are serviced by a particular device ("a user may select zone 2312 (e.g., a conference room) to display current temperature and temperature alarm data within a data window 2330 [par. 0260]").
With regard to claim 6, the combination above teaches the building system of claim 1. Sridharan in the combination further teaches wherein the instructions further cause the one or more processors to:
receive an indication of a complaint associated with an area in the building ("identifying alarms, faults, or necessary maintenance events among building assets [par. 0269]"); and
provide, via the user interface, virtual directions to and a location indication of at least one of an upstream device or a downstream device associated with the area having the indication of the complaint ("Viewing alarm, fault, and/or diagnostic data in this way can provide insight into how an alarm or fault may affect upstream or downstream assets of the problem asset, and may provide an easy to navigate and intuitive interface for viewing such data about multiple assets quickly. Viewing such data in a 3D model may also provide an indication to the physical location of a problem asset, thereby decreasing troubleshoot and/or maintenance times by aiding a user in navigating to the problem asset [par. 0269];" where Lovett in the combination teaches the virtual directions: "the augmented reality module 214 illustrates all of the mapping 300 and/or a portion of the mapping 300 in either two-dimensional (2D) or three-dimensional (3D) and overlaid on a real-time video image of the data center 100. The mapping 300 showing a current location of a user 350 relative to the target location while correctly orienting a user 350 for easy navigation to the target location [par. 0049]").
With regard to claim 7, the combination above teaches the building system of claim 6. Sridharan in the combination further teaches wherein the at least one of the upstream device or the downstream device is hidden from view by an obstacle in the building and the location indication is a highlighted rendering of the upstream device or the downstream device within the graphical model that is visible through a virtual representation of the obstacle ("Responsive to a selection of AHU object 2714 and/or an alarm associated with said object, an appearance of duct object 2910 and terminal unit objects 2912 and 2914 may alter (e.g., a color change, highlighting, etc.) to indicate that said objects are affected by AHU object 2714 [par. 0284]"), wherein the obstacle is one of a floor, a wall, or a ceiling (see [fig. 29] where, for example, duct 2910 is visible through a ceiling).
With regard to claim 8, Sridharan teaches a building system comprising one or more storage devices storing instructions thereon that, when executed by one or more processors, cause the one or more processors to ("Memory 520 can be communicably connected to processor 512 via processing circuit 510 and can include computer code for executing (e.g., by processor 512 one or more processes described herein [par. 0157]"):
ingest information associated with a physical asset of a building ("BIM data may be ingested by a space graph and stored in a graph … Objects in the BIM may be stored as digital twins of physical building assets, for example. The space graph can be a dynamic data structure allowing entities (e.g., building assets), relationships, and/or information to be stored for operating a building. The space graph can be a data structure including nodes and edges, the nodes representing particular entities and the edges between the nodes representing a relationship between the entities [par. 0165]"), the physical asset being one of a heating system, a ventilation system, or an air conditioning system ("the objects represent spaces or equipment within a building associated with the BIM. For example, a BIM may include a plurality objects representing spaces such as rooms, wings, floors, levels, zones, etc., and a plurality of objects representing equipment such as HVAC equipment (e.g., ducts, VAVs, AHUs, etc.) [par. 0158]");
determine a location of a virtual asset of a graphical model of the building, the virtual asset corresponding to the physical asset ("a BAS point associated with a temperature sensor may be mapped to a corresponding object within the BIM that provides a digital representation of a temperature sensor. In this way, temperature data from the temperature sensor may be viewed within a 3D model, generated from the BIM [par. 0226]");
cause the graphical model of the building to include a representation of the information using the location of the virtual asset ("At step 1114, a three-dimensional (3D) model is generated from the configured BIM. Step 1114 may be performed by BIM viewer 534, for example. Generating the 3D model may include processing the BIM data via a computer-aided drawing program or another type of 3D modelling engine. The 3D model generally includes a 3D graphical representation of a building or site represented by the BIM. The 3D may also include 3D graphical representations of building assets such as spaces and/or equipment [par. 0206]" and "temperature data from the temperature sensor may be viewed within a 3D model, generated from the BIM [par. 0226]"); and
cause a display device of a user device to display the graphical model within a user interface ("the 3D model is a 3D building model generated by BIM viewer 534 and presented to a user of user device 546 via UI generator 536 [par. 0247]").
Although Sridharan teaches identifying and navigating to virtual assets within the graphical model [par. 0269], Sridharan does not explicitly teach to determine, using the location of the virtual asset within the graphical model, a travel path through the building to the virtual asset, the travel path being determined based at least in part on a travel distance to the virtual asset.
In the same field of endeavor (augmented reality for system maintenance), Lovett teaches to determine, using a location of a virtual asset within a graphical model, a travel path through a building to the virtual asset ("The augmented reality creates and calculates a directional path 320 for guiding a user to one or more computer devices 302 or components 306 (e.g., a port on a server, a cable, etc.) for performing maintenance or service (such as those computer devices 302 requiring or scheduled for maintenance or service) [par. 0050]"),
the travel path being determined based at least in part on a travel distance to the virtual asset ("the augmented reality module 214 illustrates all of the mapping 300 and/or a portion of the mapping 300 in either two-dimensional (2D) or three-dimensional (3D) and overlaid on a real-time video image of the data center 100. The mapping 300 showing a current location of a user 350 relative to the target location while correctly orienting a user 350 for easy navigation to the target location [par. 0049]" and "Attention cues may be activated by the management module 220 and provide for alerts, or guides such as 'you have turned down aisle 3 and are 30 feet away from the target location.' [par. 0053]").
Lovett further teaches, "Identifying each specific computing device or component requiring maintenance or repair services is a challenge … a need exists for identifying the exact computing device requiring maintenance and/or repair without requiring manually installed service required tags or indicators [par. 0002]" and "Temperature, elevations, safety codes, building codes, fire alarms, exits, hazardous areas, and other data relating to the data center 100 may be integrated and displayed with the mapping in the augmented reality view [par. 0035]."
It would have been obvious to one having ordinary skill in the art at the time of filing the invention to have modified Sridharan's interactive model, to include Lovett's teachings of the augmented reality device and associated functions to perform maintenance, for the benefit providing a user greater convenience when viewing and performing maintenance on the building.
With regard to claim 9, the combination above teaches the building system of claim 8. Sridharan in the combination further teaches wherein the user interface is configured to:
provide an indication of one or more downstream or upstream assets associated with the physical asset ("Responsive to a selection of AHU object 2714 and/or an alarm associated with said object, an appearance of duct object 2910 and terminal unit objects 2912 and 2914 may alter (e.g., a color change, highlighting, etc.) to indicate that said objects are affected by AHU object 2714 [par. 0284]");
enable a user to modify one or more functionality parameters of the physical asset ("a current temperature, temperature setpoint, and/or temperature limits for a space are retrieved from BAS 400 or an analytics engine associated with BAS 400, or entered by a user … [par. 0263]"); and
provide an indication of one or more effects on the one or more downstream or upstream assets based on the modification of the one or more functionality parameters of the physical asset ("Responsive to a selection of AHU object 2714 and/or an alarm associated with said object, an appearance of duct object 2910 and terminal unit objects 2912 and 2914 may alter (e.g., a color change, highlighting, etc.) to indicate that said objects are affected by AHU object 2714 [par. 0284]" and "From the visual indication of interface 2900, a user can quickly determine how an alarm associated with AHU object 2714 may affect other building assets. For example, an alarm indicating a low output air temperature may indicate that rooms served by terminal units associated with terminal unit objects 2912 and/or 2914 may not reach a desire temperature setpoint [par. 0285]").
With regard to claim 10, the combination above teaches the building system of claim 8. Although Sridharan teaches the user device displaying a 3D model with interactive objects and prompts [figs. 19-33], Sridharan does not explicitly teach wherein the user device is one of a virtual reality (VR) device or an augmented reality (AR) device.
Lovett in the combination further teaches wherein a user device is one of a virtual reality (VR) device or an augmented reality (AR) device ("The augmented reality component arranges and manipulates information of the physical geographic layout and network mapping of the target location for displaying the augmented reality view in the electronic device [par. 0018]").
Lovett further teaches, "Identifying each specific computing device or component requiring maintenance or repair services is a challenge … a need exists for identifying the exact computing device requiring maintenance and/or repair without requiring manually installed service required tags or indicators [par. 0002]" and "Temperature, elevations, safety codes, building codes, fire alarms, exits, hazardous areas, and other data relating to the data center 100 may be integrated and displayed with the mapping in the augmented reality view [par. 0035]."
It would have been obvious to one having ordinary skill in the art at the time of filing the invention to have modified Sridharan's interactive model, to include Lovett's teachings of the augmented reality device and associated functions to perform maintenance, for the benefit providing a user greater convenience when viewing and performing maintenance on the building.
With regard to claim 11, the combination above teaches the building system of claim 10. Lovett in the combination further teaches wherein the user device is the AR device and the graphical model is overlaid onto a real-time view of the building ("The augmented reality component arranges and manipulates information of the physical geographic layout and network mapping of the target location for displaying the augmented reality view in the electronic device [par. 0018]").
With regard to claim 12, the combination above teaches the building system of claim 11. Lovett in the combination further teaches wherein the instructions further cause the one or more processors to:
provide, via the user interface, troubleshooting instructions for resolving a fault associated with the physical asset ("The work order 602 and the directions 604 are included by the augmented reality module 214 and displayed in an augmented realty of the mapping of the data center 100 in the electronic device 512. For example, in FIGS. 5-6, the work order 602 indicates that cable 314 is detected as defective and is connected to port 304 of server 304. The work order 602 calls for the replacement of cable 314 in port 304 [par. 0069];" where Sridharan in the combination teaches the fault of the physical asset [par. 0279]).
With regard to claim 13, the combination above teaches the building system of claim 12. Lovett in the combination further teaches wherein the troubleshooting instructions are provided via an overlay and include indications of one or more upstream or downstream devices to assess, wherein the troubleshooting instructions include multiple sequential troubleshooting steps ([fig. 6: Work Order (602) A-D] and "The work order 602 and the directions 604 are included by the augmented reality module 214 and displayed in an augmented realty of the mapping of the data center 100 in the electronic device 512. For example, in FIGS. 5-6, the work order 602 indicates that cable 314 is detected as defective and is connected to port 304 of server 304. The work order 602 calls for the replacement of cable 314 in port 304 [par. 0069];" where Sridharan in the combination teaches the indication of one or more upstream od downstream devices to assess: "diagnostic text box 3320 includes further details about specific faults (e.g., 'damper closed', 'actuator fault') for a particular building asset and may include additional information about the particular building asset such as relationship data. As described above, relationship data can be useful in troubleshooting and/or servicing building assets, by determining how a fault in a first building asset may affect upstream or downstream assets of the first asset [par. 0296]").
With regard to claim 14, the combination above teaches the building system of claim 8. Sridharan in the combination further teaches wherein instructions further cause the one or more processors to:
determine that multiple assets within the building have maintenance due ("a BIM can be a useful tool in identifying alarms, faults, or necessary maintenance events among building assets [par. 0269]" and "service requests can be generated in response to an alarm, a fault, or a scheduled maintenance [par. 0290]").
Lovett in the combination teaches to determine a travel path to service each asset of multiple assets within a building ("The augmented reality creates and calculates a directional path 320 for guiding a user to one or more computer devices 302 or components 306 (e.g., a port on a server, a cable, etc.) for performing maintenance or service (such as those computer devices 302 requiring or scheduled for maintenance or service) [par. 0050]"); and
provide, via a user interface, a virtual depiction of the travel path overlaid onto a graphical model ("the augmented reality module 214 illustrates all of the mapping 300 and/or a portion of the mapping 300 in either two-dimensional (2D) or three-dimensional (3D) and overlaid on a real-time video image of the data center 100. The mapping 300 showing a current location of a user 350 relative to the target location while correctly orienting a user 350 for easy navigation to the target location [par. 0049]" and "the directional paths 320A, 320B may be a curve, straight line, or series of 3D objects or illustrations that directs attention and/or navigates the user to the target location 504 [par. 0052]").
The combination does not explicitly teach where the travel path being determined based on a minimum travel distance required.
However, Sridharan teaches, "may provide an easy to navigate and intuitive interface for viewing such data about multiple assets quickly. Viewing such data in a 3D model may also provide an indication to the physical location of a problem asset, thereby decreasing troubleshoot and/or maintenance times by aiding a user in navigating to the problem asset [par. 0269]" and
Lovett teaches, "A link to the target location using the directional path 320 may be followed rapidly and efficiently to the target location regardless of the current position of the target location relative to the user 350 or the distance to the target location 504. The directional path 320 of the augmented reality of the data center 100 connects the user 350 directly to a cued target location [par. 0052]."
It can be seen that both Sridharan and Lovett consider reducing maintenance times by navigating to a problem asset quickly and/or efficiently. Lovett above also teaches going to the asset directly and/or in a straight line [par. 0052]. It would have been obvious to one having ordinary skill in the art at the time of filing the invention to have implemented the system taught by the combination of Sridharan and Lovett, where the travel path is determined based on a minimum travel distance required, i.e., the shortest or most direct path, for the benefit of navigating to the problem asset quickly and efficiently.
With regard to claim 15, Sridharan teaches a method comprising:
ingesting, by one or more processors of a building system ("Memory 520 can be communicably connected to processor 512 via processing circuit 510 and can include computer code for executing (e.g., by processor 512 one or more processes described herein [par. 0157]"), information associated with a physical asset of a building ("BIM data may be ingested by a space graph and stored in a graph … Objects in the BIM may be stored as digital twins of physical building assets, for example. The space graph can be a dynamic data structure allowing entities (e.g., building assets), relationships, and/or information to be stored for operating a building. The space graph can be a data structure including nodes and edges, the nodes representing particular entities and the edges between the nodes representing a relationship between the entities [par. 0165]"), the physical asset being one of a heating system, a ventilation system, or an air conditioning system ("the objects represent spaces or equipment within a building associated with the BIM. For example, a BIM may include a plurality objects representing spaces such as rooms, wings, floors, levels, zones, etc., and a plurality of objects representing equipment such as HVAC equipment (e.g., ducts, VAVs, AHUs, etc.) [par. 0158]");
determining, by the one or more processors, a location of a virtual asset of a graphical model of the building, the virtual asset corresponding to the physical asset ("a BAS point associated with a temperature sensor may be mapped to a corresponding object within the BIM that provides a digital representation of a temperature sensor. In this way, temperature data from the temperature sensor may be viewed within a 3D model, generated from the BIM [par. 0226]");
causing, by the one or more processors, the graphical model of the building to include a representation of the information using the location of the virtual asset ("At step 1114, a three-dimensional (3D) model is generated from the configured BIM. Step 1114 may be performed by BIM viewer 534, for example. Generating the 3D model may include processing the BIM data via a computer-aided drawing program or another type of 3D modelling engine. The 3D model generally includes a 3D graphical representation of a building or site represented by the BIM. The 3D may also include 3D graphical representations of building assets such as spaces and/or equipment [par. 0206]" and "temperature data from the temperature sensor may be viewed within a 3D model, generated from the BIM [par. 0226]"); and
causing, by the one or more processors, a display device of a user device to display the graphical model within a user interface ("the 3D model is a 3D building model generated by BIM viewer 534 and presented to a user of user device 546 via UI generator 536 [par. 0247]").
Although Sridharan teaches identifying and navigating to virtual assets within the graphical model [par. 0269], Sridharan does not explicitly teach to determine, by the one or more processors and using the location of the virtual asset within the graphical model, a travel path through the building to the virtual asset, the travel path being determined based at least in part on a travel distance to the virtual asset.
In the same field of endeavor (augmented reality for system maintenance), Lovett teaches to determine, by one or more processors and using a location of a virtual asset within a graphical model, a travel path through a building to the virtual asset ("The augmented reality creates and calculates a directional path 320 for guiding a user to one or more computer devices 302 or components 306 (e.g., a port on a server, a cable, etc.) for performing maintenance or service (such as those computer devices 302 requiring or scheduled for maintenance or service) [par. 0050]"),
the travel path being determined based at least in part on a travel distance to the virtual asset ("the augmented reality module 214 illustrates all of the mapping 300 and/or a portion of the mapping 300 in either two-dimensional (2D) or three-dimensional (3D) and overlaid on a real-time video image of the data center 100. The mapping 300 showing a current location of a user 350 relative to the target location while correctly orienting a user 350 for easy navigation to the target location [par. 0049]" and "Attention cues may be activated by the management module 220 and provide for alerts, or guides such as 'you have turned down aisle 3 and are 30 feet away from the target location.' [par. 0053]").
Lovett further teaches, "Identifying each specific computing device or component requiring maintenance or repair services is a challenge … a need exists for identifying the exact computing device requiring maintenance and/or repair without requiring manually installed service required tags or indicators [par. 0002]" and "Temperature, elevations, safety codes, building codes, fire alarms, exits, hazardous areas, and other data relating to the data center 100 may be integrated and displayed with the mapping in the augmented reality view [par. 0035]."
It would have been obvious to one having ordinary skill in the art at the time of filing the invention to have modified Sridharan's interactive model, to include Lovett's teachings of the augmented reality device and associated functions to perform maintenance, for the benefit providing a user greater convenience when viewing and performing maintenance on the building.
With regard to claim 16, the combination above teaches the method of claim 15. Sridharan in the combination further teaches wherein the user interface is configured to allow a user to take a virtual tour of the building ("Navigation submenu 2220 may allow a user to select a particular floor of the building represented by 3D model 1910 and be presented with a first-person perspective view 2210 of the selected floor. The user may then adjust a slider shown in navigation submenu 2220 to control a speed at which to navigate through the 3D model of the selected floor [par. 0259]").
Claims 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Sridharan in view of Lovett further in view of Conboy [U.S. Pub. 2019/0172161].
With regard to claim 17, the combination of Sridharan and Lovet teaches the method of claim 16. Although Sridharan in the combination wherein ("Navigation submenu 2220 may allow a user to select a particular floor of the building represented by 3D model 1910 and be presented with a first-person perspective view 2210 of the selected floor. The user may then adjust a slider shown in navigation submenu 2220 to control a speed at which to navigate through the 3D model of the selected floor [par. 0259]").
Sridharan does not explicitly teach where the user device is a virtual reality device.
In the same field of endeavor (virtual reality walk through), Conboy teaches wherein a user device is a virtual reality device ("using virtual reality (VR) and augmented reality (AR) technologies to support virtual inspection of a fire-protected job-site based on a 3D virtual model of the fire-protected wood-framed building [par. 0132]" and "a virtual reality (VR) enabled inspection walk-through process displayed on the client system 120 [par. 0720]").
It would have been obvious to one having ordinary skill in the art at the time of filing the invention to have modified Sridharan's interactive model, to include Lovett's teachings of the virtual reality device, for the benefit providing a user greater convenience when viewing portions of the building.
With regard to claim 18, the combination above teaches the method of claim 17. Conboy in the combination teaches the method further comprising:
receiving, by the one or more processors, one or more notes or annotations from the user device via the user interface during the virtual walk through of the building ("authorized stakeholders are enabled to walk through the 3D virtual model of the wood-framed building, using a client computing system deployed on the system network. During the virtual walk-though, the authorized stakeholder can access and review the uploaded data linked to each AR-based virtual inspection checkpoint 300 augmented with real job-site collected data … The stakeholder can also make comments and leave notes at these AR-based virtual checkpoints 300 [par. 0736]" and " displayed during a virtual reality (VR) and augmented reality (AR) supported virtual inspection process … At each virtual inspection checkpoint 300′ in the 3D virtual model, collected and uploaded certifications, verifications and documents are reviewable during the inspection walkthrough, and allowing the reviewer to take and post notes to other stakeholders represented in the system [par. 0718]"); and
causing, by the one or more processors, a second display device of an augmented reality device to overlay the graphical model and the one or more notes or annotations onto the building within a second user interface ([figs. 82A, 82C] and "displayed during a virtual reality (VR) and augmented reality (AR) supported virtual inspection process … At each virtual inspection checkpoint 300′ in the 3D virtual model, collected and uploaded certifications, verifications and documents are reviewable during the inspection walkthrough, and allowing the reviewer to take and post notes to other stakeholders represented in the system [par. 0718];" where the notes can be viewed on an augmented reality device of another reviewer/stakeholder).
It would have been obvious to one having ordinary skill in the art at the time of filing the invention to have modified the teachings of Sridharan, to include Conboy's teachings of a second display device, for the benefit of allowing multiple users to virtually walk through and make/review annotations.
Claims 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Sridharan in view of Lovett further in view of Morrison [U.S. Pub. 2017/0132841].
With regard to claim 19, the combination of Sridharan and Lovett teaches the method of claim 15. Although Sridharan in the combination teaches wherein the user interface is configured to allow a user to virtuallyview a virtual asset within an area of the graphical model ("The 3D model generally includes a 3D graphical representation of a building or site represented by the BIM. The 3D may also include 3D graphical representations of building assets such as spaces and/or equipment [par. 0206]"), Sridharan does not explicitly teach to virtually install a new virtual asset within an area of the graphical model to determine whether a corresponding physical asset will fit within a corresponding physical area of the building.
In the same field of endeavor (virtual installation), Morrison teaches to virtually install a new virtual asset within an area of a graphical model to determine whether a corresponding physical asset will fit within a corresponding physical area of a building ("A user may control the placement of the 3D image of the refrigerator to appear to be in the place of an older refrigerator which is to be replaced. However, the visualized refrigerator may relate to a product having dimensions which may not be conducive to the space allowed in the physical environment [par. 0196]").
Morrison further teaches, "allowing enhanced visualization of products within the home before having to expensively ship a physical product which may not be a good fit [par. 0201]."
It would have been obvious to one having ordinary skill in the art at the time of filing the invention to have included Morrison's teachings of visualizing new items in a building, with the teachings of Sridharan, for the benefit of determining whether a new asset would be a good physical fit when upgrading or replacing an asset.
With regard to claim 20, the combination above teaches the method of claim 19. Morrison in the combination further teaches wherein virtually installing the new virtual asset includes virtually moving the new virtual asset into and throughout the graphical model of the building to ensure that the corresponding physical asset will be able to move into and throughout the building to a location of installation ("a user may wish to see whether the selected refrigerator model would fit through a particular doorway near the kitchen, the user may control the refrigerator image to be moved around the environment and through the doorway. If any portion of the refrigerator overlaps with an edge of the doorway, the augmented reality e-commerce application may change the overlapping portion to appear as red or some other color [par. 0197]").
Response to Arguments
Applicant’s arguments with respect to claims 1, 8, and 15 have been considered but are moot in light of the new grounds of rejection necessitated by Applicant's amendment.
Citation of pertinent prior art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Somu et al. [U.S. Pub. 2023/0153477] teaches using virtual reality and/or augmented reality to view building assets and systems.
Harder et al. [U.S. Pub. 2020/0125084] teaches a building management system that identifies a building device in a fault state based on asset data.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VINCENT W CHANG whose telephone number is (571)270-1214. The examiner can normally be reached (M-F) 10:00 am - 6:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mohammad Ali can be reached at 571-272-4105. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VINCENT WEN-LIANG CHANG/
Examiner
Art Unit 2119
/MOHAMMAD ALI/Supervisory Patent Examiner, Art Unit 2119