DETAILED ACTION
This Office Action is in response to the claims filed on 10/27/2023.
Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner Notes
Examiner cites particular columns, paragraphs, figures and line numbers in the
references as applied to the claims below for the convenience of the applicant. Although
the specified citations are representative of the teachings in the art and are applied to
the specific limitations within the individual claim, other passages and figures may apply
as well. Examiner may also include cited interpretations encompassed within parenthesis, e.g. (Examiner’s interpretation), for clarity. It is respectfully requested that, in preparing responses, the applicant fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner. The entire reference is considered to provide disclosure relating to the claimed invention. The claims & only the claims form the metes & bounds of the invention. Office personnel are to give the claims their broadest reasonable interpretation in light of the supporting disclosure. Unclaimed limitations appearing in the specification are not read into the claim. Prior art was referenced using terminology familiar to one of ordinary skill in the art. Such an approach is broad in concept and can be either explicit or implicit in meaning. Examiner's Notes are provided with the cited references to assist the applicant to better understand how the examiner interprets the applied prior art. Such comments are entirely consistent with the intent & spirit of compact prosecution.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Information Disclosure Statement
No IDS was provided.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham V. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
In the event the determination of the status of the application as subject to
AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Qiuchen Lu, Vivi, et al. "Developing a dynamic digital twin at a building level: Using Cambridge campus as case study." (2019) (hereinafter referred to as “Vivi”)
in view of Cella et al. US Pub. No. 2023/0281527 A1 (hereinafter referred to as “Cella”).
Regarding claim 1, Vivi discloses A method for providing a live visualization, the method comprising (“This proposed dynamic DT architecture aims at integrating heterogeneous assets and data sources with their applications, supporting intelligent asset management, providing effective O&M management, and further bridge the gap between human relationships with buildings/cities via more intelligent, visual and sustainable channels.” Vivi [Pg.3 Sec.3]): ; committing a data representation of the physical structure to a digital twin (“Data acquisition layer is the foundation of each DT (i.e. digital twin) and collecting needed data. Due to technological advances, contactless data acquisition (e.g., RFID, QR codes, image based techniques), distributed sensor systems, wireless communication, mobile access (e.g., WiFi environment) etc. are available in markets currently. Based on different functions requirements of daily building management, techniques and systems should be connected with their physical assets and designed based on well-organised DT architecture, which is presented in Fig.1 and designed for buildings.” Vivi [Pg.4 Col.1]);
transmitting the digital twin to a remote device (“Transmission layer mainly aims at transforming collected data to the upper layers. Data would be collected from physical devices, which are attached to physical assets/spaces of a building or surrounding environment. Various communication technologies can be used in this layer, such as 5G, low-power wide area networks (LP-WAN). Among all available technologies, Wi-Fi is the well-known wireless local area network (WLAN) technology and widely used one.” Vivi [Pg.4 Col.1]); receiving an update to the digital twin from the remote device, producing an updated digital twin (“Data/model integration layer is the kernel in this architecture, including data and model storing, analysing, integrating, processing, and AI-supported decision-making functions etc. (Glaessgen and Stargel, 2012). In this architecture, real-time data analysis and processing functions would update as-is conditions building assets or environments (including work orders, up-to-date maintenance information, status).” Vivi [Pg.4 Col.2]); and rendering, , a graphical representation of the updated digital twin (“Visualised data/model managing frameworks can achieve dynamic and effective data management. Furthermore, intelligent functions (e.g., AI, machine learning modules) provide operating advanced decision-making management (e.g., energy usage control, space utilization and workplace design) [ ] The application layer is the top and implementation layer of the dynamic DT architecture that interacts with facility mangers and provides services for users.” Vivi [Pg.4 Col.2]).
Vivi fails to specifically disclose receiving, via a first user interface scene, a user specification of a physical structure and rendering, via a second user interface scene, a graphical representation of the updated digital twin.
However, Cella discloses receiving, via a first user interface scene, a user specification of a physical structure (“Referring to FIG. 160 [ ] Industrial entity digital twins 13734 may include, link or connect to, or integrate with a variety of interfaces and dashboards 13738, such as ones configured for specific workflows, roles, and users. For example, dashboards and interfaces may be configured for workers who will interact with specific machines (such as where the digital twin is used for training, workflow guidance, diagnosis of problems, and the like)” Cella [P.1870]), rendering, via a second user interface scene, a graphical representation of the updated digital twin (“Referring to FIG. 160 [ ] an industrial digital twin 13734 may show a point cloud view of the industrial setting 28720 (which, in embodiments, may be augmented, such as using 3D mapping, AR or VR systems) with relevant data collection elements presented in the point cloud view along with the point cloud. Many examples are available, such as highlighting (such as by color or motion) in the digital twin 13734, areas of the point cloud where systems are vibrating in a way that is out of the normal range (such as where severity units, as discussed elsewhere herein, exceed a threshold). Industrial entity digital twins 13734 may include, link or connect to, or integrate with a variety of interfaces and dashboards 13738, such as ones configured for specific workflows, roles, and users.” Cella [P.1870])
Vivi and Cella are analogous art as they relate to digital twin data collection and processing systems that integrate multiple sensor inputs from physical systems. Vivi discloses “a system architecture for developing dynamic DTs in building levels for integrating heterogeneous data sources, support intelligent data query, and provide smarter decision-making processes [ ] This architecture is brought to life through the development of a dynamic DT demonstrator of the West Cambridge site of the University of Cambridge. Specifically, this demonstrator integrates an as-is multi-layered IFC Building Information Model (BIM), building management system data, space management data, real-time Internet of Things (IoT)-based sensor data, asset registry data, and an asset tagging platform” [Abstract]. And Cella discloses “[u]ser interfaces configured to provide a list of conditions of interest to at least one entity [ ] and to select a condition of interest in response to a user selection from the list of conditions; and a controller configured to determine a reduced dimensionality view of the data in response to a determined structure in the data and further in response to the selected condition of interest. The reduced dimensionality view further comprises a plurality of highlighted graphical elements representing sensors from the plurality of input sensors” [Abstract].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vivi’s digital twin system architecture with Cella’s graphical user interfaces in order to provide “monitoring information to a human user” Cella [P.1866].
Regarding claim 2, Vivi further discloses wherein: the update comprises at least one live value from a real-world environment represented by the digital twin, and the graphical representation comprises a graphical representation of the at least one live value (“In this DT development, two main proposed applications
were designed, including as-is conditions monitoring and future performance prediction (shown in Figure 6)” Vivi [Pg.6 Sec.4.4]).
PNG
media_image1.png
454
576
media_image1.png
Greyscale
Regarding claim 3, the method of claim 2, Vivi further discloses wherein the at least one live value comprises at least one live measurement (“WSN refers to a collection of distributed and dedicated sensors for monitoring and recording conditions of environments and equipment (Lewis, 2004). The sensors in WSNs are called nodes and they measure environmental conditions such as indoor temperature and relative air humidity, and equipment conditions such as component vibration, surface temperature
and speed of the rotating parts. In addition to the sensor nodes, WSNs consist of gateway nodes that act as the bridge be-tween the local sensors and the remote applications such as cloud hosted databases and online web pages that visualise data.” Vivi [Pg.5 Col.1 P.2]), the method further comprising: using the at least one live measurement to interpolate measurement values across the physical structure, wherein the graphical representation comprises a graphical overlay representing the measurement values across the physical structure (“In this DT development, two main proposed applications were designed, including as-is conditions monitoring (i.e. live measurement) and future performance prediction (i.e. measurement interpolation) (shown in Figure 6).” Vivi [Pg.6 Sec.4.4]).
PNG
media_image1.png
454
576
media_image1.png
Greyscale
Regarding claim 4, the method of claim 3, Vivi discloses further comprising ; wherein using the at least one live measurement to interpolate measurement values across the physical structure comprises associating a measurement of the at least one live measurement with the device location (“Table 1 (see below) shows the types of sensors used in the WSN deployed at the IfM building and their functions.” Vivi [Pg.5 Col.1 P.3]. The live measurements are interpreted to be associated with device location because “This command contains the unique device identifiers (UDIDs) of the sensors a gateway needs to be connected with. After the initialisation phase, the sensor nodes are capable of monitoring environmental and equipment conditions, and up-load data over RF to the gateway nodes.” Vivi [Pg.5 Col.2 Ln.22).
PNG
media_image2.png
1025
581
media_image2.png
Greyscale
Vivi fails to specifically disclose receiving a user specification of a measurement device location relative to the physical structure.
However, Cella discloses receiving a user specification of a measurement device location relative to the physical structure (“The visual element may facilitate user access to information about the sensor, such as location [ ] The visual element may be determined by associating sensor identification information received from a sensor with information, such as a sensor map, that correlates sensor identification information with physical location in the environment. The information may appear in the visualization in response to the visual element representing the sensor being selected, such as by a user positioning a cursor on the sensor visual element.” Cella [P.0607]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vivi’s digital twin system architecture with the capability for a user to specify a measurement device location in order to “facilitate user access to information about the sensor (i.e. measurement device)” Cella [P.0607].
Regarding claim 5, Vivi further discloses wherein: the digital twin comprises a set of functions modeling a behavior associated with the physical structure (“DT (i.e. digital twin) is a digital model, which is a dynamic representation of an asset and mimics its real-world behaviours” Vivi [Pg.2 Col.2 P.3]. Mimicking real-world behaviors is interpreted as a set of functions modeling a behavior.); the update comprises a modification of at least one function of the set of function; the method further comprises performing a simulation of the behavior using the set of functions as modified “Different types of digital models/model (i.e. functions) can be used for different purposes in DTs. For instance, energy model can be used for energy simulation and agent-based model would model scenarios and support decision. When a DT at building levels is designed, a pre-defined modelling processes (i.e. functions) and types are required to be defined (i.e. as modified)” Vivi [Pg.4 Col.2 P.1]); and
the graphical representation comprises a display of a result of the simulation (“In this DT development, two main proposed applications were designed, including as-is conditions monitoring and future performance prediction (i.e. simulation) (shown in Figure 6).” Vivi [Pg.6 Sec.4.4]).
PNG
media_image1.png
454
576
media_image1.png
Greyscale
Regarding claim 6, the method of claim 5, Vivi further discloses further comprising: identifying a weather data source associated with a location of a real-world environment associated with the physical structure (“Data acquisition from environments and physical assets is a fundamental requirement of the proposed system architecture of the dynamic DT. [ ] an IoT-enabled wireless sensor network (WSN) for data acquisition. [ ] WSN refers to a collection of distributed and dedicated sensors
for monitoring and recording conditions of environments and equipment [ ] The sensors in WSNs are called nodes and they measure environmental conditions such as indoor temperature and relative air humidity” Vivi [Pg.5 Col.1 P.1-2]. As shown in claim 4, Vivi discloses unique device identifiers (UDIDs) of the sensors, thus enabling identification of a weather data source, i.e. sensor. Temperature and relative humidity are interpreted as weather data.), wherein performing the simulation comprises simulating an effect of weather on the physical structure using weather data from the weather data source (“In the as-is asset management monitoring function, temperature monitoring functions are designed to keep working environments comfort. In the predictive maintenance function, the health of assets is monitored and analysed in the
real-time to forecast the remaining useful life of the asset and in turn to enable effective maintenance planning” Vivi [Pg.7 Col.1 P.1]).
Regarding claim 7, Vivi further discloses wherein the remote device is an on-premise controller device installed at a real-world environment associated with the physical structure (“the developed DT is integrated with the data acquired from Building Management System (BMS), asset management system (AMS) (Planet system used in Cambridge), space management system (SMS) (MiCAD system used in Cambridge), and ‘real-time’ sensor dataset. The ‘real-time’ sensor datasets are integrated in a cloud-based computing platform (using AWS in this project). The BMS, AMS and SMS are all computer-aided control systems installed in buildings and they control the mechanical and electrical systems (e.g. power systems, heat ventilation and
air conditioning (HVAC) systems and security systems), provide services of assets and manage space utilisation separately for various buildings in Cambridge site (see Figure
5).” Vivi [Pg.6 Sec.4.4]).
PNG
media_image3.png
310
488
media_image3.png
Greyscale
Regarding claim 8, Vivi discloses A device for providing a live visualization, the device comprising: a memory capable of storing a digital twin (“Data/model integration layer is the kernel in this architecture, including data and model storing” Vivi [Pg.4 Col.2 P.2]); a communication interface (“The Monnit Mine API is an interface that allows custom developed applications to retrieve data from the Monnit gateways.” Vivi [Pg.5 Col.2 Ln.17]); ; and a processor (“In this architecture, real-time data analysis and processing functions (i.e. requires a processor) would update as-is conditions building assets or environments” Vivi [Pg.4 Col.2 P.2])
Vivi fails to specifically disclose a user interface.
However, Cella discloses a user interface (“Referring to FIG. 160 [ ] Industrial entity digital twins 13734 may include, link or connect to, or integrate with a variety of interfaces and dashboards 13738” Cella [P.1870])
The remaining limitations recite substantially the same subject matter as claim 1 and are rejected under similar rationale. Examiner maintains the same rationale for Vivi/Cella combination as claim 1.
Claims 9-14 recite substantially the same subject matter as claims 2-7, respectively, and are rejected under similar rationale.
Regarding claim 15, Vivi fails to specifically disclose A non-transitory machine-readable medium encoded with instructions for execution by a processor for providing a live visualization, the non-transitory machine-readable medium.
However, Cella discloses A non-transitory machine-readable medium encoded with instructions for execution by a processor for providing a live visualization, the non-transitory machine-readable medium (“In embodiments, one or more non-transitory computer-readable media comprising computer executable instructions that, when executed, may cause at least one processor to perform actions comprising:” Cella [P.0996], “A smart band assessment portal may also include visualization of candidate sensors” Cella [P.1220]).
The remaining limitations recite substantially the same subject matter as claim 1 and are rejected under similar rationale. Examiner maintains the same rationale for Vivi/Cella combination as claim 1.
Claims 16-20 recite substantially the same subject matter as claims 2-6, respectively, and are rejected under similar rationale.
Conclusion
The prior art made of record, listed on form PTO-892, and not relied upon is
considered pertinent to applicant's disclosure:
Rakshit (Digital Twin Workflow Simulation – US Pat. No 11556449 B2). “Systems, methods and computer program products for simulating workflows and activities of physical assets using digital twin models. User-defined simulations are performed by selectin digital twin components being analyzed during the simulation, concentrating the analysis on the selectively defined components and bypassing components that will not be simulated.” [Abstract]
Cella et al. (User Interface For Industrial Digital Twin System Analyzing Data To Determine Structures With Visualization Of Those Structures With Reduced Dimensionality – US Pub. No 2023/0196230 A1). “The reduced dimensionality view further comprises a graphical element representing at least one of: mechanical portions of a machine of the industrial environment, or a sensor from the plurality of input sensors that provided data; and providing the reduced dimensionality view to a user interface that is associated with at least one entity” [Abstract]
Lu, Qiuchen, et al. "Developing a digital twin at building and city levels: Case study of West Cambridge campus." Journal of management in engineering 36.3 (2020): 05020004. “this paper presents a system architecture for DTs, which is
specifically designed at both the building and city levels” [Abstract]
Ferrigno, E., and G. A. Barsola. "3d real time digital twin." SPE Latin America and Caribbean Petroleum Engineering Conference. SPE, 2023. “The potential applications of Digital Twins in the Oil & Gas industry are numerous. They can optimize operational efficiency, improve asset management, and reduce risks.” [Pg.15 Conclusion]
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Anthony Chavez whose telephone number is (571) 272-1036. The examiner can normally be reached Tuesday - Thursday, 10 a.m. - 5 p.m. ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Renee Chavez can be reached at (571) 270-1104 The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANTHONY CHAVEZ/ Examiner, Art Unit 2186
/RENEE D CHAVEZ/Supervisory Patent Examiner, Art Unit 2186
