Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
Applicants amendment filed on 11/20/2025 are entered. Claims 1-27 are pending in this application of which claims 1, 17, and 21 are independent.
Response to Arguments
Applicants arguments in view of amendments filed on 11/20/2025 have been fully considered and the examiner’s response is as follows:
Applicants arguments, Pages 10-17, regarding 35 U.S.C 101 rejections are considered but are moot because the new grounds of rejection, necessitated by applicant’s amendments as shown below. Therefore, the 35 U.S.C 101 rejections are modified as seen below.
Applicants arguments, Pages 18-23, regarding 35 U.S.C 103 rejections are considered but are moot because the new grounds of rejection, necessitated by applicant’s amendments, relies on additional prior art as shown below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent thereof, subject to the conditions and requirements of this title.
Claims 1, 3-5, 7-19, 21, 22, and 25-27 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of a mental process or mathematical concept without significantly more.
Claim 1.
STEP 1: Yes. The claim recites a “apparatus” which is a manufacture.
STEP 2A Prong One:
The claim recites multiple mental processes.
(c) conforming the real-time data to match the data schema;
This describes an observation, evaluation, judgment or opinion that can be done in the mind or with the aid of pen and paper. In this case an evaluation regarding if the schema is conforming.
(e) receiving, from the cloud-based digital space, instructions for controlling physical operation of the apparatus from the cloud-based digital space, the instructions having been derived from processing of the conforming data in the cloud-based digital space;
This describes an observation, evaluation, judgment or opinion that can be done in the mind or with the aid of pen and paper. In this case an evaluation regarding the conforming data from the digital space.
(h) reiterating operations (c)-(f) based on the second real-time data to achieve a real-time feedback loop for controlling the physical operation of the apparatus.
This limitation would re-invoke the analysis of the repeated limitations. So, it would once again reiterate the mental process.
STEP 2A PRONG TWO: The claim does not integrate the exception into a practical application.
STEP 2B: The claim does not recite an inventive concept or significantly more than the exception.
one or more sensors;
MPEP 2106.05(f) – This is mere instructions to apply the abstract idea using generic computer components.
one or more processors; and
MPEP 2106.05(f) – This is mere instructions to apply the abstract idea using generic computer components.
memory storing machine-readable instructions that, when executed by the one or more processors, cause the apparatus to perform operations comprising:
MPEP 2106.05(f) – This is mere instructions to apply the abstract idea using generic computer components.
(a) gathering, from the one or more sensors, real-time data regarding a physical object in a traffic region;
MPEP 2106.05(g) – This is pre solution data gathering activity.
(b) fetching, from a cloud-based digital space storing separate digital twins for human actors in the traffic region, vehicles in the traffic region, and traffic devices in the traffic region, a data schema comprising a first digital twin corresponding to the physical object;
MPEP 2106.05(g) – This is pre solution data gathering activity.
MPEP 2106.05(h) – This also just links the field of use of digital twin storage.
(d) transmitting the conforming real-time data to the cloud-based digital space;
MPEP 2106.05(g) – This is post solution data transmitting.
(f) controlling the physical operation of the apparatus based on the received instructions;
This falls under MPEP 2106.05(f) of apply it because it is recited at a high level of generality and does not give the means of controlling.
(g) gathering, from the one or more sensors, second real-time data regarding the physical object after performance of operation (f); and
MPEP 2106.05(g) – This is pre solution data gathering activity.
Conclusion: Claim 1 is directed to mental processes, not integrated into a practical application and lacks an inventive concept. Therefore, it is ineligible under 35 U.S.C 101.
Regarding Claims 3, 9, 12, 15, and 25:
These claims merely add pre solution data gathering MPEP 2106.05(g) or they narrow the input data type and link the field of use. MPEP 2106.05(h). This does not integrate the judicial exception into practical application. The claims do not resolve the issues from the claims they depend upon.
Regarding Claims 4, 8, 10-11, 13-14, and 16:
These claims merely narrow the abstract idea by specifying how the data is conformed and pre-processed, what the processing of the data comprises (storing, modeling, simulating, machine learning and prediction), and determining the availability of the digital twins. Thus, the claims remain as a mental process. MPEP 2106.05(a). This does not integrate the judicial exception into practical application. The claims do not resolve the issues from the claims they depend upon.
Regarding Claims 5 and 7:
These claims merely describe the contes of the cloud based digital space (a data lake, microservices, stored historical data), which generally links the abstract idea to a technological environment. MPEP 2106.05(h). This does not integrate the judicial exception into practical application. The claims do not resolve the issues from the claims they depend upon.
Regarding Claim 22:
These claims merely specify that the physical object and the apparatus comprise a traffic device, without reciting any operation of the traffic device, which links the field of use. MPEP 2106.05(h). This does not integrate the judicial exception into practical application. The claims do not resolve the issues from the claims they depend upon.
Claim 17.
STEP 1: Yes. The claim recites a “system” which is a manufacture.
STEP 2A Prong One:
This claim recites multiple mental processes.
the communications layer transmits instructions that have been derived from processing of the transmitted real-time data by the respective digital twin to the respective physical object; and
This describes an observation, evaluation, judgment or opinion that can be done in the mind or with the aid of pen and paper. In this case an evaluation because “processing of the transmitted real-time data” to derive instructions is analyzing the collected sensor data (evaluation) and deciding what the physical object should do based on that analysis (judgement) both practicable in the human mind, as a traffic engineer could review the same readings and decide the same actions with pen and paper.
STEP 2A PRONG TWO: The claim does not integrate the exception into a practical application.
STEP 2B: The claim does not recite an inventive concept or significantly more than the exception.
a cloud-based platform hosting separate digital twins corresponding to separate physical objects in a traffic region, the separate physical objects comprising human actors, vehicles, and traffic devices;
MPEP 2106.05(f) – This is mere generic computer components of a cloud hosting/storing.
one or more sensors;
MPEP 2106.05(f) – This is generic computer components used to apply the abstract idea.
a communications layer communicatively connecting the one or more digital twins to the physical objects, wherein:
MPEP 2106.05(f) – This is generic computer components used to connect the digital twins.
the communications layer transmits real-time data obtained from the one or more sensors regarding a respective physical object to a respective digital twin corresponding to the respective physical object; and
MPEP 2106.05(g) – This is extra solution activity of transmitting data.
the physical objects communicatively connected to the cloud-based platform via the communications layer, wherein the respective physical object performs physical operations in accordance with the transmitted instructions to achieve the end-to-end framework.
MPEP 2106.05(f) – This is mere instructions to apply it because performing unspecified “physical operations” in accordance with the instructions is recited at a high level of generality and does not give the means of performing the operations.
Conclusion: Claim 17 is directed to mental processes, not integrated into a practical application and lacks an inventive concept. Therefore, it is ineligible under 35 U.S.C 101.
Regarding Claim 18:
These claims merely describe the contents of the respective digital twin (a data lake and one or more microservices), which generally links the abstract idea to a technological environment. MPEP 2106.05(h). This does not integrate the judicial exception into practical application. The claims do not resolve the issues from the claims they depend upon.
Regarding Claim 19:
These claims recites that the respective physical object comprises “a traffic signal or an electronic traffic sign” in the alternative and under BRI the claim is satisfied by an embodiment adjusting a visual display message, which falls as insignificant extra solution activity. MPEP 2106.05(g). This does not integrate the judicial exception into practical application. The claims do not resolve the issues from the claims they depend upon.
Regarding Claims 26 and 27:
This claim merely narrows the input data type by specifying sensor species and their locations, and narrows the types of physical objects, which links the field of use. MPEP 2106.05(g). MPEP 2106.05(h). This does not integrate the judicial exception into practical application. The claims do not resolve the issues from the claims they depend upon.
Claim 21.
STEP 1: Yes. The claim recites a “apparatus” which is a manufacture.
STEP 2A Prong One:
The claim recites multiple mental processes.
conforming the real-time data to match the data schema;
This describes an observation, evaluation, judgment or opinion that can be done in the mind or with the aid of pen and paper. In this case an evaluation regarding if the schema is conforming.
receiving, from the cloud-based digital space, instructions for controlling physical operation of the apparatus from the cloud-based digital space, the instructions having been derived from processing of the conforming data in the cloud-based digital space;
This describes an observation, evaluation, judgment or opinion that can be done in the mind or with the aid of pen and paper. In this case an evaluation regarding the conforming data from the digital space.
STEP 2A PRONG TWO: The claim does not integrate the exception into a practical application.
STEP 2B: The claim does not recite an inventive concept or significantly more than the exception.
one or more sensors;
MPEP 2106.05(f) – This is mere instructions to apply the abstract idea using generic computer components.
one or more processors; and
MPEP 2106.05(f) – This is mere instructions to apply the abstract idea using generic computer components.
memory storing machine-readable instructions that, when executed by the one or more processors, cause the apparatus to perform operations comprising:
MPEP 2106.05(f) – This is mere instructions to apply the abstract idea using generic computer components.
gathering, from the one or more sensors, real-time data regarding a physical object in a traffic region;
MPEP 2106.05(g) – This is pre solution data gathering activity.
fetching, from a cloud-based digital space storing separate digital twins for human actors in the traffic region, vehicles in the traffic region, and traffic devices in the traffic region, a data schema comprising a first digital twin corresponding to the physical object;
MPEP 2106.05(g) – This is pre solution data gathering activity.
MPEP 2106.05(h) – This also just links the field of use of digital twin storage.
transmitting the conforming real-time data to the cloud-based digital space;
MPEP 2106.05(g) – This is post solution data transmitting.
controlling the physical operation of the apparatus based on the received instructions;
This falls under MPEP 2106.05(f) of apply it because it is recited at a high level of generality and does not give the means of controlling.
Conclusion: Claim 21 is directed at mental processes, not integrated into a practical application and lacks an inventive concept. Therefore, it is ineligible under 35 U.S.C 101.
Claims 2, 6, 20, 23, and 23 are found to be eligible under 35 USC 101. Independent claims may be eligible if any of the eligible dependent claim is incorporated to the independent claims to overcome the 35 USC 101 rejections.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claim(s) 1-3, 5-10, 14, 22-23, and 25 are rejected under are rejected under 35 U.S.C. 103 as being unpatentable over KUMAR (A novel digital twin-centric approach for driver intention prediction and traffic congestion avoidance), herein KUMAR, in view of WANG (Architectural Design Alternatives Based on Cloud/Edge/Fog Computing for Connected Vehicles), herein WANG, and in further view of CELLA (WO 2021222384 A1), herein CELLA.
Claim 1
Claims 1 is rejected because KUMAR teaches
An apparatus comprising:
one or more sensors:
one or more processors; and
memory storing machine-readable instructions that, when executed by the one or more processors, cause the apparatus to perform operations comprising:
“Edge analytics is gaining a lot of momentum these days. With the edge devices being embedded with sufficient processing, storage, and I/O power, they are individually as well as collectively readied to participate in the mainstream computing.”. (Section 1.32).
KUMAR also teaches (a) gathering, from the one or more sensors, real-time data regarding a physical object in a traffic region KUMAR ([Section 1.32 Relevant and real-time vehicle and traffic information through edge clouds] “The road infrastructure is also fitted with various cameras, sensors, Wi-Fi gateways, and other electronics to enable data gathering, aggregation, and communication.”)
“These devices can collect and process any incoming data and emit useful information in real time. The shared information can help the various participating sensors and actuators to plan and indulge in performing their activities with cognition, clarity, and confidence. Vehicles on the road are being stuffed with a number of purpose-specific and agnostic sensors and actuators to proactively and preemptively capture all the right and relevant data.”. (Section 1.32)
KUMAR also teaches (b) fetching, from a cloud-based digital space … , a data schema comprising a first digital twin corresponding to the physical object KUMAR ([Section 1.3.3 Digital twin] “Now, the idea of the digital twin is to have a corresponding virtual image for a physical asset at the ground. That is, the virtual entity has all the structural as well as behavioral properties as the corresponding physical element. The digital twin is to have a dynamic virtual/digital representation for each of the physical systems. This cloud-based virtual representation helps to gain a better and deeper understanding of all kinds of ground-level physical, mechanical, electrical, and electronics systems and how they team up to collaborate, corroborate, and correlate with one another in the vicinity.”)
KUMAR also teaches (c) conforming the real-time data to match the data schema KUMAR ([Section 2.3 Virtual vehicle (VV) model] “There are some key solutions needed to construct the VV model, including the technology to translate the different formats of incoming data, such as data from the roadside sensor and the network data from the intelligent transportation systems (ITS). To meet these technological needs, we plan to introduce the ontology needed to model the enormous dataset that the proposed project will handle. Similarly, the VV model requires technology to accurately describe the driver's knowledge. In the VV model, knowledge provides the matching rules, and the VV makes decisions according to the result of fact matching. However, different drivers may have different knowledge, and drivers' knowledge may change from interaction with other VV s. The matching rule must be accurate in describing the driver's knowledge, mining the user's intentions, and capturing them in the VV information space.”)
KUMAR also teaches (d) transmitting the conforming real-time data to the cloud-based digital space KUMAR ([Section 1.3.6 Blockchain technology] “Here, collecting and sending data to remote cloud servers to arrive at competent decisions are found inappropriate for real-time and low-latency applications. However, the edge data can be aggregated and transmitted to powerful cloud servers casually in batches to have a historical diagnostic and deterministic analytics at a later point in time.”)
KUMAR also teaches (e) receiving, from the cloud based digital space, instructions for controlling physical object from the cloud-based digital space KUMAR ([Section 2.2 Edge analytics-based virtual vehicle (W) networks] “Since the proposed VV model makes decisions, it needs detailed driver information, such as preferences as to which lane the driver or automated vehicle (physical object) is likely to select and route plans (controlling actuation) that are together considered as 'intention'. The VV model can obtain scalable, real-time driver intention data (receiving instructions), both captured locally from the vehicle, edge cloud, and the remote cloud (from the cloud-based digital space); by processing them in the edge and by interacting with other VV s, VV s can predict other drivers' intentions in such a way that this intention information can be used for a variety of scenarios. The VV is a virtual state of the vehicle and driver, which is processed in the edge and exists in the cloud (from the cloud-based digital space).”)
Kumar also teaches
(f) controlling the physical operation of the apparatus based on the received instructions;
“The physical vehicle or traffic controller behaves like an actuator on the road, acting upon directions from the VVN to the edge.”. (Section 2.2 Pg. 204).
“The shared information can help the various participating sensors and actuators to plan and indulge in performing their activities with cognition, clarity, and confidence.”. (Section 1.3.2 Pg. 201).
“The other requirements include detecting the presence of pedestrians and cyclists crossing the street or road to proactively issue "slow down" warnings to incoming vehicles and instantaneously modifying its own cycle to prevent collisions.”. (Section 1.3.2 Pg. 201).
Kumar also teaches
(g) gathering, from the one or more sensors, second real-time data regarding the physical object after performance of operation (f); and
“These devices can collect and process any incoming data and emit useful information in real time. The shared information can help the various participating sensors and actuators to plan and indulge in performing their activities with cognition, clarity, and confidence. Vehicles on the road are being stuffed with a number of purpose-specific and agnostic sensors and actuators to proactively and preemptively capture all the right and relevant data.”. (Section 1.32)
“Another point here is that data flows in streams. Thus, all kinds of discrete/simple, as well as complex events need to be precisely and perfectly captured and combined to be subjected to a bevy of investigations to complete appropriate actions.”. (Section 1.3.6 Pg. 202).
“These devices can collect and process any incoming data and emit useful information in real time.”. (Section 1.3.2 Pg. 201).
Kumar in view of Cella teaches
(h) reiterating operations (c)-(f) based on the second real-time data to achieve a real-time feedback loop for controlling the physical operation of the apparatus.
Kumar and CELLA teach the reiterating operations c to f that have been mapped in this rejection. Kumar teaches performing the operations as a real time feedback loop because the traffic system has to learn and act instantaneously. “… instantaneously modifying its own cycle to prevent collisions.”. (Section 1.3.2 Pg. 201). “The noteworthy factor here is that the smarter traffic system has to learn, decide, and act instantaneously to avert any kind of accidents. That is, the real-time reaction is the crucial need and, hence, the concept of edge clouds out of edge devices for collaboratively collecting different data and processing them instantaneously to spit out insights is gaining widespread and overwhelming momentum.”. (Section 1.3.6 Pg. 202).
KUMAR does not explicitly teach but CELLA teaches
storing separate digital twins for human actors in the traffic region, vehicles in the traffic region, and traffic devices in the traffic region
“According to aspects of the present disclosure, a system for monitoring navigational route data through a transportation system has real-world elements disposed therein includes a digital twin datastore and one or more processors. The digital twin datastore includes a transportation system digital twin corresponding to the transportation system and a worker digital twin corresponding to a respective worker of a set of workers within the transportation system. The one or more processors are configured to maintain, via the digital twin datastore, the transportation system digital twin to include contemporaneous positions for the set of workers within the transportation system, monitor movement of each worker in the set of workers via a sensor array, determine, in response to detecting movement of the respective worker, navigational route data for the respective worker, update the transportation system digital twin to include indicia of the navigational route data for the respective worker, and move the worker digital twin along a route of the navigational route data.”. (0132).
“… the digital twin 60136 may be configured to communicate with a user of the vehicle 60104 via a set of communication channels, such as speech, text, gestures, and the like. In embodiments, the digital twin 60136 may be configured to communicate with digital twins of other entities including digital twins of users, nearby vehicles, traffic lights, pedestrians and so on.”. (0734).
“The integrated vehicle and driver twin 61404 may be created, such as by integrating a digital twin of the vehicle 60104 with the digital twin of the driver. In embodiments, such an integration may be achieved by normalizing the 3D models used by each of the twins to represent a consistent scale, and linking via APIs to obtain regular updates of each twin (such as current operating states of the vehicle and current physiological state, posture, or the like of the driver).”. (0778).
It would have been obvious to store separate digital twins for the human actors, vehicles, and traffic devices in Kumar’s cloud-based system in order to combine the vehicle state with the driver’s physiological state to configure the vehicle response and improve safety. CELLA “In the example scenario, the integrated vehicle and the driver twin 61404 may determine that the driver’s state is “drowsy” based on an input from a set of IR cameras tracking the pupil size and eyelid movement and a set of sensors 60108 tracking the (sagging) posture and (slower) reaction time of the driver 60244. The twin may also determine that the vehicle is “unstable” based on the tracking of speed, lateral position, turning angles and moving course. The integrated vehicle and driver twin 61404 may communicate with the driver 60244 alerting the driver 60244 about the potential safety hazards driving in such a state. Alternatively, the integrated vehicle and the driver twin 61404 may take one or more steps to wake the driver like switching on music or turning up the volume and/or ensure driver and vehicle safety by switching the vehicle into an autopilot or autosteer mode.”. (0781).
KUMAR and CELLA do not explicitly teach the instructions having been derived from processing of the conforming data in the cloud-based digital space.
However, WANG teaches the instructions having been derived from processing of the conforming data in the cloud-based digital space. WANG ([Abstract] “Computation offloading techniques, such as cloud, edge, and fog computing, can help CVs process computation intensive and large-scale computing tasks.”) See also WANG ([Section III. A. Service Requirements] “In addition, as data is increasingly an important asset to each country, it is necessary to follow the local rules and regulations imposed by each nation. For example, some countries may restrict some type of data to physically stay in their territories depending on how they are used. If a vehicle is used for a certain that falls in such a restrictive category, the data transmission should strictly follow the local regulation. These days, OEMs typically do not have that level of customization as it increases the manufacturing cost significantly. However, this situation will arise as more data is shared with remote cloud or vehicles, so it is necessary to consider a system architecture design to enable data to be transmitted conforming such local regulation via support from either in-vehicle system or infrastructure.”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of WANG with KUMAR-CELLA as the references deal with hardware development systems are often used to debug and test the operation of a processing device or other devices. WANG would modify the KUMAR-CELLA combination wherein the instructions having been derived from processing of the conforming data in the cloud-based digital space. The benefits of doing so presents a holistic comparison among different cloud/edge/fog computing architectures for CVs based on functional requirements of CV systems, including advantages, disadvantages, and research challenges and proposes several open research issues of designing cloud/edge/fog computing architectures for CVs. (WANG [Abstract]).
Claim 2
Claims 2 is rejected because the combination of KUMAR, CELLA, and WANG teaches the claim 1 limitations.
CELLA and Wang do not explicitly teach but Kumar teaches
wherein the apparatus comprises a vehicle;
“Vehicles on the road are being stuffed with a number of purpose-specific and agnostic sensors and actuators to proactively and preemptively capture all the right and relevant data.”. (Section 1.3.2 Pg. 201).
the physical object comprises the vehicle or a human actor in the vehicle; and
“Sensor data would include the current GPS coordinates, current speed, and average speed of the vehicle.”. (Section 2.3 Pg. 204).
controlling the physical operation of the vehicle comprises adjusting a trajectory of the vehicle.
“VV s for driverless vehicles can make decisions about path planning and interaction with other vehicles, while VVs for non-autonomous vehicles can help drivers make decisions by mining other drivers' intentions.”. (Section 2.2 Pg. 203-204).
Claim 3
Claims 3 is rejected because the combination of KUMAR, CELLA, and WANG teaches the claim 1 limitations.
KUMAR teaches wherein the gathering of the real-time data further comprises at least one of obtaining data from one or more monitoring devices associated with the physical object, and receiving data from one or more vehicle-to-anything (V2X) communications regarding the physical object KUMAR ([Section 1.3 Embarking on the next-generation intelligent transport systems (ITS)] “Traffic lights (physical object) have become very prominent and pervasive in urban areas for enabling smooth flow of pedestrians as well as vehicle drivers. There are high-fidelity video cameras in plenty along the roads, expressways, tunnels, etc. to activate and accelerate a variety of real-time tasks for pedestrians, traffic police, and vehicle drivers. Wireless access points such as Wi-Fi, 3G, 4G, roadside units, and smart traffic lights (gathering data from one or more monitoring devices) have been deployed along the roads. Vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) (one or more vehicle-to-anything (V2X) communications) interactions enrich the application of this scenario. All kinds of connected vehicles (phyiscal) and transport systems need actionable insights in time to derive and deliver a rich set of context-aware services (gathering data). Safety is an important factor for car and road users and there are additional temporal as well as spatial services being worked out. With driverless
cars under intense development and testing, insights-driven decisions and knowledge-centric actions are very vital for next-generation transports.”) Accordingly, claim 3 is rejected based on the combination of these references.
Claim 5
Claims 5 is rejected because the combination of KUMAR, CELLA, and WANG teaches the claim 1 limitations.
KUMAR teaches wherein the first digital twin comprises a data lake and one or more microservices, the application of which influence operation of at least one of the physical object or the apparatus KUMAR ([Section 3 Experimentation and Results] “The technologies and tools used (digital twin comprises a data lake) are software-defined cloud environments, digital twin, artificial intelligence (AI) (machine and deep learning algorithms), data lake, real-time data capture, storage, processing, analytics, decision-making and action through loT edge analytics, edge and public cloud integration, etc. (one or more microservices). By leveraging the proven, potential, and promising technologies, we arrive at a framework (application), which guarantees (of which influence operation) the much-needed accuracy in decision-making and subsequent actions. The digital twin is the virtual and logical representation of physical assets (physical object) and processes. There is a direct communication between physical and digital systems to collect the latest data. Machine and deep learning algorithms are capable of analyzing big data in real time to extract actionable insights in time and the discovered knowledge gets disseminated to the particular junctions and locations to streamline the traffic movement in a smooth manner so as to avoid time wastage in those places.”) Accordingly, claim 5 is rejected based on the combination of these references.
Claim 6
Claims 6 is rejected because the combination of KUMAR, CELLA, and WANG teaches the claim 5 limitations.
KUMAR teaches wherein the processing of the conforming real-time data comprises at least one of storing the conforming data in the digital lake KUMAR ([Section 1.3.4 The machine and deep learning methods] “There are a number of standardized big data analytics solutions in the form of enabling tools and integrated platforms. These analytical solutions typically
perform batch processing (processing), which is not liked by many. We are tending toward a real-time analytics of big data (conforming data).”) See also KUMAR ([Section 1.3.5 Data lake for transport and traffic data heaps] “Data lakes are becoming commonplace across industrial verticals. All kinds of multi-structured data get stocked in a centralized place to be found, accessed, and used for extracting useful insights out of data heaps. Data scientists are using data lakes greatly in their everyday job. For setting up and sustaining insights-driven transport management systems (, data lakes are essential. We have object storage facilities in cloud environments (at least one of storing the conforming data) to facilitate the realization of data lakes (in the digital lake)… The noteworthy factor here is that the smarter traffic system
has to learn, decide, and act instantaneously to avert any kind of accidents. That is, the real-time reaction is the crucial need and, hence, the concept of edge clouds out of edge devices for collaboratively collecting different data (conforming data) and processing them (processing conforming data) instantaneously to spit out insights is gaining widespread and overwhelming momentum.”)
KUMAR also teaches modeling the physical object using the first digital twin based on the conforming real-time data KUMAR ([Section 2.2 Edge analytics-based virtual vehicle (VV) networks] “Since the proposed VV model makes decisions, it needs detailed driver information, such as preferences as to which lane the driver or automated vehicle is likely to select and route plans that are together considered as 'intention'. The VV model can obtain scalable, real-time driver intention data, both captured locally from the vehicle (modeling the physical object), edge cloud, and the remote cloud; by processing them in the edge and by interacting with other VV’s, VV’s can predict other drivers' intentions (using the digital twin) in such a way that this intention information (based on the conforming data) can be used for a variety of scenarios. The VV is a virtual state of the vehicle and driver (modeling the physical object), which is processed in the edge and exists in the cloud.”)
KUMAR also teaches simulating operation of the at least one of the physical object or the apparatus using the conforming real-time data KUMAR ([Section 3 Experimentation and results] “Our preliminary experiments were driven by the data from TAPAS [55], a system that computes mobility plans (simulating the operation) for an area population (physical object), generated from information about Germans' traveling habits and the infrastructure of the areas in which they live (using the conforming data). We used the traffic simulation software SUMo [56] to generate vehicle traces (simulating the operation of the physical object) from real data (conforming data). We divided all the roads (physical objects) into 100 segments in an area of 600 m x 600 m, numbering each segment so that individual vehicles could be linked to their trace (simulating the operation). We used the following three related algorithms to compare and evaluate the effectiveness of our approach: (i) the Max Greedy algorithm, where the sensing center selects a virtual vehicle that has the highest number of non-repeated data blocks; (ii) the Min Greedy algorithm, where a virtual vehicle is selected by the sensing center if it has both the least number of non-repeated data blocks for the sensing center and the least number of data blocks compared with the last vehicles; and (iii) the random algorithm, where virtual vehicles gather data and individually transmit it to the sensing center in a random manner.”)
KUMAR also teaches performing machine learning and prediction using the conforming real-time data KUMAR ([Section 2 The proposed approach] “We have come out with a real-time and cognitive traffic congestion avoidance solution. Having studied the current lacunae in the traffic management solutions, we have come out with an advanced, extensible, and AI-inspired solution to precisely and perfectly measure (prediction) the traffic situation in real time and the driver intention by leveraging the localized fog analytics (conforming data), the power of the digital twin along with the big data processing using competent machine learning methods (performing machine learning).”) Accordingly, claim 6 is rejected based on the combination of these references.
Claim 7
Claims 7 is rejected because the combination of KUMAR, CELLA, and WANG teaches the claim 5 limitations.
KUMAR also teaches wherein the data lake further comprises stored historical conforming data related to the first digital twin KUMAR ([Section 1.3.5 Data lake for transport and traffic data heaps] “We have object storage facilities (stored historical conforming data) in cloud environments (related to digital twin) to facilitate the realization of data lakes (data lakes).”) See also KUMAR ([Section 1.3.6 Blockchain technology] “All kinds of interactions and transactions (historical conforming data) between the various participants get securely stored through the blockchain database (centralized repository data lake storing data). Thus, in the days ahead, there will be closer and tighter integration between vehicles and the fast-growing blockchain technology… Here, collecting and sending data (historical conforming data) to remote cloud servers (related to digital twin) to arrive at competent decisions are found inappropriate for real-time and low-latency applications. However, the edge data can be aggregated and transmitted to powerful cloud servers casually in batches to have a historical diagnostic and deterministic analytics (historical conforming data) at a later point in time.”) See also KUMAR ([Figure 1].) Accordingly, claim 7 is rejected based on the combination of these references.
PNG
media_image1.png
549
673
media_image1.png
Greyscale
KUMAR Figure 1 Reference
Claim 8
Claims 8 is rejected because the combination of KUMAR, CELLA, and WANG teaches the claim 7 limitations.
KUMAR also teaches wherein the processing of the conforming real-time data in the cloud-based digital space further comprises processing of the stored historical conforming data KUMAR (KUMAR [Section 2.2 Edge analytics-based virtual vehicle (VV) networks] “To address the traffic challenges, here is a viable proposal. With the availability of powerful cameras and sensors along the roads, bridges, expressways, tunnels, signals, etc. (physical objects), a massive amount of real-time as well as historical data get captured, collected, cleaned, and stocked to be crunched (processing of the stored historical conforming data)…Since the proposed VV model makes decisions, it needs detailed driver information, such as preferences as to which lane the driver or automated vehicle is likely to select and route plans that are together considered as 'intention'. The VV model can obtain scalable, real-time driver intention data (stored historical conforming data received), both captured locally from the vehicle (physical object), edge cloud, and the remote cloud (in the cloud-based digital space); by processing them (processing the stored historical conforming data) in the edge and by interacting with other VV s, VV s can predict other drivers' intentions in such a way that this intention information can be used for a variety of scenarios. The VV is a virtual state of the vehicle and driver, which is processed in the edge and exists in the cloud.”) See also KUMAR ([Figure 2].) Accordingly, claim 8 is rejected based on the combination of these references.
PNG
media_image2.png
576
358
media_image2.png
Greyscale
KUMAR Figure 2 Reference
Claim 9
Claims 9 is rejected because the combination of KUMAR, CELLA, and WANG teaches the claim 1 limitations.
The combination of KUMAR, CELLA, and WANG does not explicitly teach requesting data associated with second digital twin to be sent to the first digital twin corresponding to the physical object.
However CELLA teaches requesting data associated with another digital twin to be sent to the digital twin corresponding to the physical object CELLA ([0009] “According to some embodiments of the present disclosure, a method for updating one or more properties of one or more transportation system digital twins is disclosed. The method includes receiving a request to update (requesting data) one or more properties (data associated) of one or more transportation system digital twins (associated with another digital twin); retrieving the one or more transportation system digital twins required to fulfill the request from a digital twin datastore; retrieving one or more dynamic models required to fulfill the request from a dynamic model datastore; selecting data sources from a set of available data sources for one or more inputs for the one or more dynamic models; retrieving data from the selected data sources; running the one or more dynamic models using the retrieved data as input data to determine one or more output values; and updating the one or more properties of the one or more transportation system digital twins based on the one or more output values of the one or more dynamic models.”) See also CELLA ([0002] “The present disclosure relates to an intelligent digital twin system that creates, manages, and provides digital twins for transportation systems (corresponding to the physical object) using sensor data and other data.”)
Claim 10
Claims 10 is rejected because the combination of KUMAR, CELLA, and WANG teaches the claim 1 limitations.
KUMAR teaches determining availability of the first digital twin prior to the transmitting of the conforming real-time data to the cloud-based digital space KUMAR ([Edge analytics-based virtual vehicle (VV) networks] “To address the traffic challenges (determining availability), here is a viable proposal. With the availability of powerful cameras and sensors along the roads, bridges, expressways, tunnels, signals, etc., a massive amount of real-time as well as historical data get captured, collected, cleaned, and stocked to be crunched (to the cloud-based digital space). One of the decision-enabling factors for proactively and preemptively avoid traffic congestion and snarl (determining availability of the digital twin) is to get the drive intention. Figure 2 vividly illustrates how the driver intention is deduced (prior to the transmitting of the conforming data) from the various data collection and the digital twin, which is formed through a virtual vehicle (VV) model (cloud-based digital space). The need here is to formulate a flexible and futuristic VV model (cloud-based digital space) to enable machine and deep learning algorithms to predict the driver intention with accuracy.”) Accordingly, claim 10 is rejected based on the combination of these references.
Claim 14
Claims 14 is rejected because the combination of KUMAR, CELLA, and WANG teaches the claim 10 limitations.
The combination of KUMAR and WANG does not explicitly teach determining availability of a third digital twin corresponding to one or more other physical objects surrounding at least one of the physical object or the neighboring physical object.
However, CELLA teaches determining availability of a digital twin corresponding to one or more other physical objects surrounding at least one of the physical object or the neighboring physical object CELLA ([0732-0733] “In embodiments, digital twin 60136 of the vehicle 60104 is a virtual replication of hardware, software, and processes in the vehicle 60104 that combines real-time and historical operational data and includes structural models, mathematical models, physical process models, software process models, etc. In embodiments, digital twin 60136 encompasses hierarchies and functional relationships between the vehicle and various components and subsystems and may be represented as a system of systems. Thus, the digital twin 60136 of the vehicle 60104 may be seen to encompass the digital twins of the vehicle subsystems (determining availability of a digital twin) like vehicle interior layout, electrical and fuel subsystems as well as digital twins of components like engine, brake, fuel pump, alternator, etc. (corresponding to one or more other physical objects). The digital twin 60136 may encompass methods and systems to represent other aspects of the vehicle environment including, without limitation a passenger environment, driver and passengers in the vehicle, environment proximal to the vehicle including nearby vehicles, infrastructure, and other objects detectable (surrounding at least one of the physical object or the neighboring physical object) through, for example, sensors of the vehicle (physical object) and sensors disposed proximal to the vehicle (physical object), such as other vehicles (physical object), traffic control infrastructure, pedestrian safety infrastructure, and the like.”) See also CELLA ([0380], [0383] “Parameters 430 may include market feedback parameters 435, such as relating to pricing, availability, location, or the like of goods, services, fuel, electricity, advertising, content, or the like. Parameters 430 may include rider state parameters 437, such as parameters relating to comfort 439, emotional state, satisfaction, goals, type of trip, fatigue and the like.”) See also CELLA ([Figure 1] and CELLA [0282] where “Fig. 1 is a diagrammatic view that illustrates an architecture for a transportation system showing certain illustrative components and arrangements relating to various embodiments of the present disclosure.”) See also CELLA ([Figure 62] and [0344] where “Fig. 62 is a schematic diagram showing the interaction between the driver and the digital twin using one or more views and modes of the interface in accordance with an example embodiment of the present disclosure.”)
Regarding Claim 22, KUMAR and WANG do not explicitly teach but CELLA teaches
The apparatus of claim 1, wherein:
the physical object comprises a traffic device; and
“In embodiments, the digital twin 60136 of the vehicle 60104 is configured to simulate the operation of the vehicle 60104 or any portion or environment thereof. In embodiments, the digital twin 60136 may be configured to communicate with a user of the vehicle 60104 via a set of communication channels, such as speech, text, gestures, and the like. In embodiments, the digital twin 60136 may be configured to communicate with digital twins of other entities including digital twins of users, nearby vehicles, traffic lights, pedestrians and so on.”. (0734).
This shows a physical object comprising a traffic device.
CELLA and WANG do not explicitly teach but KUMAR teaches
the apparatus comprises the traffic device.
“The road infrastructure is also fitted with various cameras, sensors, Wi-Fi gateways, and other electronics to enable data gathering, aggregation, and communication. The in-vehicle infotainment system readily communicates, cooperates, corroborates, and correlates with the road infrastructure modules to get synched up with one another to collectively do the real-time and secure data capture, cleansing, filtering, decision enablement, and actuation.”. (Pg. 201).
This shows a traffic device comprising a traffic signal.
Regarding Claim 23, KUMAR and WANG do not explicitly teach but CELLA teaches
The apparatus of claim 22, wherein:
the traffic device comprises a traffic signal; and
“In embodiments, the digital twin 60136 of the vehicle 60104 is configured to simulate the operation of the vehicle 60104 or any portion or environment thereof. In embodiments, the digital twin 60136 may be configured to communicate with a user of the vehicle 60104 via a set of communication channels, such as speech, text, gestures, and the like. In embodiments, the digital twin 60136 may be configured to communicate with digital twins of other entities including digital twins of users, nearby vehicles, traffic lights, pedestrians and so on.”. (0734).
This shows a traffic device comprising a traffic signal.
CELLA and WANG do not explicitly teach but KUMAR teaches
controlling the physical operation of the traffic signal comprises adjusting signal phase or timing of the traffic signal.
“The other requirements include detecting the presence of pedestrians and cyclists crossing the street or road to proactively issue "slow down" warnings to incoming vehicles and instantaneously modifying its own cycle to prevent collisions. Besides ensuring utmost safety and the free flow of traffic, all kinds of traffic data need to be captured and stocked to do specific analytics to accurately predict and prescribe the ways and means of substantially improving the traffic system. Ambulances need to get a way out through traffic-free open lanes in the midst of chaotic and cruel traffic.”. (Section 1.3.2 Pg. 201).
This shows controlling and adjusting the signal or timing of a traffic design.
Regarding Claim 25, CELLA and WANG do not explicitly teach but KUMAR teaches
The apparatus of claim 1, wherein the one or more sensors comprise at least one of:
one or more loop detectors;
one or more cameras;
one or more proximity sensors;
one or more environmental sensors;
one or more vehicle dynamics sensors; or one or more biometric sensors.
“The road infrastructure is also fitted with various cameras, sensors, Wi-Fi gateways, and other electronics to enable data gathering, aggregation, and communication. The in-vehicle infotainment system readily communicates, cooperates, corroborates, and correlates with the road infrastructure modules to get synched up with one another to collectively do the real-time and secure data capture, cleansing, filtering, decision enablement, and actuation.”. (Section 1.3.2 Pg. 201).
“To address the traffic challenges, here is a viable proposal. With the availability of powerful cameras and sensors along the roads, bridges, expressways, tunnels, signals, etc., a massive amount of real-time as well as historical data get captured, collected, cleaned, and stocked to be crunched.”. (Section 2.2 Pg. 203).
This shows the one or more sensors comprising at least one of the recited lists.
PNG
media_image3.png
743
1079
media_image3.png
Greyscale
CELLA Figure 1 Reference
PNG
media_image4.png
636
1068
media_image4.png
Greyscale
CELLA Figure 62 Reference
Claim(s) 4 is rejected under are rejected under 35 U.S.C. 103 as being unpatentable over KUMAR, in view of CELLA, in view of WANG, and in further view of WANG 2 (A Digital Twin Paradigm: Vehicle-to-Cloud Based Advanced Driver Assistance Systems), herein WANG 2.
Claim 4
Claims 4 is rejected because the combination of KUMAR, CELLA, and WANG teaches the claim 1 limitations.
The combination of KUMAR, CELLA, and WANG does not explicitly teach wherein conforming the real-time data to match the data schema comprises pre-processing one or more data fields of the real-time data, the conforming real-time data comprising data remaining after the pre-processing of the one or more data fields of the real-time data to be transmitted to the cloud-based digital space.
However, WANG 2 teaches wherein conforming the data to match the data schema comprises pre-processing one or more data fields of the data, the conforming data comprising data remaining after the pre-processing of the one or more data fields of the data to be transmitted to the cloud-based digital space WANG 2 ([Section II DIGITAL TWIN FRAMEWORK FOR CONNECTED VEHICLES] “On the other hand, the processed results (one or more data fields of the data) from the cyber world are received (again via the communication module) and serve as the actuation guidance for the entities or processes in the physical world (conforming the data to match the data schema). The connected vehicles in the physical world can be partially or fully automated (i.e., as connected and automated vehicles), or be driven by human drivers with some ADAS features. The actuation guidance sent from the cyber world will advise the automatic controller or human driver of connected vehicles to conduct cooperative/intelligent maneuvers, and in turn benefits transportation systems with respect to safety, mobility, and/or environment sustainability. The cyber world within the digital twin frame handles all of the computational efforts (the data to be transmitted to the cloud-based digital space) in this two-layer framework. It not only consists of an abstract of the physical world (i.e., the digital replicas of physical entities and processes), but also performs a few key functions. Firstly, sensed data (one or more data fields) from the physical world are cleaned (pre-processed) (such as outlier detection and removal, missing data imputation) and fused (including time synchronization). Then, the pre-processed data may be stored (data remaining after the pre-processing) in the database (e.g., for digital traceability) or be sent to the data mining & knowledge discovery module for further exploration.”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of WANG 2 with KUMAR-CELLA-WANG combination as the references deal with hardware development systems are often used to debug and test the operation of a processing device or other devices. WANG 2 would modify KUMAR-CELLA-WANG combination wherein conforming the data to match the data schema comprises pre-processing one or more data fields of the data, the conforming data comprising data remaining after the pre-processing of the one or more data fields of the data to be transmitted to the cloud-based digital space. The benefits of doing so allow users to leverage vehicle-to-cloud (V2C) communication, on-board devices can upload the data to the server through cellular network. (WANG 2 [Abstract]). Accordingly, claim 4 is rejected based on the combination of these references.
Claim(s) 11-13, 15-16, and 21 are rejected under are rejected under 35 U.S.C. 103 as being unpatentable over KUMAR, in view of CELLA, in view of WANG, and in further view of CHEN (Digital Behavioral Twins for Safe Connected Cars), herein CHEN.
Claim 11
Claim 11 is rejected because the combination of KUMAR, CELLA, and WANG teaches the claim 10 limitations.
The combination of KUMAR, CELLA, and WANG does not explicitly teach determining availability of a second digital twin corresponding to a neighboring physical object, wherein a type of the neighboring physical object is the same type as that of the physical object.
However, CHEN teaches determining availability of a digital twin corresponding to a neighboring physical object, wherein a type of the neighboring physical object is the same type as that of the physical object CHEN ([Abstract] “In this paper, we propose a framework in which behavioral models of drivers (Digital Behavioral Twins) (determining availability of a digital twin) are shared among connected cars (a type of physical object) to predict potential future actions of neighboring vehicles (corresponding to a neighboring physical object), therefore improving the safety of driving. We provide mathematical formulations of models of driver behavior and the environment, and discuss challenging problems during model construction and risk analysis. We also demonstrate that our digital twins framework can accurately predict driver behaviors and effectively prevent collisions using a case study in a virtual driving simulation environment.”) See also CHEN ([Section 1.3 Digital Behavioral Twin] “In this paper, we focus on the challenge of providing accurate context information to drivers with partial, limited view of their surroundings. In particular, we propose a novel Digital Behavioral Twin framework, which leverages the idea of model sharing (determining availability of a digital twin) to improve the safety of connected cars (a type of physical object). The overview of the framework is shown in Figure 1. With an increasing number of higher-quality sensors on board, modern vehicles have the capability to collect historical driving data. These data then are then used to construct a behavioral profile model of a driver for each vehicle, which can be used to predict his or her future behaviors under different driving contexts. Using the connected vehicle-to-vehicle (V2V) technology, these profiles are shared among a pair of neighboring vehicles (corresponding to a neighboring physical object) and used to estimate the potential risks of a collision (same type as that of the physical object) depending on the actions taken by the drivers. The risks for the available actions are then visualized to the drivers so that they can take safer actions to avoid a collision (same type as that of the physical object).”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of CHEN with KUMAR-CELLA-WANG as the references deal with hardware development systems are often used to debug and test the operation of a processing device or other devices. CHEN would modify KUMAR-CELLA-WANG wherein determining availability of a digital twin corresponding to a neighboring physical object, wherein a type of the neighboring physical object is the same type as that of the physical object. The benefits of doing so allow driver's behavioral models to better predict and prevent collisions. (CHEN [Section 1.4 Related Work]). Accordingly, claim 11 is rejected based on the combination of these references.
Claim 12
Claim 12 is rejected because the combination of KUMAR, CELLA, WANG, and CHEN teaches the claim 11 limitations.
KUMAR also teaches obtaining data from the data lake of the second digital twin corresponding to the neighboring physical object KUMAR ([Experimentation and results] “Our paper illustrates a novel intelligent traffic management framework. Intelligent traffic management is acquiring special significance as the number of smart cities across the countries is growing steadily. The much-needed intelligence is realized by accurately predicting traffic congestions and chaos at certain places and by prescribing the ways and means of moderating the traffic jams and snarls. The technologies and tools used are software-defined cloud environments, digital twin (of the digital twin), artificial intelligence (AI) (machine and deep learning algorithms), data lake (from the data lake), real-time data capture (obtaining data), storage, processing, analytics, decision-making and action through loT edge analytics, edge and public cloud integration, etc. By leveraging the proven, potential, and promising technologies, we arrive at a framework, which guarantees the much-needed accuracy in decision-making and subsequent actions. The digital twin is the virtual and logical representation of physical assets (corresponding to the physical object) and processes. There is a direct communication between physical and digital systems (from the data lake of the digital twin)to collect the latest data (obtaining data). Machine and deep learning algorithms are capable of analyzing big data in real time to extract actionable insights in time and the discovered knowledge gets disseminated to the particular junctions and locations (corresponding to the neighboring physical object) to streamline the traffic movement in a smooth manner so as to avoid time wastage in those places.”) Accordingly, claim 12 is rejected based on the combination of these references.
Claim 13
Claim 13 is rejected because the combination of KUMAR, CELLA, WANG, and CHEN teaches the claim 12 limitations.
Claim 13 is rejected because KUMAR teaches processing the obtained data in conjunction with the conforming real-time data regarding the physical object KUMAR ([Section 1.1 The transformation technologies for transport and traffic domains] “Now with the emergence of path-breaking technologies, automated tools, optimized processes and integrated platforms, researchers across the globe have started to focus on breakthrough solutions to minimize the traffic congestions and road blockages. There is a unified view that real-time decision-enabling, value-adding, and actionable data-driven insights are the need of the hour to regulate and rectify traffic issues (conforming data). That is, capturing all kinds of vehicle movement data (processing the obtained data), road capacities (regarding the physical object), driver intention, destination, and any local traffic information and subjecting them to a variety of mining, processing, and analytics is the way forward for smarter traffic management.”) Accordingly, claim 13 is rejected based on the combination of these references.
Claim 15
Claims 15 is rejected because the combination of KUMAR, WANG, and CHEN teaches the claim 13 limitations.
The combination of KUMAR, WANG, and CHEN does not explicitly teach obtaining data from a data lake of the third digital twin corresponding to the one or more other physical objects surrounding at least one of the physical object or the neighboring physical object.
However, CELLA teaches obtaining data from the data lake of the digital twin corresponding to the one or more other physical objects surrounding at least one of the physical object or the neighboring physical object CELLA ([0983] “In embodiments, executive digital twins and the additional enterprise digital twins are generated using various types of data collected from different data sources (obtaining data from the data lake of the digital twin). As discussed, the data may include real-time data 8930, historical data 8932, analytics data 8934, simulation/modeled data 8936, CRM data 8938, organizational data, such as org charts and/or an organizational digital twin 8940, an enterprise data lake 8942 (data lake of the digital twin), and market data 8944. In embodiments, the real time data 8930 may include sensor data collected from, for example, sensor systems 25 (corresponding to the one or more other physical objects) as depicted in Fig. 75. In embodiments, the real-time data 8930 may include sensor data collected from one or more IoT sensor systems, which may be collected directly from each sensor (corresponding to the one or more other physical objects) and/or by various data collection devices associated with the enterprise (surrounding at least one of the physical object or the neighboring physical object), including readers (e.g., RFID, NFC, and Bluetooth readers), beacons, gateways, repeaters, mesh network nodes, WIFI systems, access points, routers, switches, gateways, local area network nodes, edge devices, and the like.”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of CELLA with KUMAR, WANG, and CHEN as the references deal with hardware development systems are often used to debug and test the operation of a processing device or other devices. CELLA would modify KUMAR, WANG, and CHEN wherein obtaining data from the data lake of the digital twin corresponding to the one or more other physical objects surrounding at least one of the physical object or the neighboring physical object. The benefits of doing so capture subject matter expertise and to use the captured subject matter expertise in guiding newer workers or mobile electronic transportation entities to perform transportation service-related tasks. (CELLA [0006]). Accordingly, claim 15 is rejected based on the combination of these references.
Claim 16
Claims 16 is rejected because the combination of KUMAR, WANG, CHEN, and CELLA teaches the claim 15 limitations.
KUMAR also teaches processing the obtained real-time data in conjunction with the conforming data regarding the physical object KUMAR ([Section 1.1 The transformation technologies for transport and traffic domains] “Now with the emergence of path-breaking technologies, automated tools, optimized processes and integrated platforms, researchers across the globe have started to focus on breakthrough solutions to minimize the traffic congestions and road blockages. There is a unified view that real-time decision-enabling, value-adding, and actionable data-driven insights are the need of the hour to regulate and rectify traffic issues (conforming data). That is, capturing all kinds of vehicle movement data (processing the obtained data), road capacities (regarding the physical object), driver intention, destination, and any local traffic information and subjecting them to a variety of mining, processing, and analytics is the way forward for smarter traffic management.”) Accordingly, claim 16 is rejected based on the combination of these references.
Claim 21 is rejected because KUMAR in view of WANG, and in further view of CELLA teaches an apparatus comprising one or more sensors, and memory storing readable instructions, performing the gathering, fetching, conforming, transmitting, receiving, and controlling operations set forth in the rejection of claim 1 above; the citations for the claim 1 preamble and operations (a) – (f) apply as they are very similar to claim 21, which recites the same operations without the reiterating loop of the first claim.
-----
Claim(s) 17-18, 20, and 26-27 are rejected under are rejected under 35 U.S.C. 103 as being over unpatentable over KUMAR, in view of KAZI (C2PS: A digital twin architecture reference model for the cloud-based cyber-physical systems), herein KAZI, and in further view of CELLA (WO 2021222384 A1), herein CELLA.
Claim 17
Claim 17 is rejected because KUMAR teaches a cloud-based platform hosting separate digital twins corresponding to separate physical objects in a traffic region KUMAR ([Section 1.3 Embarking on next-generation intelligent transport systems (ITS)] “Every vehicle (one or more physical objects) is connected. The in-vehicle infotainment system is being fit in every kind of vehicle on the road. This in-vehicle system acts as the centralized controller and gateway for the outside world. They contribute to the communication module capturing and communicating all kinds of operational, health, and performance parameter values (corresponding to one or more physical objects) of every significant module of the vehicle to faraway cloud environments (a cloud-based platform). A cloud-hosted intelligent traffic system (ITS) (hosting one or more digital twins) has to be in place to act as the data cruncher, decision-maker, and actuator. The ITS has to be highly introduced.”) See also KUMAR ([Section 3 Experimentation and Results] “The technologies and tools used are software-defined cloud environments (a cloud-based platform), digital twin (one or more digital twins), artificial intelligence (AI) (machine and deep learning algorithms), data lake, real-time data capture, storage, processing, analytics, decision-making and action through loT edge analytics, edge and public cloud integration, etc. By leveraging the proven, potential, and promising technologies, we arrive at a framework, which guarantees the much-needed accuracy in decision-making and subsequent actions. The digital twin (one or more digital twins) is the virtual and logical representation of physical assets (one or more physical objects) and processes. There is a direct communication between physical and digital systems to collect the latest data.”)
“A cloud-hosted intelligent traffic system (ITS) has to be in place to act as the data cruncher, decision-maker, and actuator. The ITS has to be highly introduced.”. (Section 1.3 Pg. 201).
“The technologies and tools used are software-defined cloud environments, digital twin, artificial intelligence (AI) (machine and deep learning algorithms), data lake, real-time data capture, storage, processing, analytics, decision-making and action through loT edge analytics, edge and public cloud integration, etc. By leveraging the proven, potential, and promising technologies, we arrive at a framework, which guarantees the much-needed accuracy in decision-making and subsequent actions.”. (Section 3 Pg. 207).
KUMAR also teaches
one or more sensors;
“Edge analytics is gaining a lot of momentum these days. With the edge devices being embedded with sufficient processing, storage, and I/O power, they are individually as well as collectively readied to participate in the mainstream computing.”. (Section 1.32).
KUMAR also teaches a communications layer communicatively connecting the digital twins to the physical objects, wherein KUMAR ([Section 1.3 Embarking on next-generation intelligent transport systems (ITS)] “Every vehicle is connected. The in-vehicle infotainment system is being fit in every kind of vehicle on the road. This in-vehicle system acts as the centralized controller and gateway (communications layer) for the outside world. They contribute to the communication module capturing and communicating (communicatively connecting) all kinds of operational, health, and performance parameter values (one or more physical objects) of every significant module of the vehicle to faraway cloud environments (one or more digital twins).
KUMAR does not explicitly teach but CELLA teaches
the separate physical objects comprising human actors, vehicles, and traffic devices
“According to aspects of the present disclosure, a system for monitoring navigational route data through a transportation system has real-world elements disposed therein includes a digital twin datastore and one or more processors. The digital twin datastore includes a transportation system digital twin corresponding to the transportation system and a worker digital twin corresponding to a respective worker of a set of workers within the transportation system. The one or more processors are configured to maintain, via the digital twin datastore, the transportation system digital twin to include contemporaneous positions for the set of workers within the transportation system, monitor movement of each worker in the set of workers via a sensor array, determine, in response to detecting movement of the respective worker, navigational route data for the respective worker, update the transportation system digital twin to include indicia of the navigational route data for the respective worker, and move the worker digital twin along a route of the navigational route data.”. (0132).
“… the digital twin 60136 may be configured to communicate with a user of the vehicle 60104 via a set of communication channels, such as speech, text, gestures, and the like. In embodiments, the digital twin 60136 may be configured to communicate with digital twins of other entities including digital twins of users, nearby vehicles, traffic lights, pedestrians and so on.”. (0734).
“The integrated vehicle and driver twin 61404 may be created, such as by integrating a digital twin of the vehicle 60104 with the digital twin of the driver. In embodiments, such an integration may be achieved by normalizing the 3D models used by each of the twins to represent a consistent scale, and linking via APIs to obtain regular updates of each twin (such as current operating states of the vehicle and current physiological state, posture, or the like of the driver).”. (0778).
It would have been obvious to store separate digital twins for the human actors, vehicles, and traffic devices in Kumar’s cloud-based system in order to combine the vehicle state with the driver’s physiological state to configure the vehicle response and improve safety. CELLA “In the example scenario, the integrated vehicle and the driver twin 61404 may determine that the driver’s state is “drowsy” based on an input from a set of IR cameras tracking the pupil size and eyelid movement and a set of sensors 60108 tracking the (sagging) posture and (slower) reaction time of the driver 60244. The twin may also determine that the vehicle is “unstable” based on the tracking of speed, lateral position, turning angles and moving course. The integrated vehicle and driver twin 61404 may communicate with the driver 60244 alerting the driver 60244 about the potential safety hazards driving in such a state. Alternatively, the integrated vehicle and the driver twin 61404 may take one or more steps to wake the driver like switching on music or turning up the volume and/or ensure driver and vehicle safety by switching the vehicle into an autopilot or autosteer mode.”. (0781).
KUMAR and CELLA do not explicitly teach the communications layer transmits instructions that have been derived from processing of the transmitted real time data by the respective digital twins to physical object; and
However, KAZI teaches the communications layer transmits instructions that have been derived from processing of the transmitted data by the respective digital twins to the physical objects; and KAZI ([Section III. Cloud-Based Cyber-Physical System Architecture (C2PS) ] “These communication groups are created as peer-to-peer networking groups in the Peer-to-peer Relation Layer (communications layer) of the cloud hosted digital twin objects (one or more digital twins). The sensory information collected by the physical layer is stored in its own data store and also in the data store of the cloud based cyber layer. Interactions among the things can occur either through direct ad-hoc communication (e.g. Vehicular Ad-hoc Networks (VANET) in the physical layer or through the cloud layer using peer-to-peer communications among the hosted cyber objects. Important interaction information are stored by both the physical and the cyber layer. Whenever an interaction is received through the cyber layer, it is updated to the responsible physical sensor (one or more physical objects) if possible. Similarly, interactions received (derived from processing of the transmitted data) through the physical layer are transmitted (transmits instructions) to the cyber layer.”)
“Virtual sensors of a cyber thing (i.e. digital twin) are the observed outputs of a physical thing. These observations can be either raw data that will be processed by the cloud based functional units of a cyber thing to detect events or the events themselves.”. (Pg. 2055).
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of KAZI with KUMAR-CELLA as the references deal with hardware development systems are often used to debug and test the operation of a processing device or other devices. KAZI would modify KUMAR-CELLA wherein the communications layer transmits data regarding the one or more physical objects to at least the one or more corresponding digital twins. The benefits of doing so helps in identifying various degrees of basic and hybrid computation-interaction modes in this paradigm. (KAZI [Abstract]).
KUMAR and CELLA do not explicitly teach but KAZI teaches
the communications layer transmits real-time data obtained from the one or more sensors regarding physical object to a respective digital twin corresponding to the respective physical object; and
“The sensory information collected by the physical layer is stored in its own data store and also in the data store of the cloud based cyber layer.”. (Section III Pg. 2053).
“As a result, physical layer can provide real-time sensory fusion and the cyber layer can extend that experience to support delay tolerant applications for the Smart City.”. (Section I Pg. 2051).
“In the proposed C2PS, we assume that a number of independent systems connect together to perform a common goal, where network connections are omnipresent. In C2PS every physical thing is automatically accompanied by a representative digital twin hosted in the cloud.” (Section III Pg. 2053).
KUMAR and CELLA do not explicitly teach but KAZI teaches
the physical objects communicatively connected to the cloud-based platform via the communications layer, wherein the respective physical object performs physical operations in accordance with the transmitted instructions to achieve the end-to-end framework.
“Interactions among the things can occur either through direct ad-hoc communication (e.g. Vehicular Ad-hoc Networks (VANET) in the physical layer or through the cloud layer using peer-to-peer communications among the hosted cyber objects.”. (Section III Pg. 2053).
“We consider that every physical thing p E lP' is comprised of seven elements (Sp, Ap, Fp, Ep, Np, Pp, Dp)- Here, sensors Sp act as inputs to the system, at least one functional unit in the Fp that processes sensory values to identify events Ep, results are stored in data storage set Dp, actuators Ap perform actions on the physical environment, there is at least one network interface in Np, and one power supply in Pp (Fig. 3).”. (Pg. 2054).
“Cloud technology provides the cyber layer with high performance infrastructure resources and data analytics capability, which can improve the CPS feedback control.”. (Pg. 2051).
Claim 18
Claim 18 is rejected because it is the system embodiment of claims 5 and 17, with similar limitations to claims 5 and 17, and is such rejected using the same reasoning found in claims 5 and 17. KUMAR also teaches cloud-based system KUMAR ([Section 1.2 Research problem description] “There are integrated platforms (commercial-grade and open source) for enabling both the activities. These platforms are being made readily available in cloud environments. Collecting all kinds of road, car, and traffic data, carrying them to cloud platforms, subjecting the collected, curated, and cleansed data to a variety of investigations to arrive at decision-enabling insights, taking decisions on time, and plunging into appropriate actions are the major components in the workflow.”) Accordingly, claim 18 is rejected based on the combination of these references.
Claim 20
Claim 20 is rejected because it is the system embodiment of claims 6 and 17, with similar limitations to claims 6 and 17, and is such rejected using the same reasoning found in claims 6 and 17. KUMAR also teaches cloud-based system KUMAR ([Section 1.2 Research problem description] “There are integrated platforms (using the one or more corresponding digital twins) (commercial-grade and open source) for enabling both the activities. These platforms (using the one or more corresponding digital twins) are being made readily available in cloud environments. Collecting all kinds of road, car, and traffic data (one or more physical objects), carrying them to cloud platforms (using the transmitted data), subjecting the collected, curated, and cleansed data to a variety of investigations (simulating operation) to arrive at decision-enabling insights, taking decisions on time, and plunging into appropriate actions are the major components in the workflow.”) KUMAR also teaches performing machine learning and prediction using the transmitted data KUMAR ([Abstract] “Therefore, unveiling real-time, adaptive, precision-centric and predictive traffic monitoring (performing prediction), measurement, management and enhancement solutions are being insisted as an indispensable requirement toward sustainable cities. We have come out with a novel approach leveraging a few potential and promising technologies and tools such as a reliable and reusable virtual model (modeling) for vehicles (one or more physical objects), a machine learning model (performing machine learning), the IoT fog or edge data analytics, a data lake for traffic and vehicle data (using the transmitted data) on public cloud environments, and 5G communication.”) Accordingly, claim 20 is rejected based on the combination of these references.
Regarding Claim 26, CELLA and KAZI teaches but KUMAR teaches
The cloud-based system of claim 17, wherein the one or more sensors comprise at least one of:
one or more loop detectors implemented under a roadway;
one or more cameras implemented in roadside infrastructure;
one or more proximity sensors implemented in the roadside infrastructure;
one or more environmental sensors implemented in the roadside infrastructure;
one or more cameras implemented in one or more vehicles;
one or more proximity sensors implemented in the one or more vehicles;
one or more vehicle dynamics sensors implemented in the one or more vehicles;
or one or more environmental sensors implemented in the one or more vehicles.
“The road infrastructure is also fitted with various cameras, sensors, Wi-Fi gateways, and other electronics to enable data gathering, aggregation, and communication. The in-vehicle infotainment system readily communicates, cooperates, corroborates, and correlates with the road infrastructure modules to get synched up with one another to collectively do the real-time and secure data capture, cleansing, filtering, decision enablement, and actuation.”. (Section 1.3.2 Pg. 201).
This shows the one or more sensors comprising at least one of the recited lists.
Regarding Claim 27, CELLA and KAZI do not explicitly teach but KUMAR teaches
The cloud-based system of claim 26, wherein the physical objects comprise at least one of:
the one or more loop detectors implemented under the roadway;
the roadside infrastructure; or the one or more vehicles.
“The VV is a virtual state of the vehicle and driver, which is processed in the edge and exists in the cloud.”. (Section 2.2 Pg. 203).
This shows vehicles being twinned by the cloud-based system.
Claim 19 is rejected under are rejected under 35 U.S.C. 103 as being over unpatentable over KUMAR, in view of KAZI (C2PS: A digital twin architecture reference model for the cloud-based cyber-physical systems), herein KAZI, in further view of CELLA (WO 2021222384 A1), herein CELLA, and in further view of HAYEE US 20150199902 A1.
Claim 19
KUMAR and KAZI do not explicitly teach but CELLA teaches
The cloud-based system of claim 17, wherein:
the respective physical object comprises a traffic signal or an electronic traffic sign;
“In embodiments, the digital twin 60136 may be configured to communicate with a user of the vehicle 60104 via a set of communication channels, such as speech, text, gestures, and the like. In embodiments, the digital twin 60136 may be configured to communicate with digital twins of other entities including digital twins of users, nearby vehicles, traffic lights, pedestrians and so on.”. (0734).
This shows the physical object comprises a traffic sign or signal.
CELLA and KAZI do not explicitly teach but KUMAR teaches
the physical operations performed by the traffic signal in accordance with the transmitted instructions comprise adjusting signal phase or timing of the traffic signal; and
“The other requirements include detecting the presence of pedestrians and cyclists crossing the street or road to proactively issue "slow down" warnings to incoming vehicles and instantaneously modifying its own cycle to prevent collisions.”. (Section 1.3.2 Pg. 201).
This shows a traffic signal device being used to adjust timings.
KUMAR, CELLA, and KAZI do not explicitly teach but HAYEE teaches
the physical operations performed by the electronic traffic sign in accordance with the transmitted instructions comprise adjusting a visual message displayed on the electronic traffic sign.
“A road sign includes a display and a controller having a memory for storing messages to be shown on the display and providing signals to the display to show at least one stored message on the display. A communication link receives information issued for a vehicle, generates a message from the received information, and provides the generated message to the controller with instructions to show the generated message on the display.”. (Abstract).
“Dsrc-equipped portable changeable sign”. (Title).
This shows traffic device being changed from transmitted instructions.
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of HAYEE with KUMAR-CELLA-KAZI as the references deal with hardware development systems are often used to debug and test the operation of a processing device or other devices. HAYEE would modify KUMAR-CELLA-KAZI to allow the changing of visual signs in traffic. The benefit of doing so helps provide more real time info to drivers, pedestrians, and viewers of the traffic signals in situations like traffic congestion. (HAYEE [0016]).
Claim 24 is rejected under are rejected under 35 U.S.C. 103 as being over unpatentable over KUMAR, in further view of WANG (Architectural Design Alternatives Based on Cloud/Edge/Fog Computing for Connected Vehicles), in further view of CELLA (WO 2021222384 A1), herein CELLA, and in further view of HAYEE US 20150199902 A1.
Regarding Claim 24, KUMAR and WANG do not explicitly teach but CELLA teaches
The apparatus of claim 22, wherein:
the traffic device comprises an electronic traffic sign; and
“In embodiments, the digital twin 60136 of the vehicle 60104 is configured to simulate the operation of the vehicle 60104 or any portion or environment thereof. In embodiments, the digital twin 60136 may be configured to communicate with a user of the vehicle 60104 via a set of communication channels, such as speech, text, gestures, and the like. In embodiments, the digital twin 60136 may be configured to communicate with digital twins of other entities including digital twins of users, nearby vehicles, traffic lights, pedestrians and so on.”. (0734).
This shows a traffic device comprising an electronic traffic design.
KUMAR, CELLA, and WANG do not explicitly teach but HAYEE teaches
controlling the physical operation of the electronic traffic sign adjusting a visual message displayed on the electronic traffic sign.
“A road sign includes a display and a controller having a memory for storing messages to be shown on the display and providing signals to the display to show at least one stored message on the display. A communication link receives information issued for a vehicle, generates a message from the received information, and provides the generated message to the controller with instructions to show the generated message on the display.”. (Abstract).
“Dsrc-equipped portable changeable sign”. (Title).
This shows being able to control the electronic traffic sign to adjust the visual signal.
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of HAYEE with KUMAR-CELLA-WANG as the references deal with hardware development systems are often used to debug and test the operation of a processing device or other devices. HAYEE would modify KUMAR-CELLA-WANG to allow the changing of visual signs in traffic. The benefit of doing so helps provide more real time info to drivers, pedestrians, and viewers of the traffic signals in situations like traffic congestion. (HAYEE [0016]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20180174449 A1 teaches system for adaptively controlling traffic control devices having a traffic signal system, a computing network, a communication system, and a mobile device.
US 20130282154 A1 teaches system, method, and computer program product for controlling electronic signs remotely via one or more communication technologies are described. Content is identified, signs are selected, and the content is transmitted to particular signs from among a plurality of signs. Various forms of communication may be used, including cellular and satellite communications systems from a computer with access to the Internet.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NARCISO EDUARDO MONTES whose telephone number is (571)272-5773. The examiner can normally be reached Mon-Fri 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, REHANA PERVEEN, can be reached at (571) 272-3676. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.E.M./Examiner, Art Unit 2189
/REHANA PERVEEN/Supervisory Patent Examiner, Art Unit 2189
Prosecution Insights