DETAILED ACTION
Claims 16-30 are presented for examination
This office action is in response to submission of application on 10-AUGUST-2022.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on 07-JANURARY-2026 in response to the non-final office action mailed 07-AUGUST-2025 has been entered. Claims 16-30 remain pending in the application.
With regards to the 101 rejection, the rejection to claim 30 has not been overcome by the applicant’s amendments. Despite applicant’s amendments, claim 30 still remains rejected under 35 U.S.C. 101 on the basis of being an abstract idea.
With regards to the 103 rejections, the applicant’s amendments to the claims have not overcome the rejections to claims 16-30 as the former prior art overcomes the applicant’s arguments.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 16, 23, 28, and 30 are rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) in view of LU (U.S. Pub. No. US 20200110181 A1)
Regarding claim 16, CELLA substantially teaches the claim including:
A computer-implemented method for generating a digital twin of a complex system, said method comprising: receiving at least one training dataset comprising N samples, each sample including information on a state of the complex system and on at least one associated action, ([0117] In embodiments of the present disclosure, a method is provided for training an expert agent, comprising; receiving digital twin data from a set of data sources, the digital twin data including: sensor data that is received from a set of sensors (i.e. N samples) that monitor a set of monitored physical entities associated with the enterprise, the sensor data transported by a set of network entities; enterprise data streams generated by a set of enterprise assets, wherein the enterprise assets include at least one of physical entities associated with the enterprise and digital entities associated with the enterprise; structuring the digital twin data into a set of digital twin data structures that are configured to serve a plurality of different role-based digital twins; receiving a request for a role-based digital twin from a client application, wherein the role-based digital twin is configured with respect to a defined role within the enterprise; determining a subset of the structured digital twin data to corresponds to a set of states that are depicted in the role-based digital twin; providing the subset of the structured digital twin data to the client application; receiving expert agent training data sets from the client application, each expert agent training data set indicating a respective action taken by a user using the client application and one or more features that correspond to the respective action; (it should be noted that the states and actions of the digital twin would be of the states and actions that the complex system the digital twin is simulating. In this case, the enterprise.)) the information on said state including time information in relation with said at least one associated action; ([1090] In some embodiments, a digital twin may be updated with real-time data, such that the digital twin reflects the state of a thing or set of things in real-time. (the twin has state information that reflects things in real-time using time data, which would reflect the actions taken in real-time as well, hence the ‘change’ in the state of things) ) and outputting a digital twin including the topological space and transitions between said attainable states subject to said actions, for simulating behaviors of the complex system by means of said digital twin so as to properly achieve at least one task pertaining to the complex system based on said simulated behaviors. ([0541] In some embodiments, the machine learning model 3000 and the digital twin system 1700 may process sensor data and create a digital replica of a set of value chain entities of the plurality of value chain entities 652 to facilitate design, real-time simulation, predictive simulation, and/or hypothetical simulation of a related group of value chain entities. The digital replica of the set of value chain entities may use substantially real-time sensor data to provide for substantially real-time virtual representation of the set of value chain entities and provide for simulation of one or more possible future states of the set of value chain entities (i.e. tasks/goals/end points). The digital replica exists simultaneously with the set of value chain entities being replicated. The digital replica provides one or more simulations of both physical elements and properties of the set of value chain entities being replicated and the dynamics thereof, in embodiments throughout the lifestyle of the set of value chain entities being replicated. The one or more simulations may include a visual simulation, such as a wire-frame virtual representation of the one or more value chain entities 652 that may be viewable on a monitor, using an augmented reality (AR) apparatus, or using a virtual reality (VR) apparatus. The visual simulation may be able to be manipulated by a human user of the information technology system, such as zooming or highlighting components of the simulation and/or providing an exploded view of the one or more value chain entities 652.)
While CELLA does teach generating a digital twin that uses data to simulate a complex system, it does not explicitly teach:
training a generative model over states, actions and time information, to learn a topological space which represents an ensemble of attainable states of the complex system reflecting a variability of the training dataset, in an unsupervised fashion over said states, actions and time information, wherein the generative model learns a mapping to realistic samples comprised in the topological space and to realistic state transitions associated with said realistic samples subject to said actions;
However, in analogous art that similarly uses a digital twin, LU teaches:
training a generative model over states, actions and time information, to learn a topological space which represents an ensemble of attainable states of the complex system reflecting a variability of the training dataset, in an unsupervised fashion over said states, actions and time information, wherein the generative model learns a mapping to realistic samples comprised in the topological space and to realistic state transitions associated with said realistic samples subject to said actions; ([0038] In some examples, the generative model 104 and the discriminative model 105 are trained using Wasserstein distance or metric, which is a mathematic distance function defined between probability distributions on a given metric space, to measure a difference between the distribution of the flight data in the training set and the corresponding distribution of the simulated flight data, until the generative model and the discriminative model are in a game theory solution concept called Nash equilibrium. Using Wasserstein distance to train the generative model and the discriminative model can improve the speed of convergence to Nash equilibrium and the robustness of the GAN. Wasserstein distance is a metric between probability distributions. During training, the GAN can estimate the Wasserstein distance between the real flight data and the simulated flight data. The GAN can provide a robust measurement of the Wasserstein distance between the real flight data and the simulated flight data. During training, the Wasserstein distance between the real flight data and the simulated flight data may decrease. When the Wasserstein distance between the real flight data and the simulated flight data converges, i.e., the Wasserstein distance stops decreasing, the training can be finished. (The metric space is a topological space as it is used for finding a Nash equilibrium, which requires a topology. Further, the GAN uses the Wasserstein distance on the flight data, which means the metric space it embodies would be over the states of the flight. Further, training on a space is the same as learning it and the distance between the simulated and real flights would be mapping the two together in the space. Finally, the distance relates to the change in space and the variability in the change.))
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with LU‘s training of a generative model and mapping the correlation to a topological space and, with CELLA‘s digital twin generation, with a reasonable expectation of success, training a generative model over states mapping the transitions to a space, as in LU, using the state information such as action, time, and current state, as found in CELLA. A person of ordinary skill would have been motivated to better handle variability in data distribution (LU [0002]).
Regarding claim 23, CELLA further teaches:
The method according to claim 16, wherein the complex system is selected among a weather of an area, a city, a building, a production line, a power plan, a car, a plane, a drone, a boat, a submarine, a spacecraft, a brain, a biological cell. ([0353] Referring to FIG. 10, the set of applications 630 provided on the VCNP 604, integrated with the VCNP 604 and/or managed by or for the VCNP 604 and/or involving a set of value chain network entities 652 may include, without limitation, one or more of any of a wide range of types of applications, such as: a supply chain management application 812 (such as, without limitation, for management of timing, quantities, logistics, shipping, delivery, and other details of orders for goods, components, and other items); an asset management application 814 (such as, without limitation, for managing value chain assets, such as floating assets (such as ships, boats, barges, and floating platforms), real property (such as used for location of warehouses, ports, shipyards, distribution centers and other buildings), equipment, machines and fixtures (such as used for handling containers, cargo, packages, goods, and other items), vehicles (such as forklifts, delivery trucks, autonomous vehicles, and other systems used to move items), human resources (such as workers), software, information technology resources, data processing resources, data storage resources, power generation and/or storage resources, computational resources and other assets);… a risk management application 818 (such as, without limitation, for managing risk or liability with respect to a shipment, goods, a product, an asset, a person, a floating asset, a vehicle, an item of equipment, a component, an information technology system, a security system, a security event, a cybersecurity system, an item of property, a health condition, mortality, fire, flood, weather, disability, negligence, business interruption, injury, damage to property, damage to a business, breach of a contract, and others)… a facility management application 850 (such as, without limitation, for managing infrastructure, buildings, systems, real property, personal property, and other property involved in supporting a value chain, such as a shipyard, a port, a distribution center, a warehouse, a dock, a store, a 0fulfillment center, a storage facility, or others, as well as for design, management or control of systems and facilities in or around a property, such as an information technology system, a robotic/autonomous vehicle system, a packaging system, a packing system, a picking system, an inventory tracking system, an inspection system, a routing system for mobile robots, a workflow system for human assets, or the like);)
Regarding claim 28, it comprises of limitations similar to those of claim 16 and is therefore rejected for similar rationale. Regarding the additional limitation of said device being advantageously configured for executing a method for generating a digital twin according to claim 16.
CELLA further teaches
said device being advantageously configured for executing a method for generating a digital twin according to claim 16. ([1571] The methods and systems described herein may be deployed in part or in whole through a machine that executes computer software, program codes, and/or instructions on a processor. The present disclosure may be implemented as a method on the machine, as a system or apparatus as part of or in relation to the machine, or as a computer program product embodied in a computer readable medium executing on one or more of the machines.)
Regarding claim 30, CELLA further teaches:
A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to automatically carry out the steps of the method for generating a digital twin according to claim 16 or of the method for providing a sequence of actions according to claim 27. ([1571] The methods and systems described herein may be deployed in part or in whole through a machine that executes computer software, program codes, and/or instructions on a processor. The present disclosure may be implemented as a method on the machine, as a system or apparatus as part of or in relation to the machine, or as a computer program product embodied in a computer readable medium executing on one or more of the machines.)
Claim(s) 17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) and LU (U.S. Pub. No. US 20200110181 A1) in further view of RAINWATER (U.S. Pub. No. US 20180060727 A1)
While CELLA, as modified by LU, does teach claim 16, which claim 17 is dependent upon, it does not explicitly teach:
The method according to claim 16, wherein in training said generative model, at least part of said states, actions and time information of said N samples is encoded for said training.
However, in analogous art that similarly trains a model, RAINWATER teaches:
The method according to claim 16, wherein in training said generative model, at least part of said states, actions and time information of said N samples is encoded for said training. ([0047] While some embodiments of training data flow 200 have an embedding layer 220, some embodiments may create vectors from indices 215 using a one-of-k or one-hot encoding scheme.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with RAINWATER‘s encoding of training data and, with CELLA‘s, as modified by LU, digital twin generation including state, action, and time information for training, with a reasonable expectation of success, training data that is encoded, as in RAINWATER, where the training data is the state information such as action, time, and current state and then is used for training a generative model, as found in CELLA, as modified by LU. A person of ordinary skill would have been motivated to improve training accuracy (RAINWATER [0045]).
Regarding claim 19, RAINWATER further teaches:
The method according to claim 17, wherein in training said generative model, at least part of said actions and time information of said N samples is subject to a one- hot encoding. ([0047] While some embodiments of training data flow 200 have an embedding layer 220, some embodiments may create vectors from indices 215 using a one-of-k or one-hot encoding scheme.)
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1), LU (U.S. Pub. No. US 20200110181 A1), and RAINWATER (U.S. Pub. No. US 20180060727 A1) in further view of BAI (U.S. Pub. No. US 20220067534 A1)
While CELLA, as modified by LU and RAINWATER, does teach claim 17, which claim 18 is dependent upon, it does not explicitly teach:
The method according to claim 17, wherein in training said generative model, at least part of said states of said N samples is subject to a binary mask.
However in analogous art that similarly trains models, BAI teaches:
The method according to claim 17, wherein in training said generative model, at least part of said states of said N samples is subject to a binary mask. ([0028] As shown in diagram 100, the input sequence 105 may be applied with a binary mask 155. For example, given a sequence of m-dimensional input observations X of length L, the binary mask module may randomly generate a binary mask M with size m×L, where M.sub.i,j=0 masks X.sub.i,j with 0. The masked inputs from the binary mask module 155 are then fed to the encoder 130f, which encodes the inputs into the latent space.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with BAI‘s binary masking and, with CELLA‘s, as modified by LU and RAINWATER, digital twin generation including state, action, and time information for training, with a reasonable expectation of success, training data that is binary masked, as in BAI, where the data being masked is the states, as found in CELLA, as modified by LU and RAINWATER. A person of ordinary skill would have been motivated to lower the time burden of model training (BAI [0016]).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) and LU (U.S. Pub. No. US 20200110181 A1) in further view of PING (U.S. Pub. No. US 11521592 B2 )
Regarding claim 20, while CELLA, as modified by LU, does teach claim 16 which claim 20 is dependent upon, it does not explicitly teach:
The method according to claim 16, wherein the generative model is selected among a generative adversarial network, invertible generative model, normalization flows, a variational autoencoder and a transformer.
However, in analogous art that similarly uses generative models, PING teaches:
The method according to claim 16, wherein the generative model is selected among a generative adversarial network, invertible generative model, normalization flows, a variational autoencoder and a transformer. ((column 3, 35-47)Flow-based models are a family of generative models, in which a simple initial density is transformed into a complex one by applying a series of invertible transformations. One group of models are based on autoregressive transformation, including autoregressive flow (AF) and inverse autoregressive flow (IAF) as the “dual” of each other. AF is analogous to autoregressive models, which performs parallel density evaluation and sequential synthesis. In contrast, IAF performs parallel synthesis but sequential density evaluation, making likelihood-based training very slow. Parallel WaveNet distills an IAF from a pretrained autoregressive WaveNet, which obtains the best of both worlds. (column 3, 64-67) Another group of flow-based models are based on bipartite transformation, which provide likelihood-based training and parallel synthesis. Most recently, WaveGlow and FloWaveNet apply Glow and RealNVP for waveform synthesis, respectively. (column 13, 11-20) Further, FIGS. 3A and 3B depict test log-likelihoods (LLs) vs. MOS scores for likelihood-based models in Table 6 according to one or more embodiments of the present disclosure. The larger LLs roughly correspond to higher MOS scores even when we compare all models. This correlation becomes even more evident when we consider each model separately. It suggests that one may use the likelihood score as an objective measure for model selection. (it should be known that all the models listed here as possible generative models embody all listed models of this claim and that model selection is taught.))
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with PING‘s model selection of generative models and, with CELLA‘s, as modified by LU, usage of a generative model, with a reasonable expectation of success, selecting a type of generative model, as in PING, wherein the generative model is used for generating a digital twin, as found in CELLA, as modified by LU. A person of ordinary skill would have been motivated to increase model speed (PING, Column 1, 22-35).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) and LU (U.S. Pub. No. US 20200110181 A1) in further view of GAITAN (U.S. Pub. No. US 20200372349 A1)
While CELLA, as modified by LU, does teach claim 16, which claim 21 is dependent upon, it does not explicitly teach:
The method according to claim 16, wherein the at least one training dataset is preprocessed for data homogenization and harmonization in distribution.
However, in analogous art that similarly trains a model, GAITAN teaches:
The method according to claim 16, wherein the at least one training dataset is preprocessed for data homogenization and harmonization in distribution. ([0067] The second novel feature of the CLIMATEAI system is its ability to pre-process the multi-model data ensemble to reduce or remove data heterogeneity, and to augment the data ensemble further, reinforcing the underlying hidden functional dependencies among different simulated climate datasets [0121] In some embodiments, the generated candidate ensembles 770 are passed through a data pre-processing step 775 before step 780, to clean, homogenize, and/or possibly augment the ensemble datasets before forecast skill evaluation.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with GAITAN‘s data pre-processing and, with CELLA‘s, as modified by LU, training of a generative model and data distribution, with a reasonable expectation of success, pre-processing data, as in GAITAN, wherein the data is distributed and used for training, as found in CELLA, as modified by LU. A person of ordinary skill would have been motivated to increase model accuracy (GAITAN, [0008]).
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) and LU (U.S. Pub. No. US 20200110181 A1) in further view of BAI (U.S. Pub. No. US 20220067534 A1)
While CELLA, as modified by LU, does teach claim 16, which claim 22 is dependent upon, it does not explicitly teach:
The method according to claim 16, further including mapping the dataset to a latent space in training the generative model.
However, in analogous art that similarly teaches training a model, BAI teaches:
The method according to claim 16, further including mapping the dataset to a latent space in training the generative model. ([0028] As shown in diagram 100, the input sequence 105 may be applied with a binary mask 155. For example, given a sequence of m-dimensional input observations X of length L, the binary mask module may randomly generate a binary mask M with size m×L, where M.sub.i,j=0 masks X.sub.i,j with 0. The masked inputs from the binary mask module 155 are then fed to the encoder 130f, which encodes the inputs into the latent space.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with BAI‘s binary masking and, with CELLA‘s, as modified by LU, digital twin generation including state, action, and time information for training, with a reasonable expectation of success, training data that is binary masked, as in BAI, where the data being masked is the states, as found in CELLA, as modified by LU. A person of ordinary skill would have been motivated to lower the time burden of model training (BAI [0016]).
Claims 24-25 are rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) and LU (U.S. Pub. No. US 20200110181 A1) in further view of FAN (U.S. Pub. No. US 20210095331 A1)
While CELLA, as modified by LU, does teach claim 23, which claim 24 is dependent upon, it does not explicitly teach:
The method according to claim 23, wherein, for the complex system being a biological cell, the information on a state comprises at least one item of the following: omics data, such as genomic data, proteomic data, transcriptomic data, epigenomics data or metabolomic data, and/or imaging data.
However, in analogous art that similarly interacts with complex systems, FAN teaches:
The method according to claim 23, wherein, for the complex system being a biological cell, the information on a state comprises at least one item of the following: omics data, such as genomic data, proteomic data, transcriptomic data, epigenomics data or metabolomic data, and/or imaging data. ([0041] FIG. 15. Single-cell deterministic barcoding. FIG. 15 depicts the experimental procedure to perform deterministic barcoding in cells (DBiC) to detect and eventually sequence single-cell transcriptome in a massively parallel and deterministic manner, which means each cell to be analyzed by sequencing has a known combination barcode AiBj (i=1-50, j=1-50) and known location on the substrate. Therefore, other cellular characteristics such as cell size, morphology, protein signaling, and migration can be imaged and directly linked to the omics data of the same single cell obtained by sequencing.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with FAN‘s cell data and, with CELLA‘s, as modified by LU, digital twin generation including state, action, and time information for training, with a reasonable expectation of success, cell data, as in FAN, where the data is used to make a digital twin, as found in CELLA, as modified by LU. A person of ordinary skill would have been motivated to better identify features (FAN [0004]).
Regarding claim 25, FAN further teaches:
The method according to claim 24, wherein the omics data are single cell sequencing data or bulk sequencing data. ([0041] FIG. 15. Single-cell deterministic barcoding. FIG. 15 depicts the experimental procedure to perform deterministic barcoding in cells (DBiC) to detect and eventually sequence single-cell transcriptome in a massively parallel and deterministic manner, which means each cell to be analyzed by sequencing has a known combination barcode AiBj (i=1-50, j=1-50) and known location on the substrate. Therefore, other cellular characteristics such as cell size, morphology, protein signaling, and migration can be imaged and directly linked to the omics data of the same single cell obtained by sequencing.)
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) and LU (U.S. Pub. No. US 20200110181 A1) in further view of ROTHBAUER (U.S. Pub. No. US 20200095526 A1)
While CELLA, as modified by LU, does teach claim 23, it does not explicitly teach:
The method according to claim 23, wherein, for the complex system being a biological cell, the information on a state further comprises a velocity.
However, in analogous art that similarly uses complex systems, ROTHBAUER teaches:
The method according to claim 23, wherein, for the complex system being a biological cell, the information on a state further comprises a velocity. ([0069] In a preferred embodiment of the method according to the invention, the fluid comprising biological cells is applied into the at least on chamber with a velocity in the range of 4 μL/min to 8 μL/min.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with ROTHBAUER‘s cell velocity and, with CELLA‘s, as modified by LU, digital twin generation including state, action, and time information for training, with a reasonable expectation of success, cell velocity, as in ROTHBAUER, where the data is used to make a digital twin and is associated with a state, as found in CELLA, as modified by LU. A person of ordinary skill would have been motivated to improve the 3d representations (ROTHBAUER [0002]).
Claim 27 and 29 are rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) and LU (U.S. Pub. No. US 20200110181 A1) in further view of WANG (U.S. Pub. No. US 20170108834 A1) in further view of ARDIS (U.S. Pub. No. US 20210182738 A1)
Regarding claim 27, CELLA further teaches:
A computer-implemented method for providing a sequence of actions causing the evolution of a complex system from an initial state to a final state, the method comprising: generating a digital twin of the complex system with a method according to claim 16; . ([0541] In some embodiments, the machine learning model 3000 and the digital twin system 1700 may process sensor data and create a digital replica of a set of value chain entities of the plurality of value chain entities 652 )
While CELLA, as modified by LU, does teach generating a digital twin of a complex system, it does not explicitly teach:
coupling a reinforcement learning algorithm to said digital twin of the complex system, by using a policy of the reinforcement learning algorithm to select at least one action to be performed according to an action selection policy and to provide the selected one or more actions to the digital twin, the digital twin being configured to implement the selected at least one action to generate an output, and by updating parameters of the policy using a reinforcement learning procedure according to a reward signal determined from said output,
However, in analogous art that similarly teaches using a digital twin, WANG teaches:
coupling a reinforcement learning algorithm to said digital twin of the complex system, by using a policy of the reinforcement learning algorithm to select at least one action to be performed according to an action selection policy and to provide the selected one or more actions to the digital twin, the digital twin being configured to implement the selected at least one action to generate an output, and by updating parameters of the policy using a reinforcement learning procedure according to a reward signal determined from said output, ([0050] In some embodiments, the human can train his or her digital twin instance(s) to react in various ways to different situations. In this way, the human digital twins can be personalized to their real human counterpart. For example, in some embodiments, the human provides explicit instructions for how to react to certain situations. In other embodiments, machine learning may be used to refine the digital twin automatically based on the human's behavior. For example, in some embodiments, the digital twin implements a reinforcement learning model. As is generally understood in the art, reinforcement learning models use software agents that try to maximize a reward that it receives from making decisions. For the digital twin, the software agent can use direct feedback from the corresponding human to establish positive and negative rewards. Additional feedback may also be gathered from other digital twins or from other factory systems. For example, actions that result in a safety incident can be used to establish a negative reward for the digital twin, while actions that result in increases in factory efficiency can be used to establish positive rewards.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with WANG‘s reinforcement learning methods and, with CELLA‘s, as modified by LU, digital twin generation, with a reasonable expectation of success, a digital twin that is trained with reinforcement learning, as in WANG, where the twin was generated based on a complex system, as found in CELLA, as modified by LU. A person of ordinary skill would have been motivated to improve simulation optimization (WANG [0008]).
While CELLA, as modified by LU and WANG, does teach generating a digital twin and applying reinforcement learning, it does not explicitly teach:
so that the digital twin is iteratively turned from an initial state to a final state, said initial state and final state representing said initial state and final state of the complex system; outputting the sequence of actions relevant to the complex system and corresponding to said iteratively selected at least one action obtained with the reinforcement learning algorithm applied to the digital twin.
However, in analogous art that similarly teaches a reinforcement learning, ARDIS teaches:
so that the digital twin is iteratively turned from an initial state to a final state, said initial state and final state representing said initial state and final state of the complex system; outputting the sequence of actions relevant to the complex system and corresponding to said iteratively selected at least one action obtained with the reinforcement learning algorithm applied to the digital twin. ([0031] Note that Reinforcement Learning (“RL”) problems may be formulated as a Markov Decision Process (“MDP”). An MDP Mis defined as the 5-tuple (S, A, P, R, γ) where S is a set of states, A is a set of actions, R.sub.s,s′.sup.α, the immediate reward for performing action a in states and transitioning to s′, P(s′|s,α)∈[0,1] the probability of transitioning to state s′ after performing a ins and γ∈(0, 1) the discount on future rewards. Solutions to a MDP are policies π: s.fwdarw.α which map states to actions. Value iteration based RL algorithms aim to approximate the value function Q.sub.π from samples where Q.sub.π(s, α) is the expected return of performing a ins and following policy to completion: (the reinforcement learning would have to output the actions taken and all available actions as a set to the MDP. The MDP also shows the iteration across all states, including the initial, current, and ending states))
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with ARDIS‘s action output and, with CELLA‘s, as modified by LU and WANG, digital twin generation and reinforcement learning, with a reasonable expectation of success, a digital twin that is trained with reinforcement learning and outputs actions available and taken, as in ARDIS, where the twin was generated based on a complex system, as found in CELLA, as modified by LU and WANG. A person of ordinary skill would have been motivated to improve model accuracy (ARDIS [0002]).
Regarding claim 29, it comprises limitations similar to those of claim 27 and is therefore rejected for similar rationale. Further the additional claim limitation, “comprising a device for generating a digital twin according to claim 25, wherein: said at least one processor of the device for generating a digital twin is further configured” is taught by CELLA:
A device for providing a sequence of actions causing the evolution of a complex system from an initial state to a final state, comprising a device for generating a digital twin according to claim 25, wherein: said at least one processor of the device for generating a digital twin is further configured ([1571] The methods and systems described herein may be deployed in part or in whole through a machine that executes computer software, program codes, and/or instructions on a processor. The present disclosure may be implemented as a method on the machine, as a system or apparatus as part of or in relation to the machine, or as a computer program product embodied in a computer readable medium executing on one or more of the machines.)
Response to Arguments
Applicant’s arguments filed 07-JANURARY-2026 have been fully considered, but they are found to be non-persuasive
With regards to the applicant’s remarks regarding the 103 rejection in the non-final action, the applicant argues that the prior art does not teach limitations of claim 1. The examiner acknowledges this argument but has found the arguments non-persuasive.
Applicant disagrees with this reading of Cella. The cited paragraph describes samples comprising data collected from sensors, where the sensor data represent various states of the enterprise. However, it does not disclose an action associated with a state. Specifically, the paragraph explains that a client application sends a request for digital twin data and subsequently receives structured digital twin data; the digital twin data corresponding to the states. Finally, the client application generates data sets, each representing an action. The paragraph establishes a separation between the provisioning of state information and the provisioning of actions. The sequence of operations makes this explicit. The action associated with a given state is effectively derived or calculated by the client application itself, which differs from the claimed feature.
With regards to this argument, the claimed feature is sufficiently taught by the quotation of CELLA. CELLA teaches: “the digital twin data including:… wherein the role-based digital twin is configured with respect to a defined role within the enterprise; determining a subset of the structured digital twin data to corresponds to a set of states that are depicted in the role-based digital twin; providing the subset of the structured digital twin data to the client application; receiving expert agent training data sets from the client application, each expert agent training data set indicating a respective action taken by a user using the client application and one or more features that correspond to the respective action;” from the structure present, it is clear that the state data and the action are a part of a singular source of data in the digital twin, which will be used to train an expert agent. Further, the state data is passed further into the client application and further used in the training, with the action, of the expert agents. As such, the state and the action are associated and received by the digital twin. Further, regardless of the order of operations present, the actions are associated with features. Features are defined earlier in CELLA to be indicative of a state of a simulation, CELLA [0069].
The outstanding Office Action further asserts that the claimed feature "information on said state including time information in relation with said at least one associated action" is disclosed in Cella, par. [1090]: "In some embodiments, a digital twin may be updated with real-time data, such that the digital twin reflects the state of a thing or set of things in realtime." Applicant disagrees, as the paragraph only indicates that an entity is continuously monitored, resulting in a succession of states without describing an action linked to each state. Continuous monitoring may capture changes in state, but it does not include capturing the associated actions that induce those changes.
With regards to this argument, it should be noted that the claim language does not specifically state that the information captures the action itself, but rather the information on the state as associated with an action. Therefore, CELLA has no burden to show the action that caused the change in state as the information on the state as claimed only captures the state and time information as associated with an action.
The missing feature concerns the training of a generative model over states, actions and time, which constitutes the core inventive concept of claim 16. Its absence in Cella shows that Cella's digital twin architecture is different from the subject-matter of claim 16. On page 5, the outstanding Office Action argues that features missing from Cella are disclosed by Lu. Applicant respectfully disagrees. Lu relates to a system for detecting aircraft fault states using a generative adversarial network. The generative model produces simulated flight data and a discriminative model identifies anomalies.
With regards to this argument, the type of data LU produces does not disqualify it as a prior art teaching a GAN trained over states, actions, and time. The type of data relevant to the case has already been taught by CELLA and as such CELLA in combination with LU’s GAN teaches the claimed limitations. Further, the claim does not distinctly use language that would disqualify the data LU itself produces.
Applicant disagrees, as the GAN is trained using data comprising measurements from sensors and/or avionic systems, which are states, but without any time information or associated actions. In addition, the outstanding Office Action states that the Wasserstein distance defines a metric space on states and that training on a space is the same as learning it. Applicant disagrees, as training on a space consists of learning a function that maps latent vectors to sample points in the state space. This mapping approximates the distribution of the training samples, but it does not provide information about the relationships or connectivity between those samples. It does not expose the geometry or topology of the underlying state space. As a result, training does not provide knowledge of the connectivity between states and cannot model transitions between states or the effects of actions on those states. Consequently, training on a state space does not mean learning a space of attainable states.
With regards to this argument, while the flight data is by itself not encompassing of the actions or states, it is still data. Given the similarities between CELLA and LU, it would be obvious to one of ordinary skill in the art to use the data of CELLA in the GAN of LU to produce a GAN that uses the data from CELLA in place of the flight data of LU. Regarding the applicant’s further arguments, the claim language does not claim training on a state space specifically, it rather claims training on a topological space. When training on a topological space, the trained model must learn the specific geographical and topological data relevant to the space in order to train on it, and as such it does learn the space. The transitions between states and actions is not relevant as the topological space is different from a state space.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SKIELER A KOWALIK whose telephone number is (571)272-1850. The examiner can normally be reached 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela D Reyes can be reached at (571)270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SKIELER ALEXANDER KOWALIK/Examiner, Art Unit 2142
/Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
Prosecution Insights