DETAILED ACTION
Claims 16-30 are presented for examination
This office action is in response to submission of application on 10-AUGUST-2022.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on 17-AUGUST-2026 in response to the final office action mailed 18-MAY-2026 has been entered. Claims 16-30 remain pending in the application.
With regards to the 101 rejection, the rejection to claim 30 has been overcome by the applicant’s amendments.
With regards to the 103 rejections, the applicant’s amendments to the claims have not overcome the rejections to claims 16-30 as the former prior art overcomes the applicant’s arguments.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 16, 23, 28, and 30 are rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) in view of LU (U.S. Pub. No. US 20200110181 A1)
Regarding claim 16, CELLA substantially teaches the claim including:
A computer-implemented method for generating a digital twin of a complex system, said method comprising: receiving at least one training dataset comprising N samples, each sample being a tuple including information on a state of the complex system, on at least one associated action applied to the complex system in said state, and on a next state of the complex system resulting from said at least one associated action, ([0117] In embodiments of the present disclosure, a method is provided for training an expert agent, comprising; receiving digital twin data from a set of data sources, the digital twin data including: sensor data that is received from a set of sensors (i.e. N samples) that monitor a set of monitored physical entities associated with the enterprise, the sensor data transported by a set of network entities; enterprise data streams generated by a set of enterprise assets, wherein the enterprise assets include at least one of physical entities associated with the enterprise and digital entities associated with the enterprise; structuring the digital twin data into a set of digital twin data structures that are configured to serve a plurality of different role-based digital twins; receiving a request for a role-based digital twin from a client application, wherein the role-based digital twin is configured with respect to a defined role within the enterprise; determining a subset of the structured digital twin data to corresponds to a set of states that are depicted in the role-based digital twin; providing the subset of the structured digital twin data to the client application; receiving expert agent training data sets from the client application, each expert agent training data set indicating a respective action taken by a user using the client application and one or more features that correspond to the respective action; [0633] In the illustrated example, a machine learning system 2002 may receive training data 2062, outcome data 2060, and/or simulation data 2022. In embodiments, the training data may be data that is used to initially train a model. The training data may be provided by a domain expert, collected from various data sources, and/or obtained from historical records and/or scientific experimentation. The training data 2062 may include quantified properties of an item or environment and outcomes relating from the quantified properties. In some embodiments, the training data may be structured in n-tuples, whereby each tuple includes an outcome and a respective set of properties relating to the outcome. In embodiments, the outcome data 2060 includes real world data (e.g., data measured or captured from one or more of IoT sensors, value chain entities, and/or other sources). The outcome data may include an outcome and properties relating to the outcome. Outcome data may be provided by the value chain system 2030 leveraging the artificial intelligence system 2010 and/or other data sources during operation of the value chain entity system 2010. Each time an outcome is realized (whether negative or positive), the value chain entity system 2010, the artificial intelligence system 2010, as well as any other data source 2050, may output data relating to the outcome to the machine learning system 2002. In embodiments, this data may be provided to the machine-learning system via an API of the adaptive intelligent systems layer 614. Furthermore, in embodiments, the adaptive intelligent systems layer 614 may obtain data from other types of external data sources that are not necessarily a value chain entity but may provide insightful data. For example, weather data, stock market data, news events, and the like may be collected, crawled, subscribed to, or the like to supplement the outcome data (and/or training data and/or simulation data). (it should be noted that the states and actions of the digital twin would be of the states and actions that the complex system the digital twin is simulating. In this case, the enterprise.)) the information on said state including time information in relation with said at least one associated action; ([1090] In some embodiments, a digital twin may be updated with real-time data, such that the digital twin reflects the state of a thing or set of things in real-time. (the twin has state information that reflects things in real-time using time data, which would reflect the actions taken in real-time as well, hence the ‘change’ in the state of things) ) and outputting a digital twin including the topological space and transitions between said attainable states subject to said actions, for simulating behaviors of the complex system by means of said digital twin so as to properly achieve at least one task pertaining to the complex system based on said simulated behaviors. ([0541] In some embodiments, the machine learning model 3000 and the digital twin system 1700 may process sensor data and create a digital replica of a set of value chain entities of the plurality of value chain entities 652 to facilitate design, real-time simulation, predictive simulation, and/or hypothetical simulation of a related group of value chain entities. The digital replica of the set of value chain entities may use substantially real-time sensor data to provide for substantially real-time virtual representation of the set of value chain entities and provide for simulation of one or more possible future states of the set of value chain entities (i.e. tasks/goals/end points). The digital replica exists simultaneously with the set of value chain entities being replicated. The digital replica provides one or more simulations of both physical elements and properties of the set of value chain entities being replicated and the dynamics thereof, in embodiments throughout the lifestyle of the set of value chain entities being replicated. The one or more simulations may include a visual simulation, such as a wire-frame virtual representation of the one or more value chain entities 652 that may be viewable on a monitor, using an augmented reality (AR) apparatus, or using a virtual reality (VR) apparatus. The visual simulation may be able to be manipulated by a human user of the information technology system, such as zooming or highlighting components of the simulation and/or providing an exploded view of the one or more value chain entities 652.)
While CELLA does teach generating a digital twin that uses data to simulate a complex system, it does not explicitly teach:
training a generative model over states, actions and time information, to learn a topological space which represents an ensemble of attainable states of the complex system reflecting a variability of the training dataset, in an unsupervised fashion over said states, actions and time information, wherein the generative model learns a mapping to realistic samples comprised in the topological space and to realistic state transitions associated with said realistic samples subject to said actions;
However, in analogous art that similarly uses a digital twin, LU teaches:
training a generative model over states, actions and time information, to learn a topological space which represents an ensemble of attainable states of the complex system reflecting a variability of the training dataset, in an unsupervised fashion over said states, actions and time information, wherein the generative model learns a mapping to realistic samples comprised in the topological space and to realistic state transitions associated with said realistic samples subject to said actions; ([0038] In some examples, the generative model 104 and the discriminative model 105 are trained using Wasserstein distance or metric, which is a mathematic distance function defined between probability distributions on a given metric space, to measure a difference between the distribution of the flight data in the training set and the corresponding distribution of the simulated flight data, until the generative model and the discriminative model are in a game theory solution concept called Nash equilibrium. Using Wasserstein distance to train the generative model and the discriminative model can improve the speed of convergence to Nash equilibrium and the robustness of the GAN. Wasserstein distance is a metric between probability distributions. During training, the GAN can estimate the Wasserstein distance between the real flight data and the simulated flight data. The GAN can provide a robust measurement of the Wasserstein distance between the real flight data and the simulated flight data. During training, the Wasserstein distance between the real flight data and the simulated flight data may decrease. When the Wasserstein distance between the real flight data and the simulated flight data converges, i.e., the Wasserstein distance stops decreasing, the training can be finished. (The metric space is a topological space as it is used for finding a Nash equilibrium, which requires a topology. Further, the GAN uses the Wasserstein distance on the flight data, which means the metric space it embodies would be over the states of the flight. Further, training on a space is the same as learning it and the distance between the simulated and real flights would be mapping the two together in the space. Finally, the distance relates to the change in space and the variability in the change.))
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with LU‘s training of a generative model and mapping the correlation to a topological space and, with CELLA‘s digital twin generation, with a reasonable expectation of success, training a generative model over states mapping the transitions to a space, as in LU, using the state information such as action, time, and current state, as found in CELLA. A person of ordinary skill would have been motivated to better handle variability in data distribution (LU [0002]).
Regarding claim 23, CELLA further teaches:
The method according to claim 16, wherein the complex system is selected among a weather of an area, a city, a building, a production line, a power plan, a car, a plane, a drone, a boat, a submarine, a spacecraft, a brain, a biological cell. ([0353] Referring to FIG. 10, the set of applications 630 provided on the VCNP 604, integrated with the VCNP 604 and/or managed by or for the VCNP 604 and/or involving a set of value chain network entities 652 may include, without limitation, one or more of any of a wide range of types of applications, such as: a supply chain management application 812 (such as, without limitation, for management of timing, quantities, logistics, shipping, delivery, and other details of orders for goods, components, and other items); an asset management application 814 (such as, without limitation, for managing value chain assets, such as floating assets (such as ships, boats, barges, and floating platforms), real property (such as used for location of warehouses, ports, shipyards, distribution centers and other buildings), equipment, machines and fixtures (such as used for handling containers, cargo, packages, goods, and other items), vehicles (such as forklifts, delivery trucks, autonomous vehicles, and other systems used to move items), human resources (such as workers), software, information technology resources, data processing resources, data storage resources, power generation and/or storage resources, computational resources and other assets);… a risk management application 818 (such as, without limitation, for managing risk or liability with respect to a shipment, goods, a product, an asset, a person, a floating asset, a vehicle, an item of equipment, a component, an information technology system, a security system, a security event, a cybersecurity system, an item of property, a health condition, mortality, fire, flood, weather, disability, negligence, business interruption, injury, damage to property, damage to a business, breach of a contract, and others)… a facility management application 850 (such as, without limitation, for managing infrastructure, buildings, systems, real property, personal property, and other property involved in supporting a value chain, such as a shipyard, a port, a distribution center, a warehouse, a dock, a store, a 0fulfillment center, a storage facility, or others, as well as for design, management or control of systems and facilities in or around a property, such as an information technology system, a robotic/autonomous vehicle system, a packaging system, a packing system, a picking system, an inventory tracking system, an inspection system, a routing system for mobile robots, a workflow system for human assets, or the like);)
Regarding claim 28, it comprises of limitations similar to those of claim 16 and is therefore rejected for similar rationale. Regarding the additional limitation of said device being advantageously configured for executing a method for generating a digital twin according to claim 16.
CELLA further teaches
said device being advantageously configured for executing a method for generating a digital twin according to claim 16. ([1571] The methods and systems described herein may be deployed in part or in whole through a machine that executes computer software, program codes, and/or instructions on a processor. The present disclosure may be implemented as a method on the machine, as a system or apparatus as part of or in relation to the machine, or as a computer program product embodied in a computer readable medium executing on one or more of the machines.)
Regarding claim 30, CELLA further teaches:
A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to automatically carry out the steps of the method for generating a digital twin according to claim 16 or of the method for providing a sequence of actions according to claim 27. ([1571] The methods and systems described herein may be deployed in part or in whole through a machine that executes computer software, program codes, and/or instructions on a processor. The present disclosure may be implemented as a method on the machine, as a system or apparatus as part of or in relation to the machine, or as a computer program product embodied in a computer readable medium executing on one or more of the machines.)
Claim(s) 17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) and LU (U.S. Pub. No. US 20200110181 A1) in further view of RAINWATER (U.S. Pub. No. US 20180060727 A1)
While CELLA, as modified by LU, does teach claim 16, which claim 17 is dependent upon, it does not explicitly teach:
The method according to claim 16, wherein in training said generative model, at least part of said states, actions and time information of said N samples is encoded for said training.
However, in analogous art that similarly trains a model, RAINWATER teaches:
The method according to claim 16, wherein in training said generative model, at least part of said states, actions and time information of said N samples is encoded for said training. ([0047] While some embodiments of training data flow 200 have an embedding layer 220, some embodiments may create vectors from indices 215 using a one-of-k or one-hot encoding scheme.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with RAINWATER‘s encoding of training data and, with CELLA‘s, as modified by LU, digital twin generation including state, action, and time information for training, with a reasonable expectation of success, training data that is encoded, as in RAINWATER, where the training data is the state information such as action, time, and current state and then is used for training a generative model, as found in CELLA, as modified by LU. A person of ordinary skill would have been motivated to improve training accuracy (RAINWATER [0045]).
Regarding claim 19, RAINWATER further teaches:
The method according to claim 17, wherein in training said generative model, at least part of said actions and time information of said N samples is subject to a one- hot encoding. ([0047] While some embodiments of training data flow 200 have an embedding layer 220, some embodiments may create vectors from indices 215 using a one-of-k or one-hot encoding scheme.)
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1), LU (U.S. Pub. No. US 20200110181 A1), and RAINWATER (U.S. Pub. No. US 20180060727 A1) in further view of BAI (U.S. Pub. No. US 20220067534 A1)
While CELLA, as modified by LU and RAINWATER, does teach claim 17, which claim 18 is dependent upon, it does not explicitly teach:
The method according to claim 17, wherein in training said generative model, at least part of said states of said N samples is subject to a binary mask.
However in analogous art that similarly trains models, BAI teaches:
The method according to claim 17, wherein in training said generative model, at least part of said states of said N samples is subject to a binary mask. ([0028] As shown in diagram 100, the input sequence 105 may be applied with a binary mask 155. For example, given a sequence of m-dimensional input observations X of length L, the binary mask module may randomly generate a binary mask M with size m×L, where M.sub.i,j=0 masks X.sub.i,j with 0. The masked inputs from the binary mask module 155 are then fed to the encoder 130f, which encodes the inputs into the latent space.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with BAI‘s binary masking and, with CELLA‘s, as modified by LU and RAINWATER, digital twin generation including state, action, and time information for training, with a reasonable expectation of success, training data that is binary masked, as in BAI, where the data being masked is the states, as found in CELLA, as modified by LU and RAINWATER. A person of ordinary skill would have been motivated to lower the time burden of model training (BAI [0016]).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) and LU (U.S. Pub. No. US 20200110181 A1) in further view of PING (U.S. Pub. No. US 11521592 B2 )
Regarding claim 20, while CELLA, as modified by LU, does teach claim 16 which claim 20 is dependent upon, it does not explicitly teach:
The method according to claim 16, wherein the generative model is selected among a generative adversarial network, invertible generative model, normalization flows, a variational autoencoder and a transformer.
However, in analogous art that similarly uses generative models, PING teaches:
The method according to claim 16, wherein the generative model is selected among a generative adversarial network, invertible generative model, normalization flows, a variational autoencoder and a transformer. ((column 3, 35-47)Flow-based models are a family of generative models, in which a simple initial density is transformed into a complex one by applying a series of invertible transformations. One group of models are based on autoregressive transformation, including autoregressive flow (AF) and inverse autoregressive flow (IAF) as the “dual” of each other. AF is analogous to autoregressive models, which performs parallel density evaluation and sequential synthesis. In contrast, IAF performs parallel synthesis but sequential density evaluation, making likelihood-based training very slow. Parallel WaveNet distills an IAF from a pretrained autoregressive WaveNet, which obtains the best of both worlds. (column 3, 64-67) Another group of flow-based models are based on bipartite transformation, which provide likelihood-based training and parallel synthesis. Most recently, WaveGlow and FloWaveNet apply Glow and RealNVP for waveform synthesis, respectively. (column 13, 11-20) Further, FIGS. 3A and 3B depict test log-likelihoods (LLs) vs. MOS scores for likelihood-based models in Table 6 according to one or more embodiments of the present disclosure. The larger LLs roughly correspond to higher MOS scores even when we compare all models. This correlation becomes even more evident when we consider each model separately. It suggests that one may use the likelihood score as an objective measure for model selection. (it should be known that all the models listed here as possible generative models embody all listed models of this claim and that model selection is taught.))
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with PING‘s model selection of generative models and, with CELLA‘s, as modified by LU, usage of a generative model, with a reasonable expectation of success, selecting a type of generative model, as in PING, wherein the generative model is used for generating a digital twin, as found in CELLA, as modified by LU. A person of ordinary skill would have been motivated to increase model speed (PING, Column 1, 22-35).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) and LU (U.S. Pub. No. US 20200110181 A1) in further view of GAITAN (U.S. Pub. No. US 20200372349 A1)
While CELLA, as modified by LU, does teach claim 16, which claim 21 is dependent upon, it does not explicitly teach:
The method according to claim 16, wherein the at least one training dataset is preprocessed for data homogenization and harmonization in distribution.
However, in analogous art that similarly trains a model, GAITAN teaches:
The method according to claim 16, wherein the at least one training dataset is preprocessed for data homogenization and harmonization in distribution. ([0067] The second novel feature of the CLIMATEAI system is its ability to pre-process the multi-model data ensemble to reduce or remove data heterogeneity, and to augment the data ensemble further, reinforcing the underlying hidden functional dependencies among different simulated climate datasets [0121] In some embodiments, the generated candidate ensembles 770 are passed through a data pre-processing step 775 before step 780, to clean, homogenize, and/or possibly augment the ensemble datasets before forecast skill evaluation.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with GAITAN‘s data pre-processing and, with CELLA‘s, as modified by LU, training of a generative model and data distribution, with a reasonable expectation of success, pre-processing data, as in GAITAN, wherein the data is distributed and used for training, as found in CELLA, as modified by LU. A person of ordinary skill would have been motivated to increase model accuracy (GAITAN, [0008]).
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) and LU (U.S. Pub. No. US 20200110181 A1) in further view of BAI (U.S. Pub. No. US 20220067534 A1)
While CELLA, as modified by LU, does teach claim 16, which claim 22 is dependent upon, it does not explicitly teach:
The method according to claim 16, further including mapping the dataset to a latent space in training the generative model.
However, in analogous art that similarly teaches training a model, BAI teaches:
The method according to claim 16, further including mapping the dataset to a latent space in training the generative model. ([0028] As shown in diagram 100, the input sequence 105 may be applied with a binary mask 155. For example, given a sequence of m-dimensional input observations X of length L, the binary mask module may randomly generate a binary mask M with size m×L, where M.sub.i,j=0 masks X.sub.i,j with 0. The masked inputs from the binary mask module 155 are then fed to the encoder 130f, which encodes the inputs into the latent space.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with BAI‘s binary masking and, with CELLA‘s, as modified by LU, digital twin generation including state, action, and time information for training, with a reasonable expectation of success, training data that is binary masked, as in BAI, where the data being masked is the states, as found in CELLA, as modified by LU. A person of ordinary skill would have been motivated to lower the time burden of model training (BAI [0016]).
Claims 24-25 are rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) and LU (U.S. Pub. No. US 20200110181 A1) in further view of FAN (U.S. Pub. No. US 20210095331 A1)
While CELLA, as modified by LU, does teach claim 23, which claim 24 is dependent upon, it does not explicitly teach:
The method according to claim 23, wherein, for the complex system being a biological cell, the information on a state comprises at least one item of the following: omics data, such as genomic data, proteomic data, transcriptomic data, epigenomics data or metabolomic data, and/or imaging data.
However, in analogous art that similarly interacts with complex systems, FAN teaches:
The method according to claim 23, wherein, for the complex system being a biological cell, the information on a state comprises at least one item of the following: omics data, such as genomic data, proteomic data, transcriptomic data, epigenomics data or metabolomic data, and/or imaging data. ([0041] FIG. 15. Single-cell deterministic barcoding. FIG. 15 depicts the experimental procedure to perform deterministic barcoding in cells (DBiC) to detect and eventually sequence single-cell transcriptome in a massively parallel and deterministic manner, which means each cell to be analyzed by sequencing has a known combination barcode AiBj (i=1-50, j=1-50) and known location on the substrate. Therefore, other cellular characteristics such as cell size, morphology, protein signaling, and migration can be imaged and directly linked to the omics data of the same single cell obtained by sequencing.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with FAN‘s cell data and, with CELLA‘s, as modified by LU, digital twin generation including state, action, and time information for training, with a reasonable expectation of success, cell data, as in FAN, where the data is used to make a digital twin, as found in CELLA, as modified by LU. A person of ordinary skill would have been motivated to better identify features (FAN [0004]).
Regarding claim 25, FAN further teaches:
The method according to claim 24, wherein the omics data are single cell sequencing data or bulk sequencing data. ([0041] FIG. 15. Single-cell deterministic barcoding. FIG. 15 depicts the experimental procedure to perform deterministic barcoding in cells (DBiC) to detect and eventually sequence single-cell transcriptome in a massively parallel and deterministic manner, which means each cell to be analyzed by sequencing has a known combination barcode AiBj (i=1-50, j=1-50) and known location on the substrate. Therefore, other cellular characteristics such as cell size, morphology, protein signaling, and migration can be imaged and directly linked to the omics data of the same single cell obtained by sequencing.)
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) and LU (U.S. Pub. No. US 20200110181 A1) in further view of ROTHBAUER (U.S. Pub. No. US 20200095526 A1)
While CELLA, as modified by LU, does teach claim 23, it does not explicitly teach:
The method according to claim 23, wherein, for the complex system being a biological cell, the information on a state further comprises a velocity.
However, in analogous art that similarly uses complex systems, ROTHBAUER teaches:
The method according to claim 23, wherein, for the complex system being a biological cell, the information on a state further comprises a velocity. ([0069] In a preferred embodiment of the method according to the invention, the fluid comprising biological cells is applied into the at least on chamber with a velocity in the range of 4 μL/min to 8 μL/min.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with ROTHBAUER‘s cell velocity and, with CELLA‘s, as modified by LU, digital twin generation including state, action, and time information for training, with a reasonable expectation of success, cell velocity, as in ROTHBAUER, where the data is used to make a digital twin and is associated with a state, as found in CELLA, as modified by LU. A person of ordinary skill would have been motivated to improve the 3d representations (ROTHBAUER [0002]).
Claim 27 and 29 are rejected under 35 U.S.C. 103 as being unpatentable over CELLA (U.S. Pub. No. US 20210182996 A1) and LU (U.S. Pub. No. US 20200110181 A1) in further view of WANG (U.S. Pub. No. US 20170108834 A1) in further view of ARDIS (U.S. Pub. No. US 20210182738 A1)
Regarding claim 27, CELLA further teaches:
A computer-implemented method for providing a sequence of actions causing the evolution of a complex system from an initial state to a final state, the method comprising: generating a digital twin of the complex system with a method according to claim 16; . ([0541] In some embodiments, the machine learning model 3000 and the digital twin system 1700 may process sensor data and create a digital replica of a set of value chain entities of the plurality of value chain entities 652 )
While CELLA, as modified by LU, does teach generating a digital twin of a complex system, it does not explicitly teach:
coupling a reinforcement learning algorithm to said digital twin of the complex system, by using a policy of the reinforcement learning algorithm to select at least one action to be performed according to an action selection policy and to provide the selected one or more actions to the digital twin, the digital twin being configured to implement the selected at least one action to generate an output, and by updating parameters of the policy using a reinforcement learning procedure according to a reward signal determined from said output,
However, in analogous art that similarly teaches using a digital twin, WANG teaches:
coupling a reinforcement learning algorithm to said digital twin of the complex system, by using a policy of the reinforcement learning algorithm to select at least one action to be performed according to an action selection policy and to provide the selected one or more actions to the digital twin, the digital twin being configured to implement the selected at least one action to generate an output, and by updating parameters of the policy using a reinforcement learning procedure according to a reward signal determined from said output, ([0050] In some embodiments, the human can train his or her digital twin instance(s) to react in various ways to different situations. In this way, the human digital twins can be personalized to their real human counterpart. For example, in some embodiments, the human provides explicit instructions for how to react to certain situations. In other embodiments, machine learning may be used to refine the digital twin automatically based on the human's behavior. For example, in some embodiments, the digital twin implements a reinforcement learning model. As is generally understood in the art, reinforcement learning models use software agents that try to maximize a reward that it receives from making decisions. For the digital twin, the software agent can use direct feedback from the corresponding human to establish positive and negative rewards. Additional feedback may also be gathered from other digital twins or from other factory systems. For example, actions that result in a safety incident can be used to establish a negative reward for the digital twin, while actions that result in increases in factory efficiency can be used to establish positive rewards.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with WANG‘s reinforcement learning methods and, with CELLA‘s, as modified by LU, digital twin generation, with a reasonable expectation of success, a digital twin that is trained with reinforcement learning, as in WANG, where the twin was generated based on a complex system, as found in CELLA, as modified by LU. A person of ordinary skill would have been motivated to improve simulation optimization (WANG [0008]).
While CELLA, as modified by LU and WANG, does teach generating a digital twin and applying reinforcement learning, it does not explicitly teach:
so that the digital twin is iteratively turned from an initial state to a final state, said initial state and final state representing said initial state and final state of the complex system; outputting the sequence of actions relevant to the complex system and corresponding to said iteratively selected at least one action obtained with the reinforcement learning algorithm applied to the digital twin.
However, in analogous art that similarly teaches a reinforcement learning, ARDIS teaches:
so that the digital twin is iteratively turned from an initial state to a final state, said initial state and final state representing said initial state and final state of the complex system; outputting the sequence of actions relevant to the complex system and corresponding to said iteratively selected at least one action obtained with the reinforcement learning algorithm applied to the digital twin. ([0031] Note that Reinforcement Learning (“RL”) problems may be formulated as a Markov Decision Process (“MDP”). An MDP Mis defined as the 5-tuple (S, A, P, R, γ) where S is a set of states, A is a set of actions, R.sub.s,s′.sup.α, the immediate reward for performing action a in states and transitioning to s′, P(s′|s,α)∈[0,1] the probability of transitioning to state s′ after performing a ins and γ∈(0, 1) the discount on future rewards. Solutions to a MDP are policies π: s.fwdarw.α which map states to actions. Value iteration based RL algorithms aim to approximate the value function Q.sub.π from samples where Q.sub.π(s, α) is the expected return of performing a ins and following policy to completion: (the reinforcement learning would have to output the actions taken and all available actions as a set to the MDP. The MDP also shows the iteration across all states, including the initial, current, and ending states))
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with ARDIS‘s action output and, with CELLA‘s, as modified by LU and WANG, digital twin generation and reinforcement learning, with a reasonable expectation of success, a digital twin that is trained with reinforcement learning and outputs actions available and taken, as in ARDIS, where the twin was generated based on a complex system, as found in CELLA, as modified by LU and WANG. A person of ordinary skill would have been motivated to improve model accuracy (ARDIS [0002]).
Regarding claim 29, it comprises limitations similar to those of claim 27 and is therefore rejected for similar rationale. Further the additional claim limitation, “comprising a device for generating a digital twin according to claim 25, wherein: said at least one processor of the device for generating a digital twin is further configured” is taught by CELLA:
A device for providing a sequence of actions causing the evolution of a complex system from an initial state to a final state, comprising a device for generating a digital twin according to claim 25, wherein: said at least one processor of the device for generating a digital twin is further configured ([1571] The methods and systems described herein may be deployed in part or in whole through a machine that executes computer software, program codes, and/or instructions on a processor. The present disclosure may be implemented as a method on the machine, as a system or apparatus as part of or in relation to the machine, or as a computer program product embodied in a computer readable medium executing on one or more of the machines.)
Response to Arguments
Applicant’s arguments filed 17-AUGUST-2026 have been fully considered, but they are found to be non-persuasive
With regards to the applicant’s remarks regarding the 103 rejection in the non-final action, the applicant argues that the prior art does not teach the newly added limitations of claim 1. The examiner acknowledges this argument and has amended the mapping of CELLA to include paragraph 633 of its disclosure which teaches that the training data used to train the machine learning system of CELLA is a tuple which includes the data now claimed. As such, the 103 rejection over claim 1 of the present application is retained.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SKIELER A KOWALIK whose telephone number is (571)272-1850. The examiner can normally be reached 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela D Reyes can be reached at (571)270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SKIELER ALEXANDER KOWALIK/ Examiner, Art Unit 2142 /Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
Prosecution Insights