DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Chinnakannan et al (“Chinnakannan”).
Regarding claim 1, Chinnakannan teaches a method comprising:
providing a first digital twin model of a first physical object (see Chinnakannan, column 3, lines 63-67 through column 4, lines 1-27 teaching “a digital twin service 104 that implements building a digital twin that can be used to simulate behavior and changes of a physical system. In the example embodiment, the provider network includes a runtime environment 106 a of a compute service 108 that initiates/implements a digital twin 110 of a physical system (e.g., simulation of the physical system 112 at the client network/site 114 a) by instantiating and/or executing models 116 a-116 e that are built by the digital twin service 104” where “the executing models 116 and/or the runtime environment 106 a may simulate the behavior of a corresponding physical component 118 of the physical system 112, which may include simulating any internal behavior of the component and/or any number of interactions (inputs and/or outputs) between the component and any number of other components that may be processed by the component or other components” where for example “model 116 a simulates the behavior of component 118 a, receives an input to the component (e.g., environmental data such as wind speed), and provides an output of component 118 a that affects component 118 b (e.g., rpm of a blade assembly due to wind speed)” such that here any physical object such as “physical component 118” or the collection of objects comprising “physical system 112” may be modeled as a “digital twin” and can be considered to be of a first physical object );
establishing a virtual reality environment for simulating operation of the first digital twin model (see Chinnakannan, column 3, lines 63-67 through column 4, lines 1-27 teaching establishing of a virtual reality environment such as the “runtime environment 106a of a compute service 108 that initiates/implements a digital twin 110 of a physical system (e.g., simulation of the physical system 112 at the client network/site 114 a) by instantiating and/or executing models 116 a-116 e that are built by the digital twin service 104“ where “Each of the executing models 116 and/or the runtime environment 106 a may simulate the behavior of a corresponding physical component 118 of the physical system 112, which may include simulating any internal behavior of the component and/or any number of interactions (inputs and/or outputs) between the component and any number of other components that may be processed by the component or other components” and “for example model 116 a simulates the behavior of component 118 a, receives an input to the component (e.g., environmental data such as wind speed), and provides an output of component 118 a that affects component 118 b (e.g., rpm of a blade assembly due to wind speed)” where this runtime environment is a virtual reality environment that includes the properties of the digital twins and the environment that affects the digital twins being simulated in the virtual reality environment);
operating the first digital twin model in the virtual reality environment (see Chinnakannan, column 3, lines 63-67 through column 4, lines 1-27 teaching establishing of a virtual reality environment such as the “runtime environment 106a of a compute service 108 that initiates/implements a digital twin 110 of a physical system (e.g., simulation of the physical system 112 at the client network/site 114 a) by instantiating and/or executing models 116 a-116 e that are built by the digital twin service 104“ where “Each of the executing models 116 and/or the runtime environment 106 a may simulate the behavior of a corresponding physical component 118 of the physical system 112, which may include simulating any internal behavior of the component and/or any number of interactions (inputs and/or outputs) between the component and any number of other components that may be processed by the component or other components” and “for example model 116 a simulates the behavior of component 118 a, receives an input to the component (e.g., environmental data such as wind speed), and provides an output of component 118 a that affects component 118 b (e.g., rpm of a blade assembly due to wind speed)” such that the virtual reality runtime environment operates the digital twin model in the environment to “simulate the behavior of a corresponding physical component” and may do so according to how the model has been built and instantiated to be simulated in the virtual reality environment );
receiving, by the first digital twin model, operational parameters from the virtual reality environment to determine operational effectiveness of the first digital twin model in operating within the virtual reality environment (see Chinnakannan, column 3, lines 63-67 through column 4, lines 1-27 teaching establishing of a virtual reality environment such as the “runtime environment 106a of a compute service 108 that initiates/implements a digital twin 110 of a physical system (e.g., simulation of the physical system 112 at the client network/site 114 a) by instantiating and/or executing models 116 a-116 e that are built by the digital twin service 104“ where “Each of the executing models 116 and/or the runtime environment 106 a may simulate the behavior of a corresponding physical component 118 of the physical system 112, which may include simulating any internal behavior of the component and/or any number of interactions (inputs and/or outputs) between the component and any number of other components that may be processed by the component or other components” and “for example model 116 a simulates the behavior of component 118 a, receives an input to the component (e.g., environmental data such as wind speed), and provides an output of component 118 a that affects component 118 b (e.g., rpm of a blade assembly due to wind speed)” such that here the digital twin model receives operational parameters such as “wind speed” and can determine operational effectiveness such as “rpm of a blade assembly due to wind speed” when taking such parameters into effect in combination with the other parameters of the digital twin); and
enabling a user to visualize the operation of the first digital twin model in the virtual reality environment (see Chinnakannan, column 8, lines 12-67 through column 9, lines 3 teaching “As shown in FIGS. 3 and 4 , in some embodiments, the initiating/performing of the simulation may include generating a 2D or 3D visualization of the physical system based on definitions of the models that are executed. The runtime environment may send the visualization to the remote network of the client (e.g., to a display of the management device). In some embodiments, some or all of the telemetry data received by the simulation is overlaid onto the visualization. For example, rpm values may be overlaid onto and/or near a visualization of the blade assembly” such that this 3D visualization is one enabling a user to visualize the operation of the first digital twin in the VR environment in which it is being simulated; see also column 10, lines 4-67 teaching “an “observability” use case, the digital twin of the physical system enables a user to visualize and manage physical system assets with an enhanced look and feel compared to the real physical system” and a “3D static visual rendering of the physical system” and “an “advanced observability” use case may enhance the above use case and enables a user to observe the physical system rendering along with contextualized information, location specific information, and/or processed information from other the IoT service and/or other services of the provider network (e.g., data received from the client network and formatted/transformed/etc.). This use case may use real time 3D model rendering and augmented reality/extended reality (AR/XR) technologies as desired/specified” and “a “system behavioral” use case may enable a user to answer any number of what-if questions through different types (physics, machine learning/AI, transforms/metrics) of simulations of physical system components applied on the digital twins (models) of the system components (e.g., by using one or more behavior functions)” and “there are certain characteristics of the physical system and criteria (e.g., requirements) for representing a corresponding digital twin of the system” which include “a digital twin may require visualization of each component that makes the physical system identified above (e.g., spatial and time co-ordinates for rendering these components); 4) a digital twin may require digital representation of each component of the physical system with high-fidelity that enables the substitution of the physical system with the digital twin for any/all use cases (a high-fidelity digital twin of a physical system may require high-fidelity for only a subset of components present in the physical system, while one or more of the rest of the components may be represented with lower and different grades of fidelity; a different use case may require different degrees of fidelity between/for the components that make up the physical system)” such that here again these use cases enable a user to visualize the operation of the first digital twin model in the virtual reality environment as they can observe a 3D rendering of the digital twin of the asset operating and different information about the digital twin as well).
Regarding claim 2, Chinnakannan teaches all that is required as applied to claim 1 above and further teaches providing a second digital twin model of a second physical object (see Chinnakannan, column 3, lines 63-67 through column 4, lines 1-27 teaching “a digital twin service 104 that implements building a digital twin that can be used to simulate behavior and changes of a physical system. In the example embodiment, the provider network includes a runtime environment 106 a of a compute service 108 that initiates/implements a digital twin 110 of a physical system (e.g., simulation of the physical system 112 at the client network/site 114 a) by instantiating and/or executing models 116 a-116 e that are built by the digital twin service 104” where “the executing models 116 and/or the runtime environment 106 a may simulate the behavior of a corresponding physical component 118 of the physical system 112, which may include simulating any internal behavior of the component and/or any number of interactions (inputs and/or outputs) between the component and any number of other components that may be processed by the component or other components” where for example “model 116 a simulates the behavior of component 118 a, receives an input to the component (e.g., environmental data such as wind speed), and provides an output of component 118 a that affects component 118 b (e.g., rpm of a blade assembly due to wind speed)” such that here each “component” can be considered a digital twin model where for example a turbine blade digital twin component could be a first digital twin model and a gearbox component model 116b is a second digital twin model provided of a second physical object, where column 6, lines 59-67 through column 7, lines 1-56 further teaches “Based on the relationship (provided by the user for the model definition) between the model 116 a and the model 116 b, the model 116 a may send the generated output (e.g., rpm data and/or torque data) to the model 116 b of a gear box of the wind turbine. Based on the behavior simulation for the model 116 b (provided by the user for the model definition) the model 116 b may process the output of the model 116 a to generate another output for the gear box (e.g., rpm). In a similar way, the output from model 116 b may be sent to model 116 c of a generator, which processes the output (e.g., rpm) to generate another output/result (e.g., power)” such that again various second digital twin model of second physical objects could be provided).
Regarding claim 3, Chinnakannan teaches all that is required as applied to claim 2 above and further teaches operating the second digital twin model in the virtual reality environment along with the first digital twin model (see column 3, lines 63-67 through column 4, lines 1-27 teaching “a digital twin service 104 that implements building a digital twin that can be used to simulate behavior and changes of a physical system. In the example embodiment, the provider network includes a runtime environment 106 a of a compute service 108 that initiates/implements a digital twin 110 of a physical system (e.g., simulation of the physical system 112 at the client network/site 114 a) by instantiating and/or executing models 116 a-116 e that are built by the digital twin service 104” where “the executing models 116 and/or the runtime environment 106 a may simulate the behavior of a corresponding physical component 118 of the physical system 112, which may include simulating any internal behavior of the component and/or any number of interactions (inputs and/or outputs) between the component and any number of other components that may be processed by the component or other components” where for example “model 116 a simulates the behavior of component 118 a, receives an input to the component (e.g., environmental data such as wind speed), and provides an output of component 118 a that affects component 118 b (e.g., rpm of a blade assembly due to wind speed)” such that here each “component” can be considered a digital twin model where for example a turbine blade digital twin component could be a first digital twin model and a gearbox component model 116b is a second digital twin model provided of a second physical object, where column 6, lines 59-67 through column 7, lines 1-56 further teaches “Based on the relationship (provided by the user for the model definition) between the model 116 a and the model 116 b, the model 116 a may send the generated output (e.g., rpm data and/or torque data) to the model 116 b of a gear box of the wind turbine. Based on the behavior simulation for the model 116 b (provided by the user for the model definition) the model 116 b may process the output of the model 116 a to generate another output for the gear box (e.g., rpm). In a similar way, the output from model 116 b may be sent to model 116 c of a generator, which processes the output (e.g., rpm) to generate another output/result (e.g., power)” such that here the second digital twin model is operated in the virtual reality environment along with the first component such that the turbine blade and gearbox operate in the virtual environment).
Regarding claim 4, Chinnakannan teaches all that is required as applied to claim 3 above and further teaches enabling interaction between the first digital twin model and the second digital twin model in the virtual reality environment (see Chinnakannan, column 3, lines 63-67 through column 4, lines 1-27 teaching “a digital twin service 104 that implements building a digital twin that can be used to simulate behavior and changes of a physical system. In the example embodiment, the provider network includes a runtime environment 106 a of a compute service 108 that initiates/implements a digital twin 110 of a physical system (e.g., simulation of the physical system 112 at the client network/site 114 a) by instantiating and/or executing models 116 a-116 e that are built by the digital twin service 104” where “the executing models 116 and/or the runtime environment 106 a may simulate the behavior of a corresponding physical component 118 of the physical system 112, which may include simulating any internal behavior of the component and/or any number of interactions (inputs and/or outputs) between the component and any number of other components that may be processed by the component or other components” where for example “model 116 a simulates the behavior of component 118 a, receives an input to the component (e.g., environmental data such as wind speed), and provides an output of component 118 a that affects component 118 b (e.g., rpm of a blade assembly due to wind speed)” such that here each “component” can be considered a digital twin model where for example a turbine blade digital twin component could be a first digital twin model and a gearbox component model 116b is a second digital twin model provided of a second physical object, where column 6, lines 59-67 through column 7, lines 1-56 further teaches “Based on the relationship (provided by the user for the model definition) between the model 116 a and the model 116 b, the model 116 a may send the generated output (e.g., rpm data and/or torque data) to the model 116 b of a gear box of the wind turbine. Based on the behavior simulation for the model 116 b (provided by the user for the model definition) the model 116 b may process the output of the model 116 a to generate another output for the gear box (e.g., rpm). In a similar way, the output from model 116 b may be sent to model 116 c of a generator, which processes the output (e.g., rpm) to generate another output/result (e.g., power)” such that here interaction is enabled between the models where the first model may send data to the second model which affects the operation of the second model in the environment, or the second model could have values changed which changes how the output data from the first model is processed which is another form of interaction).
Regarding claim 5, Chinnakannan teaches all that is required as applied to claim 3 above and further teaches enabling operation of the first digital twin model in the virtual reality environment to affect operation of the second digital twin model in the virtual reality environment (see Chinnakannan, column 3, lines 63-67 through column 4, lines 1-27 teaching “a digital twin service 104 that implements building a digital twin that can be used to simulate behavior and changes of a physical system. In the example embodiment, the provider network includes a runtime environment 106 a of a compute service 108 that initiates/implements a digital twin 110 of a physical system (e.g., simulation of the physical system 112 at the client network/site 114 a) by instantiating and/or executing models 116 a-116 e that are built by the digital twin service 104” where “the executing models 116 and/or the runtime environment 106 a may simulate the behavior of a corresponding physical component 118 of the physical system 112, which may include simulating any internal behavior of the component and/or any number of interactions (inputs and/or outputs) between the component and any number of other components that may be processed by the component or other components” where for example “model 116 a simulates the behavior of component 118 a, receives an input to the component (e.g., environmental data such as wind speed), and provides an output of component 118 a that affects component 118 b (e.g., rpm of a blade assembly due to wind speed)” such that here each “component” can be considered a digital twin model where for example a turbine blade digital twin component could be a first digital twin model and a gearbox component model 116b is a second digital twin model provided of a second physical object, where column 6, lines 59-67 through column 7, lines 1-56 further teaches “Based on the relationship (provided by the user for the model definition) between the model 116 a and the model 116 b, the model 116 a may send the generated output (e.g., rpm data and/or torque data) to the model 116 b of a gear box of the wind turbine. Based on the behavior simulation for the model 116 b (provided by the user for the model definition) the model 116 b may process the output of the model 116 a to generate another output for the gear box (e.g., rpm). In a similar way, the output from model 116 b may be sent to model 116 c of a generator, which processes the output (e.g., rpm) to generate another output/result (e.g., power)” such that here the operating of the first model is based on its parameters which affects its output which is fed to the second model such that this changes the output of the second model ).
Regarding claim 6, Chinnakannan teaches all that is required as applied to claim 1 above and further teaches wherein enabling a user to visualize comprises enabling a user to visualize on a virtual reality device (see Chinnakannan, column 8, lines 12-67 through column 9, lines 3 teaching “As shown in FIGS. 3 and 4 , in some embodiments, the initiating/performing of the simulation may include generating a 2D or 3D visualization of the physical system based on definitions of the models that are executed. The runtime environment may send the visualization to the remote network of the client (e.g., to a display of the management device). In some embodiments, some or all of the telemetry data received by the simulation is overlaid onto the visualization. For example, rpm values may be overlaid onto and/or near a visualization of the blade assembly” such that this 3D visualization is one enabling a user to visualize the operation of the first digital twin in the VR environment in which it is being simulated; see also column 10, lines 4-67 teaching “an “observability” use case, the digital twin of the physical system enables a user to visualize and manage physical system assets with an enhanced look and feel compared to the real physical system” and a “3D static visual rendering of the physical system” and “an “advanced observability” use case may enhance the above use case and enables a user to observe the physical system rendering along with contextualized information, location specific information, and/or processed information from other the IoT service and/or other services of the provider network (e.g., data received from the client network and formatted/transformed/etc.). This use case may use real time 3D model rendering and augmented reality/extended reality (AR/XR) technologies as desired/specified” and “a “system behavioral” use case may enable a user to answer any number of what-if questions through different types (physics, machine learning/AI, transforms/metrics) of simulations of physical system components applied on the digital twins (models) of the system components (e.g., by using one or more behavior functions)” and “there are certain characteristics of the physical system and criteria (e.g., requirements) for representing a corresponding digital twin of the system” which include “a digital twin may require visualization of each component that makes the physical system identified above (e.g., spatial and time co-ordinates for rendering these components); 4) a digital twin may require digital representation of each component of the physical system with high-fidelity that enables the substitution of the physical system with the digital twin for any/all use cases (a high-fidelity digital twin of a physical system may require high-fidelity for only a subset of components present in the physical system, while one or more of the rest of the components may be represented with lower and different grades of fidelity; a different use case may require different degrees of fidelity between/for the components that make up the physical system)” such that here these are all instances of visualization of the digital twin in a virtual reality environment such that the device used to view such visualizations is a virtual reality device as it is a device used for visualizing a virtual reality, where for example Chinnakannan, column 8, lines 12-24, teaches “runtime environment may send the visualization to the remote network of the client (e.g., to a display of the management device)” such that this display is a virtual reality device as it displays the virtual reality visualization).
Regarding claim 7, Chinnakannan teaches all that is required as applied to claim 1 above and further teaches enabling a user to control the operational parameters in the virtual reality environment (see Chinnakannan, column 7, lines 3-56 teaching “During the execution of the models of the simulation of the physical system, a user may provide input (e.g., via the management device 134 and the interface 122) that indicates one or more changes to the simulation. The simulation modifier 140 may send the change(s) and/or the indication of the change(s) to the runtime environment. The runtime environment may modify the simulation to implement the indicated change(s) (e.g., forward the telemetry data to the appropriate models executing in the runtime environment and/or make changes to the models themselves) The runtime environment/model(s) may generate one or more results based on the modification(s) to the simulation” and “For example, the change may be a much higher wind speed than runtime environment is currently receiving as telemetry data. The runtime environment may generate one or more results based on the modification to the simulation. For example, a wind speed of 40 mph may be provided based on user input. A model of the blade assembly may receive the new wind speed indicating 40 mph. The model of the blade assembly may process the new wind speed (e.g., based on the behavior simulation for the model) and generate output data that indicates an increased rpm of the blade assembly. The new rpm may result in a new value for the power output of the generator. The runtime environment may send the new value for the power output (and/or the increased rpm and/or any other results of the change(s)) to one or more destinations (e.g., to the management device 134 and/or to the digital twin service or other destination within the provider network)” such that here the user can change any parameter of the environment of the digital twin or component or attribute of the digital twin in the virtual reality environment).
Regarding claims 8-14, the instant claims correspond to a “computer program product comprising a computer-readable storage medium having computer-usable program code embodied therein, the computer-usable program code configured to perform the following when executed by at least one processor” perform the same functions of the method as in claims 1-7, respectively. Note that Applicant’s Specification at paragraph 0017 explicitly disavows such a storage medium as any type of transitory signal as “computer readable storage medium, as that term is used in the present disclosure, is not to be construed as storage in the form of transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through a fiber optic cable, electrical signals communicated through a wire, and/or other transmission media” and only provides examples of non-transitory type storage media. Chinnakannan teaches such a computer program product and storage medium and processor (see Chinnakannan, column 24, lines 24-60 teaching “methods may be implemented by a computer system (e.g., a computer system as in FIG. 13 ) that includes one or more processors executing program instructions stored on a computer-readable storage medium coupled to the processors. The program instructions may implement the functionality described herein (e.g., the functionality of the digital twin service, compute service, models, and any other components that implement the techniques described herein). The various methods as illustrated in the figures and described herein represent example embodiments of methods. The order of any method may be changed, and various elements may be added, reordered, combined, omitted, modified, etc”) which performs the method as explained above in the rejections of claims 1-7, respectively. In light of this, the limitations of claims 8-14 correspond to the limitations of claims 1-7, respectively; thus they are rejected on the same grounds as claims 1-7, respectively.
Regarding claims 15-20, the instant claims recite an apparatus in the form of a “system comprising: at least one processor; at least one memory device operably coupled to the at least one processor and storing instructions for execution on the at least one processor, the instructions causing the processor to” perform the method as in claims 1-5 and 7, respectively. Chinnakannan teaches such an apparatus (see Chinnakannan, column 24, lines 24-60 teaching “methods may be implemented by a computer system (e.g., a computer system as in FIG. 13 ) that includes one or more processors executing program instructions stored on a computer-readable storage medium coupled to the processors. The program instructions may implement the functionality described herein (e.g., the functionality of the digital twin service, compute service, models, and any other components that implement the techniques described herein). The various methods as illustrated in the figures and described herein represent example embodiments of methods. The order of any method may be changed, and various elements may be added, reordered, combined, omitted, modified, etc”) which performs the method where the functions of the method correspond to those as addressed in the rejections of claims 1-5 and 7, respectively. In light of this, the limitations of claims 15-20 correspond to the limitations of claims 1-5 and 7, respectively; thus they are rejected on the same grounds as claims 1-5 and 7, respectively.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See Rovaglio et al (US PGPUB No. 2009/0319058), teaching another system in which a virtual reality environment may be provided to a user for visualization of digital twins operating in a virtual reality environment (see paragraphs 0014-0127).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SCOTT E SONNERS whose telephone number is (571)270-7504. The examiner can normally be reached Mon-Friday 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SCOTT E SONNERS/Examiner, Art Unit 2613
/XIAO M WU/Supervisory Patent Examiner, Art Unit 2613