Detailed Action
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicant
The following is a Final Office action to Application Serial Number 17/544,077, filed on December 7, 2021. In response to Examiner’s Non-Final Office Action of September 3, 2025, Applicant, on December 3, 2025, amended dependent claims 12. Claims 1-20 are pending in this application and have been rejected below.
Response to Amendment
Applicant’s amendments are acknowledged.
Regarding 35 U.S.C. § 101 rejection, the amended claims have been considered
and are insufficient to overcome the rejection. Please refer to the 35 U.S.C. § 101 rejection for further explanation and rationale.
The 35 U.S.C. § 103 rejections are hereby amended pursuant to applicants amendments. Updated 35 U.S.C. § 103 rejections have been applied to amended claims. Please refer to the § 103 rejection for further explanation and rationale.
Response to Arguments
Applicant’s arguments filed December 3, 2025 have been fully considered but they are not persuasive and/or are moot in view of the revised rejections. Applicant’s arguments will be addressed herein below in the order in which they appear in the response filed December 3, 2025.
On Pgs. 7-8, regarding the 35 U.S.C. § 103 rejection, Applicant states a reference to be available as prior art as of an earlier parent application's filing date, the subject matter relied upon must be supported by the disclosure of that parent application (see, e.g., MPEP § 2154.01(b)) requiring "that a prior-filed application to which a priority or benefit claim is made must describe the subject matter from the U.S. patent document relied upon in a rejection."). In response, Crabtree Application 18/754,140 US Publication (20240348663A1) relies upon priority from application 18/594,008 US Publication (20240205266A1) relying upon prior from application, 15/813,097 US Publication (20180165587A1) filed on November 14, 2017 which provides support in the specification and discloses the subject matter relied upon for the rejection.
On Pgs. 7-8, regarding the 35 U.S.C. § 103 rejection, Applicant states the cited references do not describe or suggest, "obtaining, by a central simulation system, a plurality of policies to be used for performing a simulation involving multiple machine learning models interacting with a target population, wherein the machine learning models are implemented on different systems" and "providing information for configuring simulators on the different systems, each simulator representing at least the machine learning model of a given one of the different systems," In response, Examiner disagrees Crabtree discloses in Fig. 3 /Par. 144 (the central system) ; Fig 17 & !8 Par. 68- enabling collaborative development, sharing, and reuse of simulation scenarios, models, and insights across multiple users, domains, and organizations, while ensuring data privacy, security, and compliance with relevant regulations and policies. In Par. 148- the AI component can analyze the sequences of actions and their impact on the system state, enabling it to learn optimal strategies and policies for achieving the desired objectives. The knowledge graph can also capture causal relationships and dependencies between different aspects of the simulation, providing a structured representation of the system dynamics. By combining vector databases for efficient state storage and retrieval with knowledge graphs for modeling state transitions and relationships, the simulation environment computing system can enable powerful real-time parameter adjustment capabilities. Crabtree inputs information for simulation in Par. 62-66- ingesting and preprocessing external data from various formats and sources, such as sensors, databases, application programming interfaces, or user input; aligning and fusing the external data with the simulation model's internal state… Please review 103 analysis below.
On Pgs. 9-11, regarding the 35 U.S.C. § 101 rejection, Applicant states a technical improvement to the machine learning model itself. The claim does not merely output a fairness report, but recites that "the at least one system updates its corresponding machine learning model based at least in part on the selected policy." Similar to how the ARP found that "adjusting the first values... of parameters" was an improvement to computer functionality in Desjardins, the "updating" of the model in claim 1 is a specific improvement to the functioning of the different systems. This solves the technical problem disclosed in the specification of enabling the optimization of long-term fairness in models without requiring the exposure of private source code or datasets. In response. Examiner finds the present claims improve an existing business process of model analysis and there are currently no functional advancement to any technology or technological field, in order for the claim elements to be considered significantly more than the abstract idea itself. Utilizing computer structure and technology to analyze policy data are all, both individually and in combination, computer functions such as receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network) and storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). Regarding Desjardins and Recentive, Examiner finds the general use of a machine learning analysis does not provide a meaningful limitation to transform the abstract idea into a practical application. Therefore, currently, the machine learning is solely used a tool to perform the instructions of the abstract idea.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1- 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claims 1-20 are directed to simulated fairness impact.
Claim 1 recites a method for simulated fairness impact, Claim 14 recites an article of manufacture for simulated fairness impact and Claim 20 recites a system for simulated fairness impact, which include obtaining a plurality of policies to be used for performing a simulation; providing information for configuring simulators on the different systems,; performing multiple iterations of the simulation for the plurality of policies, wherein, for each iteration: (i) predicts a state of the target population, (ii) provides the state of the target population to the simulators, and (iii) collects one or more metrics based on results of the simulators; and selecting and sending one of the policies to at least one of the different systems based on the collected metrics.
As drafted, this is, under its broadest reasonable interpretation, within the Abstract idea grouping of “Mental Processes” – evaluation. The recitation of “central simulation system”, “systems”, “computer program product”, “computer readable storage medium”; “computing device”, “memory” and “processor”, provide nothing in the claim elements to preclude the step from being “Mental Processes”- evaluation. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. The claims primarily recite the additional element of using computer components to perform each step. The “central simulation system”, “systems”, “computer program product”, “computer readable storage medium”; “computing device”, “memory” and “processor”, is recited at a high-level of generality, such that it amounts no more than mere instructions to apply the exception using a computer component. See MPEP 2106.05(f). Furthermore, the claim 1, claim 14, and claim 20 recite using one or more machine learning/ simulator analysis techniques. The specification discloses the machine learning analysis at a high-level of generality, providing examples of different techniques that may be applied. The general use of a machine learning analysis does not provide a meaningful limitation to transform the abstract idea into a practical application. Therefore, currently, the machine learning is solely used a tool to perform the instructions of the abstract idea. Accordingly, the additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims also fail to recite any improvements to another technology or technical field, improvements to the functioning of the computer itself, use of a particular machine, effecting a transformation or reduction of a particular article to a different state or thing, and/or an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception. See 84 Fed. Reg. 55. In particular, there is a lack of improvement to a computer or technical field in policy analysis.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as an ordered combination do not amount to significantly more than the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “central simulation system”, “systems”, “computer program product”, “computer readable storage medium”; “computing device”, “memory” and “processor”, is insufficient to amount to significantly more. (See MPEP 2106.05(f) – Mere Instructions to Apply an Exception – “Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible.” Alice Corp., 134 S. Ct. at 235). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
The claim fails to recite any improvements to another technology or technical field, improvements to the functioning of the computer itself, use of a particular machine, effecting a transformation or reduction of a particular article to a different state or thing, adding unconventional steps that confine the claim to a particular useful application, and/or meaningful limitations beyond generally linking the use of an abstract idea to a particular environment. See 84 Fed. Reg. 55. Viewed individually or as a whole, these additional claim element(s) do not provide meaningful limitation(s) to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amounts to significantly more than the abstract idea itself. With regards to receiving data and step 2B, it is M2106.05(d)- Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) and Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). With regards to step 2B and the “machine learning” and “simulator”- the machine learning is a tool to perform the judicial exception.
Examiner concludes that the additional elements in combination fail to amount to significantly more than the abstract idea based on findings that each element merely performs the same function(s) in combination as each element performs separately. The claim is not patent eligible. Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually.
Dependent Claims 2-13, and 15-19 recite the multiple machine learning models each perform a common machine learning task; each of the plurality of policies comprises one or more constraints on the common machine learning task; at least one of the machine learning models is trained based at least in part on a dataset that is specific to a given one of the different systems; at least one of: (i) source code corresponding to the at least one machine learning model is not shared during the simulation and (ii) the dataset that is specific to the given one of the different systems is not shared during the simulation; the collected metrics comprise one or more performance metrics associated with the machine learning models; maintaining, by the central simulation system, one or more fairness metrics for the simulation; aggregating the one or more fairness metrics and the one or more performance metrics to select the policy; obtaining real world data corresponding to a particular time of the simulation; and validating at least one of: the one or more collected metrics and the one or more fairness metrics based at least in part on the real world data; providing information to configure at least one other simulator of a new machine learning model; and adding the other simulator to the simulation after at least one of the iterations, wherein the adding comprises adjusting one or more parameters of one or more of the policies based on historical data associated with the new machine learning model; wherein the simulation simulates a time period of at least one year; wherein the different systems are associated with different entities, and wherein the simulators execute at least in part within in different private cloud environments of the different entities.; wherein software is provided as a service in a cloud environment for implementing the central simulation system; and further narrowing the abstract idea. These recited limitations in the dependent claims do not amount to significantly more than the above-identified judicial exceptions in Claims 1, 14 and 20. Regarding Claims, 4-5, 7, 13, 17-18 and the additional elements of “system” it is M2106.05(d)- Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information). Regarding claim 2-6, 9-12 and claim 15-19 and the additional element of machine learning model/ simulator/ simulation- the specification discloses the machine learning at a high-level of generality, providing examples of different techniques that may be applied. The general use of a machine learning technique does not provide a meaningful limitation to transform the abstract idea into a practical application. Therefore, currently, the machine learning is solely used a tool to perform the instructions of the abstract idea. Regarding Claim 12-13 and the additional element of “cloud environment”- it is field of use M2106.05(h)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention
pertains. Patentability shall not be negated by the manner in which the invention was made.
1-10 and 12-20 are rejected under 35 U.S.C. 103 as being unpatentable over Crabtree et al., US Publication No. 20240348663A1, [hereinafter Crabtree], in view of Jadon et al., US Publication No. 20220004897A1, [hereinafter Jadon].
Regarding Claim 1,
Crabtree teaches
A computer-implemented method, the method comprising: obtaining, by a central simulation system, a plurality of policies to be used for performing a simulation involving multiple machine learning models interacting with a target population, wherein the machine learning models are implemented on different systems (Crabtree Par. 56-57“ Accordingly, the inventor has conceived and reduced to practice, an artificial intelligence-driven Simulation and Experimental Design Decision Platform for reducing epistemic uncertainty in complex systems models and enabling system planning, automation and actuation. The system integrates advanced techniques from artificial intelligence, machine learning, simulation, and uncertainty quantification to generate and run scenarios, monitor progress, and adjust parameters in real-time to achieve user-defined goals. Par. 68 ;Claim 3; Par. 143; Par. 148; Fig 3; Fig. 17; Fig 18)
providing information for configuring simulators on the different systems, each simulator representing at least the machine learning model of a given one of the different systems; (Crabtree Par. 62-66“ According to an aspect of an embodiment, the one or more hardware processors are further configured for: ingesting and preprocessing external data from various formats and sources, such as sensors, databases, application programming interfaces, or user input; aligning and fusing the external data with the simulation model's internal state representation; validating and updating the simulation model's assumptions, parameters, and constraints based on the external data; and exchanging data and insights between the simulation and decision platform and external machine learning models or knowledge bases According to an aspect of an embodiment, the one or more hardware processors are further configured for assisting users in setting up, configuring, and interpreting the simulation scenarios and results through an interactive user interface that provides guided workflows, contextual help, and natural language processing capabilities. Par. 68; Par. 143; Par. 148)
performing multiple iterations of the simulation for the plurality of policies, wherein, for each iteration, the central simulation system: (i) predicts a state of the target population, (ii) provides the state of the target population to the simulators, and (iii) collects one or more metrics based on results of the simulators; (Crabtree Par. 93-“ According to an embodiment, the simulation and decision platform may engage in express synthetic data generation to identify specific system states, state transitions, agent or entity phenomena or actions (both individual or group), spatio-temporal dynamics, or to explore potential hypotheses for specific causal relationships or symbolic functions. This may include comparative evaluation of symbolic artificial intelligence and connectionist artificial intelligence (or statistical or machine learning approaches) against modeling simulation results.; Par. 106; 110; Par. 140; Par. 144-145; Par. 148)
wherein the method is carried out by at least one computing device (Crabtree Par. 191- “The exemplary computing environment described herein comprises a computing device 10 (further comprising a system bus 11, one or more processors 20, a system memory 30, one or more interfaces 40, one or more non-volatile data storage devices 50), external peripherals and accessories 60, external communication devices 70, remote computing devices 80, and cloud-based services 90.”) .
Crabtree teaches model analysis and the feature is expounded upon by Jadon:
and selecting and sending one of the policies to at least one of the different systems based on the collected metrics, wherein the at least one system updates its corresponding machine learning model based at least in part on the selected policy; (Jadon Par. 20-21-“ The techniques of this disclosure may address one or more of the challenges described above. For instance, in accordance with an example of this disclosure, a computing system, such as a flow controller, may collect data for a network having a plurality of network devices. For example, the computing system may collect flow data from the network. The flow data may include underlay flow data, overlay flow data, and/or other types of flow data. The computing system may store the data in a database. Furthermore, based on a request for a prediction that is received by the computing system, a ML system may train each respective ML model in a predetermined plurality of ML models to generate a respective training-phase prediction in a plurality of training-phase predictions. The network analysis system may automatically determine a selected ML model in the plurality of ML models based on evaluation metrics for the plurality of ML models. Additionally, the network analysis system may apply the selected ML model to generate the prediction based on the data collected from the network.”; 114-116)
Crabtree and Jadon are directed to machine learning analysis. It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have improve upon data analysis of Crabtree, as taught by Jadon, by additional model selection with a reasonable expectation of success of arriving at the claimed invention. One of ordinary skill in the art would have been motivated to make the modification to the teachings of Crabtree with the motivation of improving selected ML models for specific requests (Jadon Par. 101).
Regarding Claim 2 and Claim 15, Crabtree in view of Jadon teach The computer-implemented method of claim 1,… and The computer program product of claim 14,…
wherein the multiple machine learning models each perform a common machine learning task (Crabtree Par. 59-60-“ According to another preferred embodiment, a system for reducing epistemic uncertainty in complex systems employing an artificial intelligence-driven simulation and experimental design decision platform is disclosed, comprising one or more computers with executable instructions that, when executed, cause the system to: generate and run simulation scenarios based on user-defined objectives and constraints, and iteratively adjusting simulation parameters using machine learning techniques to optimize scenario outcomes; provide a scalable and interactive simulation environment that enables real-time monitoring, analysis, and adaptation of a simulated system based on evolving real-world conditions and user feedback; generate human-interpretable insights, explanations, and recommendations based on the simulation results, employing explainable artificial intelligence techniques to facilitate understanding and decision-making; integrate with external data sources, machine learning models, and domain-specific knowledge bases to enhance the accuracy, relevance, and compliance of simulation outcomes; and quantify and reduce the epistemic uncertainty associated with the simulated system by employing probabilistic reasoning, targeted exploration, and continuous learning from both simulated and real-world data.”)
Regarding Claim 3 and Claim 16, Crabtree in view of Jadon teach The computer-implemented method of claim 2,… and The computer program product of claim 15,…
wherein each of the plurality of policies comprises one or more constraints on the common machine learning task (Crabtree Par. 59-60-“ According to another preferred embodiment, a system for reducing epistemic uncertainty in complex systems employing an artificial intelligence-driven simulation and experimental design decision platform is disclosed, comprising one or more computers with executable instructions that, when executed, cause the system to: generate and run simulation scenarios based on user-defined objectives and constraints, and iteratively adjusting simulation parameters using machine learning techniques to optimize scenario outcomes; provide a scalable and interactive simulation environment that enables real-time monitoring, analysis, and adaptation of a simulated system based on evolving real-world conditions and user feedback; generate human-interpretable insights, explanations, and recommendations based on the simulation results, employing explainable artificial intelligence techniques to facilitate understanding and decision-making; integrate with external data sources, machine learning models, and domain-specific knowledge bases to enhance the accuracy, relevance, and compliance of simulation outcomes; and quantify and reduce the epistemic uncertainty associated with the simulated system by employing probabilistic reasoning, targeted exploration, and continuous learning from both simulated and real-world data.”)
Regarding Claim 4 and Claim 17, Crabtree in view of Jadon teach The computer-implemented method of claim 1,… and The computer program product of claim 14,…
wherein at least one of the machine learning models is trained based at least in part on a dataset that is specific to a given one of the different systems. (Crabtree Par. 162-166-“ In some implementations, the federated learning process may comprise the following steps: a. Local Model Training: Each participating store trains a local model using its own customer data. The local models are trained on the same model architecture and hyperparameters as the global model. b. Gradient Aggregation: Instead of sharing the raw data, each store computes the gradients (i.e., the direction and magnitude of model updates) based on its local training. These gradients are then securely transmitted to a central server (i.e., simulation and decision platform 100). c. Secure Aggregation: The central server receives the gradients from all participating stores and performs secure aggregation using techniques like secure multi-party computation or homomorphic encryption. This ensures that the individual gradients are not exposed and only the aggregated update is computed. d. Global Model Update: The aggregated gradients are used to update the global model parameters. The updated global model is then distributed back to the participating stores. e. Iteration: Steps a-d are repeated for multiple rounds until the global model converges or a desired level of performance is achieved. Federated learning helps in training accurate models while maintaining data privacy and security. It enables the simulation and decision platform 100 to leverage data from multiple sources without the need for direct data sharing.”)
Regarding Claim 5 and Claim 18, Crabtree in view of Jadon teach The computer-implemented method of claim 4,… and The computer program product of claim 17,…
wherein at least one of: (i) source code corresponding to the at least one machine learning model is not shared during the simulation and (ii) the dataset that is specific to the given one of the different systems is not shared during the simulation. (Crabtree Par. 162-166-“ According to some aspects, model training and optimization computing system 500 provides federated learning techniques via model management subsystem 501. Federated learning is a distributed machine learning approach that enables training models on decentralized data without the need for data sharing. This is particularly useful when dealing with sensitive or confidential data, as it allows for collaborative model training while preserving data privacy. As an example, in the context of the grocery store simulation, suppose there are multiple grocery store chains participating in the simulation, each with their own customer data. Federated learning can be used to train a global model for predicting customer behavior without requiring the individual stores to share their data directly.”)
Regarding Claim 6 and Claim 19, Crabtree in view of Jadon teach The computer-implemented method of claim 1,… and The computer program product of claim 14,…
wherein the collected metrics comprise one or more performance metrics associated with the machine learning models. (Crabtree Par. 123-“ After a simulation run is completed, the user interface can present the results and insights generated by the simulation and decision platform. This can include outputs such as a summary of the simulation results, including key metrics, performance indicators, and outcomes; visual representations of the simulation outcomes, such as charts, graphs, heat maps, or interactive dashboards; comparative analysis of different scenarios or parameter variations to identify the impact of different decisions or interventions; insights and recommendations generated by the AI-driven analysis, highlighting potential optimizations, risks, or opportunities; and explanations and interpretations of the simulation results, providing context and rationale for the observed outcomes.”)
Regarding Claim 7,
The computer-implemented method of claim 6, comprising: maintaining, by the central simulation system, one or more fairness metrics for the simulation. (Crabtree Par. 123; Par. 169-“ According to the embodiment, model training and optimization computing system 500 can provide compliance assistance 504 (e.g., data sharing, model sharing, regulatory and legal constraints, etc.). Privacy-preserving machine learning techniques aim to protect sensitive information during the model training and inference processes. These techniques ensure that the model does not leak or expose private data while still allowing for effective learning and prediction. For example, in the grocery store simulation, customer data may contain sensitive information such as personal identifiers, purchase history, or demographic details. Privacy-preserving techniques can be applied to ensure that this sensitive information is not compromised during the model training and optimization process.”)
Regarding Claim 8,
The computer-implemented method of claim 7, comprising: aggregating the one or more fairness metrics and the one or more performance metrics to select the policy. (Crabtree Par. 170-171-“ d. Federated Learning with Differential Privacy: Federated learning can be combined with differential privacy techniques to provide an additional layer of privacy protection. Noise can be added to the local gradients before aggregation, ensuring that individual contributions are masked. Privacy-preserving techniques help in building trust and ensuring compliance with data protection regulations, such as the European Union's General Data Protection Regulation (GDPR) or the United States' Health Insurance Portability and Accountability Act (HIPAA). The simulation and decision platform can incorporate these techniques to safeguard sensitive information while still leveraging the power of machine learning for optimization and decision-making.; Par. 121”)
Regarding Claim 9,
The computer-implemented method of claim 7, comprising: obtaining real world data corresponding to a particular time of the simulation; and validating at least one of: the one or more collected metrics and the one or more fairness metrics based at least in part on the real world data. (Crabtree Par. 92-“ According to an aspect, the system's ability to generate synthetic data sets from simulated data, machine learning and AI and generative AI models, statistical models and empirical observations (including adjusted/smoothed/augmented) can enable improved experimental design and in particular evaluation of information gain potential from uncertainty reduction efforts, to include real world and synthetic experiments for predictive value, experimental design value (e.g., drug or molecule or materials or quantum or diagnostic research) or for suggesting potential proofs or analytical solutions via symbolic models that might be evaluated for fit.”)
Regarding Claim 10,
The computer-implemented method of claim 1, comprising: providing information to configure at least one other simulator of a new machine learning model; and adding the other simulator to the simulation after at least one of the iterations, wherein the adding comprises adjusting one or more parameters of one or more of the policies based on historical data associated with the new machine learning model. (Crabtree Par. 65-“ According to an aspect of an embodiment, the one or more hardware processors are further configured for: ingesting and preprocessing external data from various formats and sources, such as sensors, databases, application programming interfaces, or user input; aligning and fusing the external data with the simulation model's internal state representation; validating and updating the simulation model's assumptions, parameters, and constraints based on the external data; and exchanging data and insights between the simulation and decision platform and external machine learning models or knowledge bases.; Par. 168; Par. 157-158; Par. 174-175”)
Regarding Claim 12,
The computer-implemented method of claim 1, wherein the different systems are associated with different entities, and wherein the simulators execute at least within different private cloud environments of the different entities. (Crabtree Par. 69-70 –“ According to an aspect of an embodiment, the system may also be deployed across a separated heterogeneous computing environment such as a mix of cloud resources, edge computing devices, content delivery networks or equivalent, end-user computers (e.g. laptops or workstations), mobile devices, wearables, robotics platforms.”; Par. 103; Par. 200-“ . Using external communication devices 70, networks may be configured as local area networks (LANs) for a single location, building, or campus, wide area networks (WANs) comprising data networks that extend over a larger geographical area, and virtual private networks (VPNs) which can be of any size but connect computers via encrypted communications over public networks such as the Internet 75;)
Regarding Claim 12,
The computer-implemented method of claim 1, wherein the simulators execute in different private cloud environments. (Crabtree Par. 69-70; Par. 200-“ . Using external communication devices 70, networks may be configured as local area networks (LANs) for a single location, building, or campus, wide area networks (WANs) comprising data networks that extend over a larger geographical area, and virtual private networks (VPNs) which can be of any size but connect computers via encrypted communications over public networks such as the Internet 75;)
Regarding Claim 13,
The computer-implemented method of claim 1, wherein software is provided as a service in a cloud environment for implementing the central simulation system. (Crabtree Par. 108-“ According to the embodiment, platform 100 is configured as a cloud-based computing platform comprising various system or sub-system components configured to provide functionality directed to the execution of epistemic uncertainty quantification and reduction using simulations and an AI system to provide guidance and support. According to the embodiment, the system architecture comprises the following main components: an artificial intelligence computing system 200, a simulation environment computing system 300, a scenario generation computing system 400, a model training and optimization computing system 500, a data filtering computing system 101, a distributed computational graph (DCG) computing system 102, and one or more databases 103. In some embodiments, systems 101-103, and 200-500 may each be implemented as standalone software applications or as a services/microservices architecture which can be deployed (via platform 100) to perform a specific task or functionality.;)
Regarding Claim 14,
Crabtree teaches
A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to: obtain, by a central simulation system, a plurality of policies to be used for performing a simulation involving multiple machine learning models interacting with a target population, wherein the machine learning models are implemented on different systems (Crabtree Par. 56-57“ Accordingly, the inventor has conceived and reduced to practice, an artificial intelligence-driven Simulation and Experimental Design Decision Platform for reducing epistemic uncertainty in complex systems models and enabling system planning, automation and actuation. The system integrates advanced techniques from artificial intelligence, machine learning, simulation, and uncertainty quantification to generate and run scenarios, monitor progress, and adjust parameters in real-time to achieve user-defined goals. Par. 68 ;Claim 3; Par.193)
provide information for configuring simulators on the different systems, each simulator representing at least the machine learning model of a given one of the different systems; (Crabtree Par. 62-66“ According to an aspect of an embodiment, the one or more hardware processors are further configured for: ingesting and preprocessing external data from various formats and sources, such as sensors, databases, application programming interfaces, or user input; aligning and fusing the external data with the simulation model's internal state representation; validating and updating the simulation model's assumptions, parameters, and constraints based on the external data; and exchanging data and insights between the simulation and decision platform and external machine learning models or knowledge bases According to an aspect of an embodiment, the one or more hardware processors are further configured for assisting users in setting up, configuring, and interpreting the simulation scenarios and results through an interactive user interface that provides guided workflows, contextual help, and natural language processing capabilities.. Par. 68)
perform multiple iterations of the simulation for the plurality of policies, wherein, for each iteration, the central simulation system: (i) predicts a state of the target population, (ii) provides the state of the target population to the simulators, and (iii) collects one or more metrics based on results of the simulators; (Crabtree Par. 93-“ According to an embodiment, the simulation and decision platform may engage in express synthetic data generation to identify specific system states, state transitions, agent or entity phenomena or actions (both individual or group), spatio-temporal dynamics, or to explore potential hypotheses for specific causal relationships or symbolic functions. This may include comparative evaluation of symbolic artificial intelligence and connectionist artificial intelligence (or statistical or machine learning approaches) against modeling simulation results.; Par. 106; 110; Par. 140; Par. 144-145; Par. 148)
Crabtree teaches model analysis and the feature is expounded upon by Jadon:
and select and send one of the policies to at least one of the different systems based on the collected metrics, wherein the at least one system updates its corresponding machine learning model based at least in part on the selected policy; (Jadon Par. 20-21-“ The techniques of this disclosure may address one or more of the challenges described above. For instance, in accordance with an example of this disclosure, a computing system, such as a flow controller, may collect data for a network having a plurality of network devices. For example, the computing system may collect flow data from the network. The flow data may include underlay flow data, overlay flow data, and/or other types of flow data. The computing system may store the data in a database. Furthermore, based on a request for a prediction that is received by the computing system, a ML system may train each respective ML model in a predetermined plurality of ML models to generate a respective training-phase prediction in a plurality of training-phase predictions. The network analysis system may automatically determine a selected ML model in the plurality of ML models based on evaluation metrics for the plurality of ML models. Additionally, the network analysis system may apply the selected ML model to generate the prediction based on the data collected from the network.”; 114-116)
Crabtree and Jadon are directed to machine learning analysis. It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have improve upon data analysis of Crabtree, as taught by Jadon, by additional model selection with a reasonable expectation of success of arriving at the claimed invention. One of ordinary skill in the art would have been motivated to make the modification to the teachings of Crabtree with the motivation of improving selected ML models for specific requests (Jadon Par. 101).
Regarding Claim 20,
Crabtree teaches
A system comprising: a memory configured to store program instructions; a processor operatively coupled to the memory to execute the program instructions to: obtain, by a central simulation system, a plurality of policies to be used for performing a simulation involving multiple machine learning models interacting with a target population, wherein the machine learning models are implemented on different systems (Crabtree Par. 56-57“ Accordingly, the inventor has conceived and reduced to practice, an artificial intelligence-driven Simulation and Experimental Design Decision Platform for reducing epistemic uncertainty in complex systems models and enabling system planning, automation and actuation. The system integrates advanced techniques from artificial intelligence, machine learning, simulation, and uncertainty quantification to generate and run scenarios, monitor progress, and adjust parameters in real-time to achieve user-defined goals. Par. 68 ;Claim 3; Par.191)
provide information for configuring simulators on the different systems, each simulator representing at least the machine learning model of a given one of the different systems; (Crabtree Par. 62-66“ According to an aspect of an embodiment, the one or more hardware processors are further configured for: ingesting and preprocessing external data from various formats and sources, such as sensors, databases, application programming interfaces, or user input; aligning and fusing the external data with the simulation model's internal state representation; validating and updating the simulation model's assumptions, parameters, and constraints based on the external data; and exchanging data and insights between the simulation and decision platform and external machine learning models or knowledge bases According to an aspect of an embodiment, the one or more hardware processors are further configured for assisting users in setting up, configuring, and interpreting the simulation scenarios and results through an interactive user interface that provides guided workflows, contextual help, and natural language processing capabilities.. Par. 68)
perform multiple iterations of the simulation for the plurality of policies, wherein, for each iteration, the central simulation system: (i) predicts a state of the target population, (ii) provides the state of the target population to the simulators, and (iii) collects one or more metrics based on results of the simulators; (Crabtree Par. 93-“ According to an embodiment, the simulation and decision platform may engage in express synthetic data generation to identify specific system states, state transitions, agent or entity phenomena or actions (both individual or group), spatio-temporal dynamics, or to explore potential hypotheses for specific causal relationships or symbolic functions. This may include comparative evaluation of symbolic artificial intelligence and connectionist artificial intelligence (or statistical or machine learning approaches) against modeling simulation results.; Par. 106; 110; Par. 140; Par. 144-145; Par. 148)
Crabtree teaches model analysis and the feature is expounded upon by Jadon:
and select and send one of the policies to at least one of the different systems based on the collected metrics, wherein the at least one system updates its corresponding machine learning model based at least in part on the selected policy; (Jadon Par. 20-21-“ The techniques of this disclosure may address one or more of the challenges described above. For instance, in accordance with an example of this disclosure, a computing system, such as a flow controller, may collect data for a network having a plurality of network devices. For example, the computing system may collect flow data from the network. The flow data may include underlay flow data, overlay flow data, and/or other types of flow data. The computing system may store the data in a database. Furthermore, based on a request for a prediction that is received by the computing system, a ML system may train each respective ML model in a predetermined plurality of ML models to generate a respective training-phase prediction in a plurality of training-phase predictions. The network analysis system may automatically determine a selected ML model in the plurality of ML models based on evaluation metrics for the plurality of ML models. Additionally, the network analysis system may apply the selected ML model to generate the prediction based on the data collected from the network.”; 114-116)
Crabtree and Jadon are directed to machine learning analysis. It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have improve upon data analysis of Crabtree, as taught by Jadon, by additional model selection with a reasonable expectation of success of arriving at the claimed invention. One of ordinary skill in the art would have been motivated to make the modification to the teachings of Crabtree with the motivation of improving selected ML models for specific requests (Jadon Par. 101).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Crabtree et al., US Publication No. 20240348663A1, [hereinafter Crabtree], in view of Jadon et al., US Publication No. 20220004897A1, [hereinafter Jadon] and in further view of Zhang et al., US Publication No. 20190087469A1[hereinafter Zhang] .
Crabtree in view of Jadon teach model analysis and the feature is expounded upon by Zhang:
wherein the simulation simulates a time period of at least one year (Zhang Par. 53-“ AMSS splits the population into “in-market” and “out-of-market” individuals. The number of individuals in the market for a certain category of goods can change over time. Some changes are seasonal, i.e., they repeat in a regular pattern, say once a year. For example, the travel category has an annual seasonality that responds to the school year, national holidays, weather patterns, etc. There can also be more general trends that affect “in-market” population. Examples include the rising adoption of smartphones leads to a growing market for apps, the effect of economic factors on luxury goods, and the effect of gasoline prices on SUV sales. AMSS allows the modeler to specify both seasonal patterns and more general trends in the rate of market participation.”;)
Crabtree, Jadon and Zhang are directed to machine learning analysis. It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have improve upon data analysis of Crabtree in view of Jadon, as taught by Zhang, by additional model analysis with a reasonable expectation of success of arriving at the claimed invention. One of ordinary skill in the art would have been motivated to make the modification to the teachings of Crabtree in view of Jadon with the motivation of demonstrating accuracies, inaccuracies, and/or model bias with respect to a performance metric (Zhang Abstract).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure WO 2019/099478 Al to Crabtree et al.- Abstract A system and methods for operating an outcome modeling engine that incorporates a wide range of input data from various sources including (but not limited to) scientific advances in data analytics, agent-based modeling, discrete event simulation, and the mathematics of entropy to aid in making better decisions about real-world socio-technical systems. ; US20180165587A1 to Crabtree et al.- A system and methods for operating an outcome modeling engine that incorporates a wide range of input data from various sources including (but not limited to) scientific advances in data analytics, agent-based modeling, discrete event simulation, and the mathematics of entropy to aid in making better decisions about real-world socio-technical systems.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Chesiree Walton, whose telephone number is (571) 272-5219. The examiner can normally be reached from Monday to Friday between 8 AM and 5 PM. If any attempt to reach the examiner by telephone is unsuccessful, the examiner’s supervisor, Patricia Munson, can be reached at (571) 270-5396. The fax telephone numbers for this group are either (571) 273-8300 or (703) 872-9326 (for official communications including After Final communications labeled “Box AF”).
Another resource that is available to applicants is the Patent Application Information Retrieval (PAIR). Information regarding the status of an application can be obtained from the (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAX. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, please feel free to contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Applicants are invited to contact the Office to schedule an in-person interview to discuss and resolve the issues set forth in this Office Action. Although an interview is not required, the Office believes that an interview can be of use to resolve any issues related to a patent application in an efficient and prompt manner.
Sincerely,
/CHESIREE A WALTON/Examiner, Art Unit 3624
Prosecution Insights