DETAILED ACTION
Remarks
Applicant presents a request for continued examination dated 16 June 2026 in response to the 26 March 2026 final Office action (the “Previous Action”).
Claims 1, 14 and 18 are amended.
Claims 1 and 3-21 are pending. Claims 1, 14 and 18 are the independent claims.
Any unpersuasive arguments are addressed in the “Response to Arguments” section below. Any new ground(s) of rejection were necessitated by Applicant’s amendments.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 16 June 2026 has been entered.
Examiner Notes
Examiner cites particular columns, paragraphs, figures and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Response to Arguments
Applicant’s arguments are moot in view of the withdrawn rejections and new ground(s) of rejection below, necessitated by Applicant’s amendments.
Claim Objections
The Previous Action’s specification object is withdrawn in view of Applicant’s amendments to the claims.
Claim Rejections - 35 USC § 112
The Previous Action’s § 112 rejections are withdrawn in view of Applicant’s claim amendments unless reproduced herein.
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1 and 3-21 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
As to claim 1, the claim refers to causing cascading failure “in the application model.” There does not appear to be sufficient support in the originally filed specification for these features. For example, par. [0095] discloses that “the application fails.” This suggests the actual application fails, as opposed to failures “in the application model” as claimed. No portion of the originally filed specification appears to support what is claimed.
As to claims 3-13 and 21, the claims are dependent on claim 1 but do not cure the deficiencies of that claim. Accordingly, they are rejected for the same reasons.
As to claim 14, the claim includes the same new matter as claim 1 and is rejected for the same reasons.
As to claims 15-17, the claims are dependent on claim 14 but do not cure the deficiencies of that claim. Accordingly, they are rejected for the same reasons.
As to claim 18, the claim includes the same new matter as claim 1 and is rejected for the same reasons.
As to claims 19-20, the claims are dependent on claim 18 but do not cure the deficiencies of that claim. Accordingly, they are rejected for the same reasons.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-5, 7 and 14-20 are rejected under 35 U.S.C. 103 as being unpatentable over Theimer et al. (US 10,986,013) (art of record – hereinafter Theimer) in view of Kammhoff, (“Algorithms for Efficient Chaos Experiment Selection in Microservice Architectures”) (art of record – hereinafter Kammhoff) in view of Roberts et al. (US 11,080,157) (art of record – hereinafter Roberts).
As to claim 1, Theimer discloses a system to use machine learning systems to create chaos testing scenarios on cloud-based applications (this preamble is not treated as limiting, see M.P.E.P. § 2111.02), comprising:
a storage device in a cloud-based computing network; (e.g., Theimer, Fig. 11 and associated text, col. 28 ll. 1-2: system memory may be configured to store instructions accessible by processors(s) 9010 [in a cloud-based computing network at least as indicated by the cloud of the figure])
one or more hardware processors operating on a server in the cloud-based computing network communicatively coupled to the storage device, wherein the one or more hardware processors use machine learning processes to execute instructions that are stored in the storage device to cause the system (e.g., Theimer, Fig. 11 and associated text, col.27 ll. 40-47: a server [in the cloud-based computing network as indicated by the cloud in the figure] that implements the technologies described herein may include a general-purpose computer system that includes or is configured to access one or more computer-accessible media. FIG. 11 illustrates such a general-purpose computing device 9000; col. 28 ll. 1-2: system memory may be configured to store instructions accessible by processors(s) 9010; col. 29 ll. 4-14: a medium may be included in device 9000 as memory 9020. A plurality of media may store instructions that when executed on one or more processors implement the methods described above [which include machine learning processes, see below]) to:
one or more chaos testing simulation scenarios that target the points of failure (e.g., Theimer, col. 11 ll. 15-20: fault recommendation engines 159 for suggesting fault injection modes and/or schedules [chaos testing simulation scenarios, see, e.g., par. [0037] of the specification]; col. 22 ll. 35-40: a client may submit a request 806 to obtain such recommendations. The corresponding response message 808 may include descriptors of one or more categories of faults; col. 5 ll. 50-55: an individual descriptor may indicate, for example one or more target resources or artifacts [points of failure] whose behavior, operation or content is going to be to modified as a result of fault injection) and cause a cascading failure, each chaos testing simulation scenario comprising a plurality of disruptions or failures scheduled consecutively; (e.g., Thiemer, col. 5 ll. 59-60: the catalog may include descriptors covering a wide variety of faults; col. 4 ll. 1-4: individual descriptors indicating respective sequences of events [a sequence of events being consecutive events]; col. 24 l. 54: events that represent faults; col. 14 ll. 46-57: faults may represent failures 418; Fig. 3 and associated text, col. 13 ll. 48-63: event E4, followed by E33, followed by error/failure E77 may be detected by analytics tools. The tools may detect the correlation E77 between the combination of E4 and E33 as an error-causing multi-service fault; col. 4 ll. 23-27: causal chains of events may be identified and included in the catalog)
perform chaos testing based on the one or more chaos testing simulation scenarios, causing the failure (e.g., Theimer, col. 11 ll. 24-28: clients may use recommended fault injection strategies [chaos testing simulation scenarios]; col. 3 ll. 15-20 top deliberately inject a fault “(i.e., to cause an undesired event or sequence of events that may in turn, in at least some cases, cause an application to misbehave or fail)”).
generate at least one recommendation to revise code of the application based on the chaos testing (e.g., Theimer, col. 5 ll. 8-27: records generated may be examined to detect the impact of different faults on the application. Using such information provided by the FIS, the client may introduce changes [revisions] into App1 [which is code]. The FIS may also be able to provide recommendations on the kinds of changes that a client may need to make to make an application more robust)
Theimer does not explicitly disclose to create an application model of an application based on input data received from a network of applications, the application comprising a plurality of microservices that make up the application, wherein the application model represents a plurality of dependencies between or among the plurality of microservices; using a machine learning system and based on the application model, simulate one or more impacts of disruptions or failures on the application; using the machine learning system and based on the one or more impacts, identify points of failure of the application in the plurality of microservices; failure in the application model; and to perform chaos testing based on the one or more chaos testing simulation scenarios, causing the failure in the application model.
However, in an analogous art, Kammhoff discloses:
to create an application model of an application based on input data received from a network of applications, the application comprising a plurality of microservices that make up the application, wherein the application model represents a plurality of dependencies between or among the plurality of microservices; (e.g., Kammhoff, p. 2 Fig. 1.1 and associated text and last par: the architectural model, containing information about the different services of a microservice architecture [application], is extracted by analyzing traces [and see figure the architectural information is from the “System”, i.e., the microservice architecture] p. 8 par. 2: microservices generally communicate over a network; p. 18 par. 1: the information provided by the simplified model is: the operations a service [microservice] has, Dependencies between operations)
failure in the application model (see immediately above)
perform chaos testing based on the one or more chaos testing simulation scenarios, causing the failure in the application model (see immediately above).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the machine learning model and chaos simuation scenarios causing cascading failure taught by Theimer to include creating an application model of an application based on input data received from a network of applications, the application comprising a plurality of microservices that make up the application, wherein the application model represents a plurality of dependencies between or among the plurality of microservices; causing the failure in the application model and performing chaos testing based on the one or more chaos testing simulation scenarios, causing the failure in the application model, as taught by Kammhoff. Creating such a model would provide the advantage of a means of describing the services of the application, their relationships and any used reliance patterns. (See Kammhoff, p. 16 Sec. 2.6.1 and Theimer at col. 11 ll. 15-20, 28-31). Performing chaos testing that causes a failure in the application model would provide the advantage of means of avoiding problems for a user of the live system. (See Kammhoff, p. 2 last par.).
Further, in an analogous art, Roberts discloses:
using a machine learning system and based on the application model, simulate one or more impacts of disruptions or failures on the application; (e.g., Roberts, col. 4 ll. 54-62: a model [application model] of computing resources in the system architecture may be described using a resource specification document; col. 5 ll. 9-12: architecture analysis 120 may use, as input, a resource specification document; col. 5 ll. 62-66: using architecture analysis 120, the system 100 may determine one or more similar system architectures; col. 6 ll. 15-21: the system may generate recommended resiliency tests to determine resiliency of the tested architecture against known scenarios associated with similar architectures; col. 10 ll. 10-15: the request interception component 445 may hold requests to simulate port latency to simulate failure scenarios; col. 3 ll. 58-63: the system may analyze the results of the tests to determine one or more effects of the tested behaviors)
using the machine learning system and based on the one or more impacts, (e.g., Roberts, col. 5 ll. 69-61: known failures scenarios or risks 172A-172N may be determined [learned] using automated resiliency testing for a variety of system architectures; col. 2 ll. 36-38: to run the tests to determine the impact of the tested behaviors on the resiliency of the system) identify points of failure of the application (e.g., Roberts, col. 9 ll. 24-26: the system 100 may perform resiliency testing 140 for a later version of the same system architecture; col. 5 ll. 30-34: the recommendations may be generated based on known [learned] failure scenarios or risks associated with the resources in the system architecture; col. 5 ll. 39-42: for example, the repository 170 may indicate that a data streaming service has common failures associated with a timeout value) in the plurality of microservices, (e.g., Roberts, col. 1 ll.. 23-32: a distributed system operated by an online store may include an ordering system that processes customer orders. Operations may be implemented using multiple services [microservices] or other computing resources) and predict one or more chaos testing simulation scenarios that target the points of failure (e.g., Roberts, col. 5 ll. 35-37:the repository 170 may indicate that particular types of computing resources are vulnerable to particular failures or risks; col. 2 ll. 32-36: a resiliency analyzer may identify potential failures via a repository of known failure scenarios, recommend resiliency tests [chaos testing scenarios] associated with the potential failures, and run the tests; col. 3 ll. 58-63: the system may analyze the results of the tests to determine one or more effects of the tested behaviors. For example, the effects may include whether or not the performance of a system architecture is vulnerable to behaviors associated with the known failure scenario).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the application model and recommendation of chaos testing scenarios that target points of failure in a plurality or microservices and cause a cascading failure taught by Theimer/Kammhoff, such that a machine learning system is used to simulate one or more impacts of disruptions or failures on the application based on the application model; and the machine learning system is used to identify points of failure of the application in the plurality of microservices based on the one or more impacts and predict one or more chaos testing simulation scenarios that target the points of failure, as taught by Roberts, as Roberts would provide the advantage of a means of testing points in the application known to be vulnerable to certain scenarios and determining whether or not the system is resilient to those scenarios. (See Roberts, col. 5 ll. 30-34, col. 7 ll. 30-35).
As to claim 3, Theimer/Kammhoff/Roberts discloses the system of claim 1 (see rejection of claim 1 above), Theimer further discloses:
further comprising application code instructions to cause the system to recognize patterns of code behavior that may lead to instability or failures (e.g., Theimer, col. 4 ll. 20-27: causal chains of application misbehavior causing events may be identified and included in the catalog).
As to claim 4, Theimer/Kammhoff/Roberts discloses the system of claim 3 (see rejection of claim 3 above), Theimer further discloses:
wherein a chaos simulation scenario comprises one or more instructions to inject faults, introduce latency, or simulate resource constraints in areas of the code where recognized patterns are detected (e.g., Theimer, col. 11 ll. 15-20: fault recommendation engines 159 for suggesting fault injection modes [chaos simulation scenarios]; col. 22 ll. 35-40: a client may submit a request 806 to obtain such recommendations. The corresponding response message 808 may include descriptors of one or more categories of faults; col. 4 ll. 1-4: a catalog of fault category descriptors with individual descriptors describing sequences of one or more events at one or more resources).
As to claim 5, Theimer/Kammhoff/Roberts discloses the system of claim 1 (see rejection of claim 1 above), but Theimer does not explicitly disclose further comprising application code instructions to cause the system to prioritize the one or more chaos testing simulation scenarios based on one or more of code complexity, criticality of components, and historical failure data.
However, in an analogous art, Kammhoff discloses:
further comprising application code instructions to cause the system to prioritize the one or more chaos testing simulation scenarios based on one or more of code complexity, criticality of components, and historical failure data (e.g., Kammhoff, p. 71 Sec. 4.5 using fault injection results to change the priority of future fault injections [chaos testing simulation scenarios]; p. 72 ll. 1-9: if a fault injection reveals a fault, it moves up to the next highest priority queue. If a fault inject doesn’t reveal a fault, it moves to the next lower priority queue).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Theimer by prioritizing the chaos testing simulation scenarios based on one or more of code complexity, criticality of components, and historical failure data, as taught by Kammhoff, as Kammhoff would provide the advantages of a means of revealing faults as early as possible and a means of utilizing a scenario that returns the best possible results. (See Kammhoff, p. 13 Sec. 2.5, p. 14 Sec. 2.5.2 par. 1).
As to claim 7, Theimer/Kammhoff/Roberts discloses the system of claim 1 (see rejection of claim 1 above), but Theimer does not explicitly disclose wherein a particular machine learning process of the machine learning processes is a large language model.
However, in an analogous art, Kammhoff discloses:
wherein a particular machine learning process of the machine learning processes is a large language model (e.g., Kammhoff, p. 27 Sec. 2.7.4 artificial neural networks [large language models per par. [0138] of the specification] are used to solve a variety of problems).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the machine learning processes of Theimer to include machine learning processes such as a large language model, as taught by Kammhoff, as Kammhoff would provide the advantages of massive parallelism, generalization ability, adaptivity and fault tolerance. (See Kammhodff, p. 27 Sec. 2.7.4)
As to claim 14, it is a method claim having limitations substantially the same as those of claim 1. Accordingly, it is rejected for substantially the same reasons.
As to claim 15, Theimer/Kamhoff/Roberts discloses the system of claim 14 (see rejection of claim 14 above), but Theimer does not explicitly disclose further comprising mapping the dependencies of the application.
However, in an analogous art, Kammhoff discloses:
further comprising mapping the dependencies of the application (e.g., Kammhoff pp. 17-18 Listing 2.2: see code, it includes “dependencies” shown association with [mapped to] an operation and microservice).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Theimer to include further comprising mapping the dependencies of the application, as taught by Kammhoff, as Kammhoff would provide the advantage of a means of describing dependencies between operations of a microservice. (See Kammhoff, p. 16 Sec. 2.6.1 par. 3).
As to claim 16, it is a method claim whose limitations are substantially the same as claim 4. Accordingly, it is rejected for substantially the same reasons.
As to claim 17, it is a method claim whose limitations are substantially the same as claim 7. Accordingly, it is rejected for substantially the same reasons.
As to claim 18, it is a computer program product claim having limitations substantially the same as those of claim 1. Accordingly, it is rejected for substantially the same reasons.
Further limitations, disclosed by Theimer, include:
a non-transitory computer-readable medium having computer-readable program instructions embodied thereon, the computer-readable program instructions causing one or more processors (e.g., Theimer, col. 29 ll. 4-14) to: (see rejection of claim 1 above).
As to claim 19, it is a computer program product claim whose limitations are substantially the same as claim 5. Accordingly, it is rejected for substantially the same reasons.
As to claim 20, it is a computer program product claim whose limitations are substantially the same as claim 6. Accordingly, it is rejected for substantially the same reasons.
Claims 6 is rejected under 35 U.S.C. 103 as being unpatentable over Theimer (US 10,986,013) in view of Kammhoff, (“Algorithms for Efficient Chaos Experiment Selection in Microservice Architectures”) in view of Roberts et al. (US 11,080,157) in further view of Arunachalam et al. (US 10,795,793) (art of record – hereinafter Arunachalam) and Rossi et al. (US 2009/0254887) (art of record – hereinafter Rossi).
As to claim 6, Theimer/Kammhoff/Roberts discloses the system of claim 5 (see rejection of claim 5 above), and further discloses chaos testing simulation scenarios (see rejection of claim 1 above) but does not explicitly disclose further comprising application code instructions to cause the system to: monitor a status of the application during the performance of the chaos testing; compare the status to an operational status that occurs when chaos conditions are not present and an expected operational status of the application when under chaos conditions; and revise the one or more chaos testing simulation scenarios based on the comparison.
However, in an analogous art, Arunachalam discloses further comprising application code instructions cause the one or more hardware processors to:
monitor a status of the application during performance of the testing; (e.g., Arunachalam, col. 11 ll. 57-60: the system monitors the distributed computing system to record an outcome of the simulated system failure)
compare the status to an operational status that occurs when chaos conditions are not present and an expected operational status of the application when under chaos conditions; (e.g., Arunachalam, col. 12 ll. 1-3: at block 250 , the system determines whether the monitored outcome matches the expected outcome of the simulated system failure; col. 6 ll. 31-38: the expected output of the simulated failures scenario may include an expected state to verify that the system has failed according to the failures [expected operational status of the application when under chaos conditions] and an expected state after recover operations have been invoked [expected operational status that occurs when chaos conditions are not present]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the execution of chaos scenarios used and chaos testing of an application of Theimer/Kammhoff/Roberts, by monitoring the status of the application during the testing and comparing the status to an operational status that occurs when chaos conditions are not present and an expected operational status of the application when under chaos conditions, as taught by Arunachalam, as Arunachalam would provide the advantage of a means of verifying when the application both fails as expected and recovers as expected in response to the chaos testing simulation scenarios. (See Arunachalam, col. 6 ll. 31-38).
Further, in an analogous art, Rossi discloses to:
revise, based on comparing the status to the operational status, the one or more scenarios (e.g., Rossi, par. [0017]: the test scenario may pass when the actual results match the corresponding expected results; par. [0020]: the curve 215 plots the cumulative weight of the executed test scenarios that actually passed; par. [0021]: the expected trend curve 210 and the actual trend curve 215 are compared to track the progress of the test process; par. [0053]: the variance indicator represents a different between the actual trend and the expected trend; par. [0054]: when the variance indicator reaches the threshold, the test process may be deemed not proceeding as planned. Suitable recovery actions may be taken. It may be possible to simplify the execution of the remaining test scenarios [revise the one or more scenarios]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the execution of chaos testing simulation scenarios, monitoring and comparison of Theimer/Kammhoff/Roberts/Arunachalam noted above by revising the one or more scenarios based on the comparison, as taught by Rossi, as Rossi would provide the advantage of a means of ensuring the test process proceeds as planned. (See Rossi, par. [0030]).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Theimer (US 10,986,013) in view of Kammhoff, (“Algorithms for Efficient Chaos Experiment Selection in Microservice Architectures”) in view of Roberts (US 11,080,157) in further view of Arunachalam (US 10,795,793) and Brown (“Machine learning explained”)
As to claim 8, Theimer/Kammhoff/Roberts discloses the system of claim 7 (see rejection of claim 1) but does not explicitly disclose wherein the large language model further generates the one or more chaos testing simulation scenarios based on textual descriptions of objectives provided by a user.
However, in an analogous art, Arunachalam discloses:
generat[ing] the one or more chaos testing simulation scenarios based on textual descriptions of objectives provided by a user (e.g., Arumanchalam, col. 5 ll. 9-13, a simulated failure scenario, which may be a randomized scenario used to test an application. These randomized scenarios may be used in an overall chaos engineering approach; col. 4 ll. 22-24: utilizing natural language formats to allow develops to define simulated failure scenarios and expected output comes of the scenarios; col. 3 l. 65 – col. 4 l. 2: a failure analysis system receives the natural language inputs defining a failure scenario [textual descriptions], translates the natural language inputs into executable code that executes the identified type of failure according to the properties of the simulated failure [generates chaos testing simulation scenarios]; col 13 ll. 42-50: the natural language definition 410 defines variables to be set in the executable code [see figure 4, at objective being, e.g., alarm = <state2> or “terminate process on cloud instances]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the large language model and chaos testing simulation scenarios of Theimer/Kammhoff/Roberts to include generating the one or more chaos testing simulation scenarios based on textual descriptions of objects provided by a user, as taught by Arumanchalam, as Arumanchalam would provide the advantage of a mean for developers to perform robust testing without manually writing code or knowing specific information about the computer systems on which the tested application is deployed. (See Arumanchalam, col. 4 ll. 22-31).
Further, in an analogous art, Brown discloses:
wherein the large language model performs the task (e.g., Brown, p. 2 par. 4: artificial intelligence systems are used to perform complex tasks in a way that is similar to humans; p. 4 par. 3: successful machine learning algorithms can do different things; p. 6 “neural networks”: neural networks are a commonly used, specific class of machine learning algorithms).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the generation of documentation for chaos testing simulation scenarios taught by Theimer/Kammhoff/Roberts such that a large language model generates the documentation, as taught by Brown. Brown would provide the advantages of a less time-consuming means of training a computer to perform the task and a means for the system to improve over time. (See Brown p. 3 par. 2 and last par.).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Theimer (US 10,986,013) in view of Kammhoff, (“Algorithms for Efficient Chaos Experiment Selection in Microservice Architectures”) in view of Roberts (US 11,080,157) in further view of Rohithakshappa et al. (US 11,537,502) (art of record – hereinafter Rohithakshappa).
As to claim 9, Theimer/Kammhoff/Roberts discloses the system of claim 7 (see rejection of claim 1 above) but does not explicitly disclose wherein the large language model analyzes documentation and code comments of the application to extract data related to architecture, and the plurality of dependencies of the application.
However, in an analogous art, Rohithakshappa discloses:
wherein the large language model analyzes documentation and code comments of the application to extract data related to architecture and the plurality of dependencies of the application (e.g., Rohithakshappa, col. 16 ll. 34-48: the ML anomaly detection model 156b analyzes method codes “(e.g., comprising JavaDoc comments associated with the respective execution method)” [documentation and code comments]. Here, the ML anomaly detection model 156b is further trained to read class files to identify method attributes of each of the training execution methods. Specifically, the ML anomaly detection model 156b is trained to detect method attributes such as variable types, operations used in the execution method [data related to architecture], use of any libraries [dependencies] or application programming interfaces, annotations within the execution method, and/or the like; col. 17 ll. 20-25: the ML anomaly detection model 156b is a deep learning model program).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the large language model of Theimer/Kammhoff/Roberts such that the large language model analyzes documentation and code comments of the application to extract data related to the architecture, dependencies, and critical components of the application as taught by Rohithakshappa, as Rohithakshappa would provide the advantage of a means of acquiring data related to the architecture and dependencies specified in the application code. (See Rohithakshappa, col. 16 ll. 34-48).
Claims 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over Theimer (US 10,986,013) in view of Kammhoff, (“Algorithms for Efficient Chaos Experiment Selection in Microservice Architectures”) in view of Roberts (US 11,080,157) in further view of Brown, “Machine Learning, explained” (art of record – hereinafter Brown).
As to claim 10, Theimer/Kammhoff/Roberts discloses the system of claim 7 (see rejection of claim 1 above) but does not explicitly disclose wherein the large language model further reviews one or more changes to the code and assess their impact on chaos engineering practices.
However, in an analogous art, Roberts discloses:
further review[ing] one or more changes to the code and assess their impact on chaos engineering practices (e.g., Roberts, col. 9 ll. 51-61: if, based on resiliency testing [chaos engineering practices, see above], version 1 of the system architecture was not susceptible to a particular behavior or risk but version 2 is, then the system 100 may report the difference. Version comparison 155 may determine any differences [code changes, because the architectures comprise software] between version 1 of the architecture 181B and version 2 of the architecture and the system may notify the developer that the differences may be responsible for the failure).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the large language model of Theimer/Kammhoff to include reviewing code changes to the code and assessing their impact on chaos engineering practices, as taught by Roberts, as Roberts would provide the advantage of a means of tracking the resiliency of a system over time. (See Roberts, col. 9 ll. 40-42).
Further, in an analogous art, Brown discloses
wherein the large language model performs the task (e.g., Brown, p. 4 par. 3: successful machine learning algorithms can do different things; p. 6 pars. 4-5: deep learning networks are neural networks with many layers. Like neural networks, deep learning is modeled on the way the human brain works and powers many machine learning uses, like autonomous vehicles, chatbots, and medical diagnostics; p. 4 par. 4: the function of a machine learning algorithm can be predictive, meaning the system uses the data to predict what will happen).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the reviewing of code changes and assessing their impact on chaos engineering practices taught by Theimer/Kammhoff/Roberts such that a large language model performs certain tasks, as taught by Brown, a as Brown would provide the advantage of processing extensive amounts of data and explaining what happened (see Brown, p. 6 “Deep Learning”, p. 4 par. 4) as well as a less time-consuming means of training a computer to perform the task and a means for the system to improve over time. (See Brown p. 3 par. 2 and last par.).
As to claim 11, Theimer/Kammhoff/Roberts the system of claim 7 (see rejection of claim 7 above) but does not explicitly disclose wherein the large language model further develops natural language interfaces to interact with users.
However, in analogous art, Brown discloses:
wherein the large language model further develops natural language interfaces to interact with users (e.g., Brown, p. 6 par. 2: neural networks [large language models] are a commonly used class of machine learning algorithms; p. 7 par. 4: companies are using machine learning in several ways, including: p. 8 par. 3: chatbots in which customers or clients don’t speak to humans but instead interact with a machine. These algorithms use machine learning and natural language processing, with the bots learning from past conversations to come up with appropriate responses).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the processes of Theimer/Kammhoff/Roberts to include a large language model developing natural language interfaces to interact with users, as taught by Brown, as Brown would provide the advantage of a means providing appropriate responses to users without the need for a human. (See Brown p. 8 par. 3).
As to claim 12, Theimer/Kammhoff/Roberts discloses the system of claim 7 (see rejection of claim 7 above), Theimer further discloses:
generat[ing] documentation for the one or more chaos testing simulation scenarios (e.g., Theimer, col. 4 ll. 64-67: an overall resiliency report may be provided to a client for the client’s applications, indicating the extent to which the applications were able to withstand various types of fault events).
Theimer/Kammhoff/Roberts does not explicitly disclose wherein the large language model performs this task.
However, in an analogous art, Brown discloses
wherein the large language model generates documentation (e.g., Brown, p. 4 par. 3: successful machine learning algorithms can do different things; p. 6 pars. 4-5: deep learning networks are neural networks with many layers. Like neural networks, deep learning is modeled on the way the human brain works and powers many machine learning uses, like autonomous vehicles, chatbots, and medical diagnostics; p. 4 par. 4: the function of a machine learning system can be descriptive, meaning that the system uses the data to explain what happened).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the generation of documentation for chaos testing simulation scenarios taught by Theimer/Kammhoff/Roberts such that a large language model generates the documentation, as taught by Brown, as Brown would provide the advantage of a means of and explaining what happened with extensive amounts of data (See Brown, p. 6 “Deep Learning”, p. 4 par. 4). Brown would provide the advantages of a less time-consuming means of training a computer to perform the task and a means for the system to improve over time. (See Brown p. 3 par. 2 and last par.).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Theimer (US 10,986,013) in view of Kammhoff, (“Algorithms for Efficient Chaos Experiment Selection in Microservice Architectures”) in view of Roberts (US 11,080,157) in further view of Gudka et al. (US 2020/0167255) (art of record – hereinafter Gudka) and Brown, (“Machine Learning, explained”)
As to claim 13, Theimer/Kammhoff/Roberts discloses the system of claim 7 (see rejection of claim 7 above), but does not explicitly disclose wherein the large language model generates natural language reports summarizing results of the one or more chaos testing simulation scenarios
However, in an analogous art, Gudka discloses:
generat[ing] natural language reports summarizing results of the one or more testing simulation scenarios (e.g., Gudka, Fig. 3 and associated text, par. [0044]: the simulation results 300 may result from simulations performed on simulation instances [see figure, the results include natural language]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the large language model and chaos testing simulation scenarios of Theimer/Kammhoff/Brown such that the model of Theimer/Kammhoff/Brown generates natural language reports summarizing the results of testing the simulation scenarios, as taught by Gudka, as Gudka would provide the advantage of a means of providing results readable by a human. (See Fig. 3 of Gudka).
Further, in an analogous art, Brown discloses:
wherein the large language model further generates reports summarizing results (e.g. Brown, p. 4 par. 4: the function of a machine learning system can be descriptive, meaning that the system uses the data to explain what happened; p. 6 “Deep learning”: deep learning powers many machine learning uses, such as chatbots).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the generation of documentation for chaos testing simulation scenarios taught by Theimer/Kammhoff/Roberts such that a large language model generates the docuymentation, as taught by Brown, as Brown would provide the advantage of a means of and explaining what happened with extensive amounts of data (See Brown, p. 6 “Deep Learning”, p. 4 par. 4). Brown would provide the advantages of a less time-consuming means of training a computer to perform the task and a means for the system to improve over time. (See Brown p. 3 par. 2 and last par.).
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Theimer (US 10,986,013) in view of Kammhoff, (“Algorithms for Efficient Chaos Experiment Selection in Microservice Architectures”) in view of Roberts (US 11,080,157) in further view of Arunachalam (US 10,795,793).
As to claim 21, Theimer/Kammhoff discloses the system of claim 1 (see rejection of claim 1 above) Theimer further discloses:
wherein the instructions for performing the chaos testing based on the one or more chaos testing simulation scenarios cause the one or more hardware processors to perform operations (see rejection of claim 1 above and Theimer at col. 29 ll. 4-14) and
an application response to chaos testing (e.g., Theimer, col. 4 l. 64 – col. 5 l. 8: an oerall resiliency report may be provided to client indicating the extent to which the applications were able to withstand various types of fault event [again, the injecting of faults being chaos testing], or the types of application errors/failures that occurred as a result of the introduction of the faults. For example, a client may be informed that some response to requests directed to App1 were delayed slightly).
Theimer/Kammhoff does not explicitly disclose operations comprising: receiving application data associated with the application, wherein the performance metrics include application data includes a first set of performance metrics during standard operating conditions and a second set of performance metrics during the disruptions or the failures, comparing the application data with model data of the application model; and based on comparing the application data with the model data of the application model, identifying one or more anomalies during the disruptions or the failures; and generating one or more alerts based on the one or more anomalies identified during the disruptions or the failures.
However, in an analogous art, Roberts discloses:
operations comprising:
receiving application data associated with the application, wherein the performance metrics include application data includes a first set of performance metrics during standard operating conditions and a second set of performance metrics during the disruptions or the failures, (e.g., Roberts, col. 7 ll. Resiliency metrics may 182A-182Z may be collected before [during standard operating conditions], during [during the disruptions or failures] and after a resiliency test. The “before” metrics may be collected to determine a steady state before fault injection is performed)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the operations of Theimer/Kammhoff to include receiving application data associated with the application, wherein the performance metrics include application data includes a first set of performance metrics during standard operating conditions and a second set of performance metrics during upset conditions, and wherein the upset condition comprise network delays or power interruptions, as taught by Roberts, as Roberts would provide the advantage of a means of determining a steady state before fault injection is performed, as well as a means of determine the effects of disruption or failure. (See Roberts, col. 7 l. 55 – col. 8 l. 3).
Further, in an analogous art, Arunachalam discloses:
comparing the application data with model data of the application model; (e.g., Arunachalam, col. 3 59-60: performing failure testing on an application; col. 3 l. 65 – col. 4 l. 7: a failure analysis system monitors a computing system for the outcome of the simulated failure. The failure analysis system compares the outcome of the simulated failure to an expected outcome)
based on comparing the application data with the model data of the application model, identifying one or more anomalies during the disruptions or the failures; (e.g., Arunachalam, col. 3 59-60: performing failure testing on an application; col. 4 ll. 3-15: The failure analysis system compares the outcome of the simulated failure to an expected outcome. If the application did not respond to the simulated failure scenario in the expected manner, the system can flag the simulated failure scenario for further review by a development team) and
generating one or more alerts based on the one or more anomalies identified during the disruptions or the failures (see immediately above).
It would have been obvious to one of ordinary skill in the art to modify the receiving of performance metrics, model data and response of an application to chaos testing of Theimer/Kammhoff/Roberts/Beck by comparing the metrics with model data and predicting a response based on the comparison, as taught by Arunachalam, as Arunachalam would provide the advantage of a means of ensuring the application responds to disruptions or failures in the expected manner. (See Arunachalam, col. 4 ll. 11-21).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TODD AGUILERA whose telephone number is (571)270-5186. The examiner can normally be reached M-F 11AM - 7:30PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hyung S Sough can be reached at (571)272-6799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TODD AGUILERA/Primary Examiner, Art Unit 2192
