DETAILED ACTION
Remarks
The present application was 30 October 2024 and is a continuation of 18/387,348 filed on 6 November 2023, which is a continuation-in-part of 17/961,106 (now issued patent no. 11,847,046) filed on 6 October 2022.
Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner Notes
Examiner cites particular columns, paragraphs, figures and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged. Applicant has not complied with one or more conditions for receiving the benefit of an earlier filing date as follows:
The later-filed application must be an application for a patent for an invention which is also disclosed in the prior application (the parent or original nonprovisional application or provisional application). The disclosure of the invention in the parent application and in the later-filed application must be sufficient to comply with the requirements of 35 U.S.C. 112(a) or the first paragraph of pre-AIA 35 U.S.C. 112, except for the best mode requirement. See Transco Products, Inc. v. Performance Contracting, Inc., 38 F.3d 551, 32 USPQ2d 1077 (Fed. Cir. 1994).
The disclosure of the prior-filed application Application No. 18/387,348 (but also 17/961,106) fails to provide adequate support or enablement in the manner provided by 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph for one or more claims of this application. Specifically, the prior-filed application fails to provide adequate support or enablement for:
The “…wherein the application model represents a plurality of dependencies between or among the plurality of services…” in claim 1;
The “…chaos testing scenario comprising timing information for applying the plurality of chaos tests…” in claim 1;
The “…wherein the chaos testing scenario is predicted to cause a cascading failure of the application…” in claim 1;
The “…wherein the machine learning algorithm is trained using interactions between historical chaos tests within historical chaos testing data…” in claim 1;
The “…identify, based on the testing scenario applied to the set of microservices, one or more points of failure of the application…” in claim 1;
The “…wherein the application model represents a plurality of dependencies between or among the plurality of services…” in claim 2;
The “…chaos testing scenario comprising timing information for applying the plurality of chaos tests…” in claim 2;
The “…wherein the machine learning algorithm is trained to predict, based on interactions between historic tests, testing scenarios…” in claim 2;
The “…identifying…based on the testing scenario applied to the set of microservices, one or more points of failure of the application…” in claim 2;
The “…wherein the testing scenario is predicted, based on training of the machine learning algorithm, to cause a cascading failure of the particular application…” in claim 3;
The “…wherein the application model represents a plurality of dependencies between or among the plurality of services…” in claim 11;
The “…chaos testing scenario comprising timing information for applying the plurality of chaos tests…” in claim 11;
The “…wherein the machine learning algorithm is trained to predict, based on interactions between historic tests, testing scenarios…” in claim 11;
The “…identifying based on the testing scenario applied to the set of microservices, one or more points of failure of the application…” in claim 11;
The “…wherein the testing scenario is predicted, based on training of the machine learning algorithm, to cause a cascading failure of the particular application…” in claim 12.
Specification
The specification is objected to for the following informalities:
It fails to provide clear support or antecedent basis for the terms: “timing information”, “interactions between historical chaos tests” or “interactions between historic tests”. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o).
It fails to provide proper antecedent basis for claimed features noted above as lacking priority the parent application(s).
Drawings
The drawings are acceptable for examination purposes.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 6, 8, 11-12, 15 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Theimer et al. (US 10,986,013) (hereinafter Theimer) in view of Kammhoff (“Algorithms for Efficient Chaos Experiment Selection in Microservice Architectures”) (hereinafter Kammhoff).
As to claim 1, Theimer discloses a system for using machine learning to predict testing scenarios related to cascading failures of cloud-based applications, (this preamble is not treated as limiting, see M.P.E.P. § 2111.02), the system comprising:
a storage device in a cloud-based computing network; (e.g., Theimer, Fig. 11 and associated text, col. 28 ll. 1-2: system memory may be configured to store instructions accessible by processors(s) 9010 [in a cloud-based computing network at least as indicated by the cloud of the figure]) and
one or more processors operating in the cloud-based computing network communicatively coupled to the storage device, wherein the one or more processors execute instructions that are stored in the storage device to cause the system (e.g., Theimer, Fig. 11 and associated text, col.27 ll. 40-47: a server [in the cloud-based computing network as indicated by the cloud in the figure] that implements the technologies described herein may include a general-purpose computer system that includes or is configured to access one or more computer-accessible media. FIG. 11 illustrates such a general-purpose computing device 9000; col. 28 ll. 1-2: system memory may be configured to store instructions accessible by processors(s) 9010; col. 29 ll. 4-14: a medium may be included in device 9000 as memory 9020. A plurality of media may store instructions that when executed on one or more processors implement the methods described above) to:
a chaos testing scenario comprising a plurality of chaos tests and timing information for applying the plurality of chaos tests to a set of microservices of the plurality of microservices of the application, (e.g., Theimer, col. 4 ll. 4-5: to test application resiliency to faults; col. 2 ll. 62-63: resources of one or more computing services [microservices] may be used to execute the logic of the application; col. 8 ll. 30-33: a client may submit requests to FIS 150 to introduce combinations of one or more faults into the environments within which the applications are being run; col. 8 ll. 49-58: a client may specify the fault injection mode “(e.g., whether randomly selected faults are to be injected, whether a deterministic sequence of faults selected by the client is to be used, etc.)” the time period for which faults are to be introduced) wherein the chaos testing scenario is predicted (e.g., Theimer, col. 1 ll. 40-45: anticipating all the different events and event sequences that could potentially lead to application errors or sub-par behavior becomes extremely difficult for clients; col. 11 ll. 30-39: the FIS may in some embodiments identify as recommended faults, a set of faults that (1) have occurred at a rate above a threshold in the past (b) have led to application misbehaviors in the past; col. 20 ll. 50-57: for some fault categories in the catalog a “severity” attribute may be provided. Such a severity attribute may be represented as a number, with 1 representing a low probability of causing a failure at an application and 100 representing a high probability) to cause a cascading failure of the application (e.g., Thiemer, col. 5 ll. 59-60: the catalog may include descriptors covering a wide variety of faults; col. 4 ll. 1-4: individual descriptors indicating respective sequences of events; col. 24 l. 54: events that represent faults; col. 14 ll. 46-57: faults may represent failures 418; Fig. 3 and associated text, col. 13 ll. 48-63: event E4, followed by E33, followed by error/failure E77 may be detected by analytics tools. The tools may detect the correlation E77 between the combination of E4 and E33 as an error-causing multi-service fault; col. 4 ll. 23-27: causal chains of events may be identified and included in the catalog)
perform the chaos testing scenario by applying the plurality of chaos tests to the set of microservices of the application according to the timing information; (e.g., Theimer, col. 23 ll. 43-45: a client may submit an InitiateFaultInjection request 814 to cause the FIS to begin introducing faults at a target environment; col. 8 ll. 49-58: a client may specify the time period for which faults are to be introduced; col. 6 ll. 28-30: a target environment associated with an application of the client which utilizes one or more services)
identify, based on the chaos testing scenario applied to the set of microservices, one or more points of failure of the application; (e.g., Theimer, col. 10 ll. 56-65: resilience results generators 158 may provide results of the requested fault injections. The results may be a list of faults that were injected. In other embodiments the FIS may capture data indicating the impact of the fault injections, e.g., whether one or more processes implementing the application failed [i.e.. those processes are points of failure]; col. 5 ll. 10-15: operation records used by the application may be examined to detect the impact; col. 13 ll. 37-40: service-C operation records 312 include indications of two occurrences of errors/failures E77 at times corresponding to timestamp T753 and T943 [so service C is also a point of failure]) and
generate recommendations to revise code of the application based on the one or more points of failure (e.g., Theimer, col. 5 ll. 8-27: records generated may be examined to detect the impact of different faults on the application. Using such information provided by the FIS, the client may introduce changes [revisions] into App1 [which is code]. The FIS may also be able to provide recommendations on the kinds of changes that a client may need to make to make an application more robust).
Theimer does not explicitly disclose to receive an application model of an application, the application comprising a plurality of microservices that make up the application, wherein the application model represents a plurality of dependencies between or among the plurality of microservices; input the application model into a machine learning algorithm to obtain a chaos testing scenario, or wherein the machine learning algorithm is trained using interactions between historical chaos tests within historical chaos testing data.
However, in an analogous art Kammhoff discloses:
to receive an application model of an application, the application comprising a plurality of microservices that make up the application, wherein the application model represents a plurality of dependencies between or among the plurality of microservices; (e.g., Kamhoff, p. 17 last par.: the following, more simplified model of generated architecture information; p. 18 par. 1: the information provided by the simplified model is the services of the microservice architecture, the operations a service has, dependencies between operations)
input the application model into a machine learning algorithm to obtain a chaos testing scenario, (e.g., Kammhoff, p. 64 Sec. 4.4.5 pars. 2-3: the core component of this algorithm is an artificial neural network multi-layer perceptron classifier. The input to the classifier are features about fault injections that were previously extracted from the architectural model. Features considered are the dependencies of the operation and the instances of the service that the operation belongs to. Using these features, as input the classifier returns a percentage value. The resulting probabilities are used for selecting a fault injection [chaos testing scenario]; p. 3 par. 2: fault injections that are to be used in chaos experiments) and wherein the machine learning algorithm is trained using interactions between historical chaos tests within historical chaos testing data; (e.g., Kammhoff, p. 64 last two pars.: training is required for this approach. Training happens by sampling a batch of memories [historical data]. Memories are experiences: fault injections and their corresponding reward; p. 3 par. 2: fault injections that are to be used in chaos experiments; p. 11 par. 2: the injector injects the fault into the target system [i.e., interacts with it]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the chaos testing scenarios of Theimer such that they are obtained by receiving an application model of an application, the application comprising a plurality of microservices that make up the application, the application model representing a plurality of dependencies between or among the plurality of microservices and inputting the application model into a machine learning algorithm trained using interactions between historical chaos tests within historical chaos testing data, as taught by Kammhoff, as Kammhoff would provide the advantage of a means of selecting fault injection scenarios based on analysis of a target microservice application using a machine learning model, as suggested by Theimer. (See Theimer, col. 11 ll. 15-20, 28-31).
As to claim 2, Theimer discloses a method comprising:
receiving, by a processor, an instruction (e.g., Theimer, col. 28 ll. 1-2: system memory 9020 may be configured to store instructions and data accessible by processor(s) 9010; col. 28 ll. 21-26: instructions implementing functions, such as those described above are stored within system memory 9020) for testing a type of failure associated with a particular application (see below);
a testing scenario comprising a plurality of tests for the particular application and timing information for applying the plurality of tests, (e.g., Theimer, col. 4 ll. 4-5: to test application resiliency to faults; col. 2 ll. 62-63: resources of one or more computing services [microservices] may be used to execute the logic of the application; col. 8 ll. 30-33: a client may submit requests to FIS 150 to introduce combinations of one or more faults into the environments within which the applications are being run; col. 8 ll. 49-58: a client may specify the fault injection mode “(e.g., whether randomly selected faults are to be injected, whether a deterministic sequence of faults selected by the client is to be used, etc.)” the time period for which faults are to be introduced)
executing, by the processor, the testing scenario by applying the plurality of tests to the particular application according to the timing information; (e.g., Theimer, col. 23 ll. 43-45: a client may submit an InitiateFaultInjection request 814 to cause the FIS to begin introducing faults at a target environment; col. 8 ll. 49-58: a client may specify the time period for which faults are to be introduced; col. 6 ll. 28-30: a target environment associated with an application of the client which utilizes one or more services)
identifying, by the processor, based on the testing scenario applied to the particular application, one or more points of failure in one or more microservices of the plurality of microservices; (e.g., Theimer, col. 10 ll. 56-65: resilience results generators 158 may provide results of the requested fault injections. The results may be a list of faults that were injected. In other embodiments the FIS may capture data indicating the impact of the fault injections, e.g., whether one or more processes implementing the application failed [i.e.. those processes are points of failure]; col. 5 ll. 10-15: operation records used by the application may be examined to detect the impact; col. 13 ll. 37-40: service-C operation records 312 include indications of two occurrences of errors/failures E77 at times corresponding to timestamp T753 and T943 [so service C is also point of failure]) and
generating, by the processor, recommendations to revise code of the particular application based on the one or more points of failure (e.g., Theimer, col. 5 ll. 8-27: records generated may be examined to detect the impact of different faults on the application. Using such information provided by the FIS, the client may introduce changes [revisions] into App1 [which is code]. The FIS may also be able to provide recommendations on the kinds of changes that a client may need to make to make an application more robust).
Theimer does not explicitly disclose accessing, by the processor, a particular application model of the particular application, the particular application comprising a plurality of microservices that make up the particular application, wherein the particular application model represents a plurality of dependencies between or among the plurality of microservices; inputting, by the processor, the particular application model into a machine learning algorithm to generate a testing scenario.
However, in an analogous art, Kammhoff discloses:
accessing, a particular application model of the particular application, the particular application comprising a plurality of microservices that make up the particular application, wherein the particular application model represents a plurality of dependencies between or among the plurality of microservices; (e.g., Kammhoff, p. 17 last par.: the following, more simplified model of generated architecture information; p. 18 par. 1: the information provided by the simplified model is the services of the microservice architecture, the operations a service has, dependencies between operations)
inputting, the particular application model into a machine learning algorithm to generate a testing scenario, (e.g., Kammhoff, p. 64 Sec. 4.4.5 pars. 2-3: the core component of this algorithm is an artificial neural network multi-layer perceptron classifier. The input to the classifier are features about fault injections that were previously extracted from the architectural model. Features considered are the dependencies of the operation and the instances of the service that the operation belongs to. Using these features, as input the classifier returns a percentage value. The resulting probabilities are used for selecting a fault injection [chaos testing scenario]; p. 67 ll. 2-3: the artificial neural network algorithm; p. 65 ll. 4-10: the following will describe the functions implemented by this algorithm. Generate all fault injections from the provided architectural model; p. 3 par. 2: fault injections that are to be used in chaos experiments) wherein the machine learning algorithm is trained to predict, based on interactions between historic tests, testing scenarios that cause types of failures of applications; (e.g., Kammhoff, p. 64 last two pars.: training is required for this approach. Training happens by sampling a batch of memories [historical data]. Memories are experiences: fault injections and their corresponding reward; p. 3 par. 2: fault injections that are to be used in chaos experiments; p. 11 par. 2: the injector injects the fault into the target system [i.e., interacts with it]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the chaos testing scenarios and processor of Theimer such that the scenarios are generated by a processor accessing an application model of an application, the application comprising a plurality of microservices that make up the application, the application model representing a plurality of dependencies between or among the plurality of microservices and inputting the application model into a machine learning algorithm trained to predict testing scenarios that cause types of failures of applications using interactions between historical chaos tests within historical chaos testing data, as taught by Kammhoff, as Kammhoff would provide the advantage of a means of selecting fault injection scenarios based on analysis of a target microservice application using a machine learning model, as suggested by Theimer. (See Theimer, col. 11 ll. 15-20, 28-31).
As to claim 3, Theimer/Kammhoff discloses the method of claim 2 (see rejection of claim 2 above), Theimer further discloses:
wherein the testing scenario is predicted (e.g., Theimer, col. 1 ll. 40-45: anticipating all the different events and event sequences that could potentially lead to application errors or sub-par behavior becomes extremely difficult for clients; col. 11 ll. 30-39: the FIS may in some embodiments identify as recommended faults, a set of faults that (1) have occurred at a rate above a threshold in the past (b) have led to application misbehaviors in the past; col. 20 ll. 50-57: for some fault categories in the catalog a “severity” attribute may be provided. Such a severity attribute may be represented as a number, with 1 representing a low probability of causing a failure at an application and 100 representing a high probability) to cause a cascading failure of the particular application (e.g., Thiemer, col. 5 ll. 59-60: the catalog may include descriptors covering a wide variety of faults; col. 4 ll. 1-4: individual descriptors indicating respective sequences of events; col. 24 l. 54: events that represent faults; col. 14 ll. 46-57: faults may represent failures 418; Fig. 3 and associated text, col. 13 ll. 48-63: event E4, followed by E33, followed by error/failure E77 may be detected by analytics tools. The tools may detect the correlation E77 between the combination of E4 and E33 as an error-causing multi-service fault; col. 4 ll. 23-27: causal chains of events may be identified and included in the catalog).
Theimer does not explicitly disclose wherein the testing scenario is predicted, based on training of the machine learning algorithm to cause the failure.
However, in an analogous art, Kammhoff discloses:
wherein the testing scenario is predicted, based on training of the machine learning algorithm, to cause a failure of the particular application (e.g., Kammhoff, p. 33 par. 3: an artificial neural network function to calculate probabilities for a test case to show a fault; p. 64 Sec. 4.4.5 second to last par.: training is required for this approach).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the testing scenarios predicted to cause a cascading failure of the application such that the prediction is made based on training a machine learning algorithm, as taught by Kammhoff, as Kammhoff, would provide the advantage of a means of providing consistently good, high efficiency good results. (See Kammhoff, p. 103 sec. 6.3.2).
As to claim 6, Theimer/Kammhoff discloses the method of claim 2 (see rejection of claim 2 above), Theimer further discloses the processor (see rejection of claim 2 above) but Theimer does not explicitly disclose further comprising prioritizing, by the processor, the testing scenario over other testing scenarios based on one or more of code complexity, criticality of components, and historical failure data.
However, in an analogous art, Kammhoff discloses:
further comprising prioritizing the testing scenario over other testing scenarios based on one or more of code complexity, criticality of components, and historical failure data (e.g., Kammhoff, p. 71 Sec. 4.5 using fault injection results to change the priority of future fault injections [chaos testing simulation scenarios]; p. 72 ll. 1-9: if a fault injection reveals a fault, it moves up to the next highest priority queue. If a fault inject doesn’t reveal a fault, it moves to the next lower priority queue).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the processor of Theimer to include prioritizing the chaos testing simulation scenarios based on one or more of code complexity, criticality of components, and historical failure data, as taught by Kammhoff, as Kammhoff would provide the advantages of a means of revealing faults as early as possible and a means of utilizing a scenario that returns the best possible results. (See Kammhoff, p. 13 Sec. 2.5, p. 14 Sec. 2.5.2 par. 1).
As to claim 8, Theimer/Kammhoff discloses the method of claim 2 (see rejection of claim 2 above), but Theimer does not wherein the machine learning algorithm comprises a large language model.
However, in an analogous art, Kammhoff discloses:
wherein the machine learning algorithm comprises a large language model (e.g., Kammhoff, p. 27 Sec. 2.7.4 artificial neural networks [large language models per par. [0138] of the specification] are used to solve a variety of problems).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the machine learning processes of Theimer to include machine learning processes such as a large language model, as taught by Kammhoff, as Kammhoff would provide the advantages of massive parallelism, generalization ability, adaptivity and fault tolerance. (See Kammhoff, p. 27 Sec. 2.7.4)
As to claim 11, it is a computer-readable media claim having limitations substantially the same as those of claim 2. Accordingly it is rejected for substantially the same reasons. Further limitations, disclosed by Theimer, include:
one or more non-transitory, computer-readable media storing instructions that, when executed by one or more processors, cause operations (e.g., Theimer, Fig. 11 element 9020 and associated text) comprising: (see rejection of claim 2 above).
As to claim 12, it is a computer-readable media claim having limitations substantially the same as those of claim 3. Accordingly, it is rejected for substantially the same reasons.
As to claim 15, it is a computer-readable media claim having limitations substantially the same as those of claim 6. Accordingly, it is rejected for substantially the same reasons.
As to claim 17, it is a computer-readable media claim having limitations substantially the same as those of claim 8. Accordingly, it is rejected for substantially the same reasons.
Claim 4-5, 9, 13-14 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Theimer (US 10,986,013) in view of Kammhoff, (“Algorithms for Efficient Chaos Experiment Selection in Microservice Architectures”) in further view of Rohithakshappa et al. (US 11,537,502) (hereinafter Rohithakshappa).
As to claim 4, Theimer/Kammhoff discloses the method of claim 2 (see rejection of claim 2 above) but does not explicitly disclose wherein the machine learning algorithm is further trained to identify patterns within the code of the particular application that are indicative of instability or failures.
However, in an analogous art, Rohithakshappa discloses:
wherein the machine learning algorithm is further trained to identify patterns within the code of the particular application that are indicative of instability or failures (e.g., Rohithakshappa, abstract: the system activates a machine learning (ML) anomaly detection plug-in for analyzing the program code; col. 13 ll. 42-4: the system may determine, via the anomaly detection engine component, a first pattern; col. 14 ll. 3-8: the system may determine, via the anomaly detection component, one or more anomalies associated with the program code in response to detecting that the first pattern is an anti-pattern. The anti-pattern is associated with an architectural defect or flow in the program code; col. 14 ll. 13-16: the system may detect the first pattern is the anti-pattern in response to determining the validation of the sequence order is unsuccessful “(e.g., that the current sequence would fail if executed)”; col. 1 ll. 30-33: flaws in the code which may render the final code unusable).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the machine learning algorithm of Theimer/Kammhoff to include identifying patterns within the code of the particular application that are indicative of instability or failures, as taught by Rohithakshappa, as Rohithakshappa would provide the advantage of a means of avoiding malfuncitiong of the program. (See Rohithakshappa, col. 1 ll. 22-26).
As to claim 5, Theimer/Kammhoff/Rohithakshappa discloses the method of claim 4 (see rejection of claim 4 above), Theimer further discloses:
wherein the testing scenario further comprises one or more instructions to inject faults, introduce latency, or simulate resource constraints in areas of the code in which the patterns are identified (e.g., Theimer, col. 11 ll. 15-20: fault recommendation engines 159 for suggesting fault injection modes [chaos simulation scenarios]; col. 22 ll. 35-40: a client may submit a request 806 to obtain such recommendations. The corresponding response message 808 may include descriptors of one or more categories of faults; col. 4 ll. 1-4: a catalog of fault category descriptors with individual descriptors describing sequences of one or more events at one or more resources)
As to claim 9, Theimer/Kammhoff discloses the method of claim 8 (see rejection of claim 8 above) but does not explicitly disclose wherein the large language model analyzes documentation and code comments of the particular application to extract data related to architecture and the plurality of dependencies of the particular application.
However, in an analogous art, Rohithakshappa discloses:
wherein the large language model analyzes documentation and code comments of the particular application to extract data related to architecture and the plurality of dependencies of the particular application (e.g., Rohithakshappa, col. 16 ll. 34-48: the ML anomaly detection model 156b analyzes method codes “(e.g., comprising JavaDoc comments associated with the respective execution method)” [documentation and code comments]. Here, the ML anomaly detection model 156b is further trained to read class files to identify method attributes of each of the training execution methods. Specifically, the ML anomaly detection model 156b is trained to detect method attributes such as variable types, operations used in the execution method [data related to architecture], use of any libraries [dependencies] or application programming interfaces, annotations within the execution method, and/or the like; col. 17 ll. 20-25: the ML anomaly detection model 156b is a deep learning model program).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the large language model of Theimer/Kammhoff such that the large language model analyzes documentation and code comments of the application to extract data related to the architecture, dependencies, and critical components of the application as taught by Rohithakshappa, as Rohithakshappa would provide the advantage of a means of acquiring data related to the architecture and dependencies specified in the application code. (See Rohithakshappa, col. 16 ll. 34-48).
As to claim 13, it is a computer-readable media claim having limitations substantially the same as those of claim 4. Accordingly, it is rejected for substantially the same reasons.
As to claim 14, it is a computer-readable media claim having limitations substantially the same as those of claim 5. Accordingly, it is rejected for substantially the same reasons.
As to claim 18, it is a computer-readable media claim having limitations substantially the same as those of claim 9. Accordingly, it is rejected for substantially the same reasons.
Claims 7 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Theimer (US 10,986,013) in view of Kammhoff, (“Algorithms for Efficient Chaos Experiment Selection in Microservice Architectures”) in further view of Arunachalam et al. (US 10,795,793) (hereinafter Arunachalam) and Rossi et al. (US 2009/0254887) (hereinafter Rossi).
As to claim 7, Theimer/Kammhoff discloses the system of claim 5 (see rejection of claim 6 above), but does not explicitly disclose monitoring, by the processor, a status of the application during execution of the testing scenario; comparing, by the processor, the status to an operational status that occurs when testing scenarios are not executing; and revising, by the processor, based on comparing the status to the operational status, the testing scenario.
However, in an analogous art, Arunachalam discloses further comprising application code instructions cause the one or more hardware processors to:
monitoring, by the processor, a status of the application during execution of the testing scenario; (e.g., Arunachalam, col. 11 ll. 57-60: the system monitors the distributed computing system to record an outcome of the simulated system failure)
comparing, by the processor, the status to an operational status that occurs when testing scenarios are not executing; (e.g., Arunachalam, col. 12 ll. 1-3: at block 250 , the system determines whether the monitored outcome matches the expected outcome of the simulated system failure; col. 6 ll. 31-38: the expected output of the simulated failures scenario may include an expected state to verify that the system has failed according to the failures and an expected state after recover operations have been invoked [operational status that occurs when testing scenarios are not executing]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the execution of chaos scenarios used and chaos testing of an application of Theimer/Kammhoff/Roberts, by monitoring the status of the application during the testing and comparing the status to an operational status that occurs when chaos conditions are not present and an expected operational status of the application when under chaos conditions, as taught by Arunachalam, as Arunachalam would provide the advantage of a means of verifying when the application both fails as expected and recovers as expected in response to the chaos testing simulation scenarios. (See Arunachalam, col. 6 ll. 31-38).
Further, in an analogous art, Rossi discloses to:
revising, by the processor, based on comparing the status to the operational status, the testing scenario (e.g., Rossi, par. [0017]: the test scenario may pass when the actual results match the corresponding expected results; par. [0020]: the curve 215 plots the cumulative weight of the executed test scenarios that actually passed; par. [0021]: the expected trend curve 210 and the actual trend curve 215 are compared to track the progress of the test process; par. [0053]: the variance indicator represents a different between the actual trend and the expected trend; par. [0054]: when the variance indicator reaches the threshold, the test process may be deemed not proceeding as planned. Suitable recovery actions may be taken. It may be possible to simplify the execution of the remaining test scenarios [revise the one or more scenarios]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the execution of chaos testing simulation scenarios, monitoring and comparison of Theimer/Kammhoff/Roberts/Arunachalam noted above by revising the one or more scenarios based on the comparison, as taught by Rossi, as Rossi would provide the advantage of a means of ensuring the test process proceeds as planned. (See Rossi, par. [0030]).
As to claim 16, it is a computer-readable media claim having limitations substantially the same as those of claim 7. Accordingly, it is rejected for substantially the same reasons.
Claims 10 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Theimer (US 10,986,013) in view of Kammhoff, (“Algorithms for Efficient Chaos Experiment Selection in Microservice Architectures”) in further view of Yanamala et al. (US 2022/0358031) (hereinafter Yanamala)
As to claim 10, Theimer/Kammhoff discloses the method of claim 8 (see rejection of claim 8 above) but does not explicitly disclose wherein the large language model reviews one or more changes to the code and assesses an impact of the one or more changes on testing practices.
However, in an analogous art, Yanamala discloses:
wherein the large language model reviews one or more changes to the code and assesses an impact of the one or more changes on testing practices (e.g., Yanamala, par. [0120]: the AI module 728 may predict modules that may be impacted based on code block addition, modification based on one or more machine learning algorithms “(e.g., using artificial neural network, K-means algorithm, association rule learning, etc.)” [a neural network being a large language model per paragraph [0138] of the specification]; par. [0121]: the AI module 728 may determine modified test scripts of impacted modules and/or test scripts that need to be added for the impacted modules; Fig. 9 and associated text, par. [0136]: FIG. 9 illustrates an artificial neural network; par. [0143]: the output from an output node [of the neural network] may be in indication of modules impacted and/or new/revised test scripts).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the large language model of Theimer/Kammhoff such that the large language model reviews one or more changes to the code and assesses an impact of the one or more changes on testing practices, as taught by Yanamala, as Yanamala would provide the advantage of a means of ensuring portions of the code do not remain untested when code is changed. (See Yanamala, par. [0032]).
As to claim 19, it is a computer-readable media claim reciting a portion of the limitation of claim 10. The limitations of claim 19 are thus taught by or obvious over the prior art for reasons substantially the same as those set forth with respect to claim 10.
As to claim 20, it is a computer-readable media claim reciting a portion of the limitation of claim 10. The limitations of claim 20 are thus taught by or obvious over the prior art for reasons substantially the same as those set forth with respect to claim 10.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TODD AGUILERA whose telephone number is (571)270-5186. The examiner can normally be reached M-F 11AM - 7:30PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hyung S Sough can be reached at (571)272-6799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TODD AGUILERA/Primary Examiner, Art Unit 2192
