Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
The application claims priority to a Provisional 63/364,157, filed on May 4, 2022.
Response to Arguments
Applicant's arguments filed February 27, 2026 have been fully considered but they are not persuasive.
In page 1 of the remarks, Applicant states that claims 8-14 have been canceled to render the objections moot, as well as any rejections applied to claims 8-14. The term “actual” has also been removed in the amendments.
Applicant states that claims 1-3, 6-10, 13-17, and 20 were previously rejected under 35 U.S.C. § 112(a) (mistakenly described as “35 U.S.C. § 112(b)”) as not satisfying the enablement requirement, when the Office Action rejected the claims under 112(a) for lack of written description. Applicant states that the Office Action (“OA”) on November 28, 2025 does not consider what a person of ordinary skill in the art would understand at the time that the application was filed, and that the Specification is sufficient, which is “all that is required by the Federal Circuit and the MPEP” Allergan, Inc. v. Sandoz Inc., 796 F.3d 1293, 1310, 115 USPQ2d 2012, 2023 (Fed. Cir. 2015) ("Only a sufficient description enabling a person of ordinary skill in the art to carry out an invention is needed."); MPEP 2164. Applicant refutes that a specific process or algorithm is required, as stated in the previous OA at page 12 (“there is no process as to how this is performed, or an algorithm as to how this step is performed.”). Furthermore, in page 2 of the remarks, Applicant states that claims 1 is directed to a concrete, stepwise software testing pipeline, implemented by an orchestrator/bot within the automation pipeline, with various types of tests, as described in paragraphs [0040]-[0046], [0049]-[0057], and [0060]-[0061]. Applicant also states that the state of the art reflects that CI/CD pipelines with automated test staging, orchestration, logging and analysis were well-known at the time of filing, as stated in Eizenman (US 20190171550 A1), in various sections of the reference described in pages 2-3 of the remarks. Furthermore, in page 3 of the remarks, Applicant states that in page 3 of the remarks, that under the Wands factors (MPEP 2164.01(a)), implementing claim 1 requires, at most, routine engineering choices, and not undue experimentation. In pages 3-7 of the remarks, Applicant addresses each of the claim limitations of claim 1 that were previously rejected under 112(a), and provides sections of the Specification as to where the limitations are supported in the Specification.
Examiner disagrees with the Applicant, and states that although the Applicant utilizes the Specification of their own invention and utilizes the Specification of the prior art to assert that the technology that is also recited in the invention was “well-known” at the time of filing, the Applicant merely repeats the sections of the Specification where the claim limitation was originally cited, and at times, provides a general overview of how the limitation is performed in the context of the invention. Examples include “testing… with unit tests… test one or more functions of the code”, also describing paragraph [0052] of the Applicant’s Specification as testing the JavaScript function of the code, but does not specify how the JavaScript code is tested in further detail. Furthermore, the “testing… with contract tests… compatibility and communication”, although amended to now include the compatibility and communication between two code components, remains insufficiently described regarding a “shared agreement” between two code components using either PACT, Spring Cloud Contact, or other equivalent software in paragraph [0054] of the Specification. As a whole, despite the limitations reciting routine computer tasks that need not be exhaustively enumerated to avoid undue experimentation, as described in MPEP 2164.01, the limitations remain rejected for lack of written description for not going into further detail as to how the Applicant intend for these limitations be performed in the invention, and appear to merely recite the desire result language in the claim limitations, and provide a scenario in an attempt to fulfill the written description requirement. As a result of insufficient explanations as to how the claim limitations of claim 1 are supported by written description, the limitations of claim 1 remain rejected under 112(a) for lack of written description. It is not enough that one skilled in the art could write a program to achieve the claimed function because the specification must explain how the inventor intends to achieve the claimed function to satisfy the written description requirement. See, e.g., Vasudevan Software, Inc. v. MicroStrategy, Inc., 782 F.3d 671, 681-683, 114 USPQ2d 1349, 1356, 1357 (Fed. Cir. 2015).
In pages 7-8, Applicant states that claims 1-3, 6-10, 13-17, and 20 stand rejected under 35 U.S.C. § 103 as allegedly rendered obvious by U.S. Patent Application Publication No. 2019/0171550 to Eizenman et al. ("Eizenman", D1) and U.S. Patent Application Publication No. 2022/0303302 to Hwang et al. ("Hwang", D2). Claims 4-5, 11-12, and 18-19 stand rejected under 35 U.S.C. § 103 as allegedly being obvious over Eizenman in view of Hwang in further view of U.S. Patent No. 10,210,074 to Szerenyi ("Szerenyi", D3). Applicant states that the claims have been amended to include component tests "executed against mocked data," (ii) contract tests that test compatibility/communication "according to a shared agreement between the two code components," and (iii) a concrete failure-classification operation in which a bot determines failure patterns based on "test execution meta data," and isolates failures due to a test framework or environmental instability from failures due to application code issues by "correlating" (a) test-event data, (b) environment identifiers including browser version and operating system, and (c) build modification information identifying modified methods and components of the code (collectively, the "correlation-based failure classification limitations"). Despite certain limitations not being present in the amendments present in the claim limitations as of February 27, 2026, the amendments narrow claim 1 to a specific technical failure-triage mechanism tied to particular data sources and correlations, not merely generic “analysis” of tests.
Applicant further argues, in pages 9-10 of the remarks, with regards to the limitation of “failure patterns/actual application code issues”, that Eizenman does not disclose the amended “correlation-based failure classification limitaitons, and that nothing in Eizenman suggests “isolating framework/environment instability failures from application code failures, nor doing so by correlating the particular triad of (a) test-event data, (b) environment identifiers (browser version/OS), and (c) build modification information identifying modified methods/components”, as required by claim 1, and that Eizenman discloses collection of test events, tracking failed tests, and identifying coverage “quality holes”, different from the correlation-based root-cause classification required by claim 1. Applicant also states that Hwang does not resolve these issues, as it is directed to “security risk analysis” using operational data and machine learning, including building DevOps pipelines and "localizing security issues" using logs, and relates to identifying and localizing security issues using operational/production datasets, not triaging automated test failures into framework/environment instability versus application code faults using the specific correlations now recited. Applicant also states that Szerenyi is directed to scaling browser-based testing using multiple virtual machines and automation scripts, but that Szerenyi does not suggest classifying failures by correlating test-event data with environment identifiers and build modification information to separate framework/environment instability failures from application code failures, and that parallel browser/OS execution is not the same as the specific correlation-based failure triage mechanism now claimed.
Examiner disagrees with the arguments present in amended claim 1, as the reference of Eizenman remains applicable over the limitation of “determining… failure patterns and failures due to application code issues”, as Eizenman states that in paragraph [0121] describes tests run in Figs. 5 and 7, for test scoring post-execution and test quality coverage calculations, respectively, the analysis engine collects tests from queues, including TestEvents queue 1008 in Fig. 10A, which contains information about tests that have been performed and changes to code. Also, paragraph [0096] states that Fig. 10A receives information regarding quality holes, which is described in paragraph [0096] as holes in testing that prevent code from being tested or covered for the entirety of run time or deployed environment, and code fails when deployed, corresponding to failures due to application code issues, with the test coverage holes corresponding to application code issues, and that paragraph [0015] describes that “test coverage holes” as a determination of at least a portion of the code that has been modified, has not adequately been tested by tests that have been run, which is further described as code that can fail when deployed to a particular environment or run time situation due to lack of previous testing. Hwang also states that, in paragraph [0044] describes step 208 of Fig. 2, with a semantic graph being constructed, and the graph can show changes over time in the source code as well, which corresponds to modification information that identifies modified methods and components of the code that lead to the failures, and in paragraph [0042] Security risk program 110a/110b captures information relating to the security issues, and will identify repeating issues or patterns, the problem is localized with application and system logs, corresponding to the isolating of failures due to environmental instability from failures due to code issues, effectively separating failures of instability and code issues as described in the invention of the Applicant. Finally, with regards to the reference of Szerenyi, it suggests the classification of failures by correlating test-event data with environment identifiers and build modification information in [Col. 7, lines 39-48] Each of the plurality of browsers run by virtual machines may be a same type of browser, or different type of browser, where the types of browsers are equivalent to browser versions identified to suggest environmental identifiers including a browser version, with the failure by correlating test-event data and environment identifiers including operating system described in Hwang’s paragraph [0050] security patterns can be fed into a classifier and the model output includes classes and types including vulnerability score and impact on operating systems. It would be obvious to combine the browser version operational data of Szerenyi with the limitations present in Hwang to increase efficiency by having different sets of browsers be performed across different VMs, and that the testing platforms can be done on the different sets and types of browsers, as taught by Szerenyi [Col. 8, lines 16-47].
Finally, in page 11 of the remarks, it is stated that the newly added limitations regarding component tests executed against mocked data and contract tests performed according to a "shared agreement" between two code components impose further distinctions over the cited art. Applicant states that Eizenman broadly lists types of "tests" and environments (e.g., unit, component, integration, but that Eizenman does not describe the claimed, mocked-data execution constraint for component tests, nor does it disclose contract tests as being performed according to a shared agreement between communicating components (as opposed to generic dependency/coverage selection).
Applicant’s arguments with respect to claim 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The rejections against component tests being executed against mocked data remains, and the mocked data does not appear to be entered into the amendments of February 27, 2026. As for the “shared agreement” between two code components, although none of the prior art teaches or suggests the shared agreement portion of the claimed limitation, it is stated that Gupta (US 11151025 B1), in section [Col. 13, lines 20-26] Structure variations are used to perform the contract tests, where the testing planner module 411 in Fig. 4 enables functionality for contract testing by deploying test case algorithm that looks for variations in the response structures and identifies consumers 427 impacted by performing such a scan, with the variations in structure in the code of two different consumers are tested using test case algorithms, and as a result, the test cases between code portions of two different consumers corresponds to a shared agreement between the two code components. This would be obvious to add the reference of Gupta to Eizenman to test for different response types based on the code components, as the test cases are used based upon the different types of response status codes received by end users to test for differences in code components based upon the users’ systems to determine variations in how the code is structured, as described in [Col. 14, lines 3-13] of Gupta.
Claim Rejections - 35 USC § 112(a)
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-3, 6-7, 15-17, and 20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claims contain subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventors, at the time the application was filed, had possession of the claimed invention.
In claim 1, the limitation of ‘receiving, by an orchestrator computer program in an automation pipeline, code for shift-left testing, wherein the code comprises a plurality of code components’ has no support as to how the reception of code is performed in the specification. In pages 4-5, paragraph [0019], line 4-5, the applicant recites the step of receiving code from a repository, but there is no process as to how this is performed, or an algorithm as to how this step is performed. This makes it unclear to one of ordinary skill in the art of how to make and/or use the invention of the applicant.
The limitation of ‘testing, by the orchestrator computer program, the code with unit tests, wherein the unit tests comprise narrowly-scoped tests that exercise units of the code to verify proper behavior’ has no support as to how this result is achieved in the specification. In page 9, paragraph [0041], line 3 of the applicant, a unit test is described as testing the smallest piece of code, such as a function, which a unit test is performed to verify proper behavior, and gives no explanation as to what proper behavior is in a unit test, which makes it unclear to one of ordinary skill in the art of how to make and/or use the invention of the applicant.
Similar issues also exist for the limitation of ‘testing, by the orchestrator computer program, the plurality of components of the code with component tests, wherein the component tests exercise the plurality of components of the code in isolation from each other’, as there is also little support for this in the specification of the applicant. In page 10, paragraph [0042], lines 2-5 of the applicant, a component test is described as functional tests tested in isolation, and using mock data, without any explanation as to how or what the purpose is behind picking the components, and there is no algorithm to this process. This makes it unclear to one of ordinary skill in the art of how to make and/or use the invention of the applicant.
Similar issues also exist for the limitation of ‘testing, by the orchestrator computer program, the code with contract tests, wherein the contract tests two code components for compatibility and communication’, as there is also little support for this in the specification of the applicant. In page 10, paragraph [0043], lines 2-8 of the applicant, an integration test is performed between two separate components that are compatible and able to communicate with one another, with no indication on how this is determined, or how the components are chosen to be tested. This makes it unclear to one of ordinary skill in the art of how to make and/or use the invention of the applicant.
Similar issues also exist for the limitation of ‘testing, by the orchestrator computer program, the code with integration tests, wherein the integration tests integration of the plurality of components with each other’, as there is also little support for this in the specification of the applicant. In page 10, paragraph [0044], lines 2-5 of the applicant, the integration tests are described as having components of user interface being tested with services, which is not explained in the claims, and a process for how this is performed is missing from the specification. This makes it unclear to one of ordinary skill in the art of how to make and/or use the invention of the applicant.
Similar issues also exist for the limitation of ‘deploying, by the orchestrator computer program, the code to an end-to-end testing environment and testing the code with end-to-end automation tests, wherein the end-to-end automation tests exercise the code from start to finish’, as there is also little support for this in the specification of the applicant. In page 10-11, paragraph [0045], lines 2-9 of the applicant, an end-to-end automation test goes through the code of the applicant to test it against various services, with no explanation as to how the applicant tests the code with various cases, and an algorithm is not present in the applicant. This makes it unclear to one of ordinary skill in the art of how to make and/or use the invention of the applicant.
Similar issues also exist for the limitation of ‘determining, by a bot of the orchestrator computer program, failure patterns and failures due to actual application code issues’, as there is also little support for this in the specification of the applicant. In page 11, paragraph [0048], lines 1-3 of the Applicant, it is stated that if any part of the code fails any of the tests, the code will not progress through the automation pipeline, and the developer may be notified of the failure, with no further support or algorithm being provided to support this process of the invention, as well as how to detect failures or failure patterns in code. This makes it unclear to one of ordinary skill in the art of how to make and/or use the invention of the applicant.
The limitation of “isolating, by the orchestrator computer program. the failures by correlating test-event data, environment identifiers including browser version and operating system” and “building, by the orchestrator computer program, modification information identifying modified methods and components of the code to classify the failures into (i) framework/environment instability and (ii) application code issues” are not supported in the Specification is rejected as they constitute new matter, and are currently not described in the Specification of the Applicant. Nowhere in the current Specification as filed are the limitations described to be performed in the invention. Applicant has not pointed out where the new (or amended) claim limitations are supported, nor does there appear to be a written description of the claim limitations “isolating, by the orchestrator computer program. the failures by correlating test-event data, environment identifiers including browser version and operating system” and “building, by the orchestrator computer program, modification information identifying modified methods and components of the code to classify the failures into (i) framework/environment instability and (ii) application code issues” in the application as filed.
Similar issues also exist for the limitation of ‘scanning, by the orchestrator computer program, the code for security vulnerabilities’, as there is also little support for this in the specification of the applicant. In page 11, paragraph [0046], lines 2-5 of the applicant, scanning for security issues tests code for various vulnerabilities, including flaws, glitches, and weaknesses to be exploited by an attacker, with no indication as to how these vulnerabilities are determined, and no algorithm is provided. This makes it unclear to one of ordinary skill in the art of how to make and/or use the invention of the applicant.
Similar issues also exist for the limitation of ‘deploying, by the orchestrator computer program, the code to an end-to-end testing environment and performing production validation testing on the code, wherein the production validation testing is executed on a plurality of threads, wherein the production validation testing is performed on a first browser version in a first thread, and a second browser version in a second thread, wherein the production validation testing is performed on a first operating system in a third thread, and a second operating system in a fourth thread’, as there is also little support for this in the specification of the applicant. In page 11, paragraph [0049], lines 1-2 of the applicant, code goes into a production staging environment for validation of the code, with no algorithm as to how this is performed or what the criteria for validation is supposed to be. This makes it unclear to one of ordinary skill in the art of how to make and/or use the invention of the applicant.
Similar issues also exist for the limitation of ‘deploying, by the orchestrator computer program, the code to a production environment’, as there is also little support for this in the specification of the applicant. In page 11, paragraph [0049], lines 2-4 of the applicant, code that passes the validation testing is put into a production environment, and how code passes from the validation testing phase and be deployed into a production environment is unclear, with no algorithm or indication of how this occurs. This makes it unclear to one of ordinary skill in the art of how to make and/or use the invention of the applicant.
Furthermore, the term ‘automation pipeline’ is stated in claim 1, line 2, and is mentioned throughout the claim, without any indication as to what it specifically represents, or how it is created by the applicant. Therefore, unclear how one of ordinary skill in the art is meant to make and/or use the invention of the applicant.
Dependent claims 2-3, and 6-7 are claims that depend upon their respective independent claims, and as a result of claim 1 being rejected for written description regarding the limitations outlined above, claims 2-3, and 6-7 inherit the rejections of claim 1 above.
Independent claim 15 shares limitations with that of independent claim 1 above, and as a result, is also rejected for lack of written description for similar reasons as claim 1 above.
Dependent claims 16-17, and 20 are claims that depend upon their respective independent claims, and as a result of claim 15 being rejected for written description regarding the limitations outlined above, claims 16-17, and 20 inherit the rejections of claim 15 above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 6-7, 15-17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Eizenman (US 20190171550 A1) in view of Hwang et al. (US 20220303302 A1), hereinafter Hwang, Szerenyi (US 10210074 B1), and Gupta (US 11151025 B1).
Regarding claim 1, Eizenman discloses “a method for shift-left testing, comprising: receiving, by an orchestration computer program in an automation pipeline, code for shift-left testing, wherein the code comprises a plurality of code components” ([0057] Developments in a continuous testing environment occurs, and the developments happening on the left side of testing stages corresponds to the shift-left testing. Fig. 1B shows a method of an invention that receives code to evaluate, with steps 2-4 corresponding to an automation pipeline, as it performs the various tasks to verify software builds. Paragraph [0057] describes a method as per Fig. 1B, paragraph [0050] describes a cloud application 122 as per Fig. 1A, corresponding to an orchestrator computer program.);
“testing, by the orchestration computer program, the code with unit tests, wherein the unit tests comprise exercising units of the code to test one or more functions of the code” ([0057] Fig. 1B, steps 2 and 3. [0071] Unit tests are done for each method in the build according to the results, show to be tested at least once. Paragraph [0057] describes a method as per Fig. 1B, paragraph [0050] describes a cloud application 122 as per Fig. 1A, corresponding to an orchestrator computer program.);
“testing, by the orchestration computer program, the plurality of components of the code with component tests, wherein the component tests exercise the plurality of components of the code in isolation from each other” ([0057] Fig. 1B, steps 2 and 3. [0070] Components of code are also considered for testing. [0112] Component in the prior art is a portion of code, containing code lines, and/or micro-services. A package in the code can be considered a component. Paragraph [0057] describes a method as per Fig. 1B, paragraph [0050] describes a cloud application 122 as per Fig. 1A, corresponding to an orchestrator computer program.);
“testing, by the orchestration computer program, the code with contract tests, wherein the contract tests test compatibility and communication between two code components according to a shared agreement between the two code components” ([0057] Fig. 1B, steps 2 and 3. [Claim 104] Components of the code can be tested for dependency with a particular code area or code family/component, which corresponds to testing two code components for communication and compatibility. Paragraph [0057] describes a method as per Fig. 1B, paragraph [0050] describes a cloud application 122 as per Fig. 1A, corresponding to an orchestrator computer program.);
“testing, by the orchestration computer program, the code with integration tests, wherein the integration tests integration of the plurality of components with each other” ([0057] Fig. 1B, steps 2 and 3. [0061] Integration tests are disclosed. [0027] Tests are performed for integration between components, which corresponds to the integration tests being performed to determine this aspect of the invention. Paragraph [0057] describes a method as per Fig. 1B, paragraph [0050] describes a cloud application 122 as per Fig. 1A, corresponding to an orchestrator computer program.);
“deploying, by the orchestration computer program, the code to an end-to-end testing environment and testing the code with end-to-end automation tests, wherein the end-to-end automation tests exercise the code from start to finish” ([0101] Fig. 7, stage 9, tests are done in a deploying environment, or a customer application environment, which corresponds to the end-to-end testing environment. [0105] End-to-end test is executed against a whole 'integration build' and passes through one or more components of the build. Paragraph [0057] describes a method as per Fig. 1B, paragraph [0050] describes a cloud application 122 as per Fig. 1A, corresponding to an orchestrator computer program.);
“determining, by a bot of the orchestrator computer program, failure patterns and failures due to application code issues” ([0121] Based on tests run in Figs. 5 and 7, for test scoring post-execution and test quality coverage calculations, respectively, the analysis engine collects tests from queues, including TestEvents queue 1008 in Fig. 10A, which contains information about tests that have been performed and changes to code, which go through a core calculations service 1010. Paragraph [0126] describes a failed test queue 1020 in Fig. 10A receiving calculations as to which tests were executed and their status, providing information to failed test service 1022 to perform an analysis of what the test results are for a report of failed test patterns are identified. Furthermore, in paragraph [0128], a quality holes queue 1028 in Fig. 10A receives information regarding quality holes, which is described in paragraph [0096] as holes in testing that prevent code from being tested or covered for the entirety of run time or deployed environment, and code fails when deployed, corresponding to failures due to application code issues.);
“deploying, by the orchestration computer program, the code to an end-to-end testing environment and performing production validation testing on the code, wherein the production validation testing is executed on a plurality of threads, wherein the production” ([0105] Code coverage and quality are determined via a plurality of test tools in an end-to-end environment, wherein multiple tests passing through one or more components correspond to a plurality of threads. Paragraph [0057] describes a method as per Fig. 1B, paragraph [0050] describes a cloud application 122 as per Fig. 1A, corresponding to an orchestrator computer program.);
“and deploying, by the orchestration computer program, the code to a production environment” ([0063] Fig. 1B, step 5. [0060] When build quality is determined as sufficient, the build can go into production. Paragraph [0057] describes a method as per Fig. 1B, paragraph [0050] describes a cloud application 122 as per Fig. 1A, corresponding to an orchestrator computer program.).
Eizenman does not appear to disclose, but Hwang teaches the limitation of “scanning, by the orchestration computer program, the code for security vulnerabilities” ([0021] The mapping and analysis of source codes and the types of security issues that can occur in code corresponds to scanning code for security vulnerabilities. [0038] Pipelines are built according to the applicant at step 204 in Fig. 2, including a DevOps pipeline to test computing data, including the code of the prior art of Hwang, and with [0039], a pipeline can also have the functionality of preventing and detecting of client impacting events, which can include the analyzing of source code and security issues of paragraph [0021] of the prior art of Hwang. The preventing and detecting of ‘client impacting events’, or security issues in the code in a pipeline corresponds to the applicant’s limitation of scanning the code for security issues at a pipeline. Fig. 1, security risk program 110a corresponds to the orchestration computer program.);
“isolating, by the orchestrator computer program. the failures by correlating test-event data, environment identifiers including operating system” ([0040] Fig. 2, block 206, security issues are localized based on data collected at block 202 by a security risk program and used to train a machine learning model. The security risk program is also the entity responsible for localizing the security issues based on. [0042] Security risk program 110a/110b captures information relating to the security issues, and will identify repeating issues or patterns, the problem is localized with application and system logs, corresponding to the isolating of failures due to environmental instability from failures due to code issues. [0050] Security patterns can be fed into a classifier and the model output includes classes and types including vulnerability score and impact on operating systems.);
“building, by the orchestrator computer program, modification information identifying modified methods and components of the code to classify the failures into (i) framework/environment instability and (ii) application code issues” (Paragraph [0044] describes step 208 of Fig. 2, with a semantic graph being constructed, and the graph can show changes over time in the source code as well, which corresponds to modification information that identifies modified methods and components of the code that lead to the failures. [0042] Security risk program 110a/110b captures information relating to the security issues, and will identify repeating issues or patterns, the problem is localized with application and system logs, corresponding to the isolating of failures due to environmental instability from failures due to code issues.);
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Eizenman and Hwang before them, to include Hwang’s ‘scanning, by the orchestration computer program, the code for security vulnerabilities’ in Eizenman’s method performing ‘receiving, at an automation pipeline, code for shift-left testing’. One would have been motivated to make such a combination to enhance security by allowing the invention to detect security attacks or vulnerabilities early on to decrease the risk in the operational systems, and to lower costs of mitigating the security issues, as taught by Hwang [0019].
Eizenman discloses the limitation of “deploying, […] the code to an end-to-end testing environment […] wherein the production validation testing is executed on a plurality of threads”. Eizenman does not appear to fully disclose, but Szerenyi teaches the limitation of “wherein the production validation testing is performed on a first browser version in a first thread, and a second browser version in a second thread, wherein the production validation testing is performed on a first operating system in a third thread, and a second operating system in a fourth thread” ([Col. 7, lines 39-48] One of the parameters used for assigning scripts for testing include whether different types of browsers are used to test the code across VMs. [Col. 16, lines 3-10] It is emphasized that multiple instances of a variety of operating systems can exist through a hypervisor and the various VMs present containing the OSes.);
And “environment identifiers including browser version” ([Col. 7, lines 39-48] Each of the plurality of browsers run by virtual machines may be a same type of browser, or different type of browser, where the types of browsers are equivalent to browser versions identified used in the virtual machine, along with an operating system of the computing resources of the VM.);
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Eizenman, Hwang, and Szerenyi before them, to include Szerenyi’s “wherein the production validation testing is performed on a first browser version in a first thread, and a second browser version in a second thread, wherein the production validation testing is performed on a first operating system in a third thread, and a second operating system in a fourth thread” and “environment identifiers including browser version” in Eizenman’s method performing “deploying, […] the code to an end-to-end testing environment […] wherein the production validation testing is executed on a plurality of threads”. One would have been motivated to make such a combination to increase efficiency by having different sets of browsers be performed across different VMs, and that the testing platforms can be done on the different sets and types of browsers, as taught by Szerenyi [Col. 8, lines 16-47], and being able to test resources on multiple operating systems to conserve resources, while allowing different tests to occur on varying hardware, as taught by Szerenyi [Col. 22, lines 46-49].
Eizenman in view of Hwang, and Szerenyi does not appear to teach, but Gupta teaches the limitation of “contract tests according to a shared agreement between the two code components” ([Col. 13, lines 20-26] Structure variations are used to perform the contract tests, where the testing planner module 411 in Fig. 4 enables functionality for contract testing by deploying test case algorithm that looks for variations in the response structures and identifies consumers 427 impacted by performing such a scan. The variations in structure in the code of two different consumers are tested using test case algorithms, and as a result, the test cases between code portions of two different consumers corresponds to a shared agreement between the two code components.);
Therefore, one of ordinary skill in the art would have been capable of applying this known method of "contract tests according to a shared agreement between the two code components" in an orchestration computer program for scanning the code for security vulnerabilities and the results would have been predictable to one of ordinary skill in the art. The one of ordinary skill in the art would have been motivated to test for different response types based on the code components, as the test cases are used based upon the different types of response status codes received by end users to test for differences in code components based upon the users’ systems to determine variations in how the code is structured (Gupta, [Col. 14, lines 3-13]).
Regarding claim 2, Eizenman in view of Hwang, Szerenyi, and Gupta teaches the method of claim 1 as recited above. Eizenman also discloses ‘wherein the component tests are limited to a system under test’ ([0057] Fig. 1B performs tests with a system shown in Fig. 1A, and in conjunction with paragraph [0061], the tests, including the component tests, are performed and limited to a system that is being tested, corresponding to the claim.).
Regarding claim 3, Eizenman in view of Hwang, Szerenyi, and Gupta teaches the method of claim 1 as recited above. Eizenman does not appear to disclose, but Hwang teaches the limitation of ‘wherein the security vulnerabilities comprise encryption vulnerabilities and third party software development kit violations’ ([0031] When code using old encryption methods is detected, it could raise a security violation in a cloud environment.).
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Eizenman, Hwang, Szerenyi, and Gupta before them, to include Hwang’s ‘wherein the security vulnerabilities comprise encryption vulnerabilities and third party software development kit violations’ in Eizenman’s method performing ‘receiving, at an automation pipeline, code for shift-left testing’. One would have been motivated to make such a combination to enhance security by having the code be broken down into segments, and such that a security risk program 110a/110b will be capable of identifying security risks, as taught by Hwang [0031] and [0035].
Regarding claim 6, Eizenman in view of Hwang, Szerenyi, and Gupta teaches the method of claim 1 as recited above. Eizenman does not appear to disclose, but Hwang teaches the limitation of ‘further comprising: isolating, by a bot, failures due to a test framework or environmental instability from failures due to code issues’ ([0040] Fig. 2, block 206, security issues are localized based on data collected at block 202 by a security risk program and used to train a machine learning model. The security risk program is also the entity responsible for localizing the security issues based on. [0042] Security risk program 110a/110b captures information relating to the security issues, and will identify repeating issues or patterns, the problem is localized with application and system logs, corresponding to the isolating of failures due to environmental instability from failures due to code issues.).
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Eizenman, Hwang, Szerenyi, and Gupta before them, to include Hwang’s ‘further comprising: isolating, by a bot, failures due to a test framework or environmental instability from failures due to code issues’ in Eizenman’s method performing ‘receiving, at an automation pipeline, code for shift-left testing’. One would have been motivated to make such a combination to increase efficiency by allowing issues to be remediated faster while reducing costs when this process is performed earlier in the development of software, as taught by Hwang [0019].
Regarding claim 7, Eizenman in view of Hwang, Szerenyi, and Gupta teaches the method of claim 1 as recited above. Eizenman does not appear to disclose, but Hwang teaches the limitation of ‘wherein the bot is trained with historical data’ ([0023] The machine learning analysis may be created using historical data collected through shift-left activities, including the testing phases. When taking into account the model that a security risk program utilizes).
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Eizenman, Hwang, Szerenyi, and Gupta before them, to include Hwang’s ‘wherein the bot is trained with historical data’ in Eizenman’s method performing ‘receiving, at an automation pipeline, code for shift-left testing’. One would have been motivated to make such a combination to increase efficiency by having a model that learns from the shift left activities, and identify the issues that can occur in the code of a program being developed before it can grow to become a major issue, and remediate common mistakes or issues that can occur, as taught by Hwang [0021].
Regarding claim 15, Eizenman in view of Hwang, Szerenyi, and Gupta teaches the method of independent claim 1 as recited above, as independent claim 15 is a non-transitory computer readable storage medium with similar limitations to independent claim 1 above. Eizenman also discloses “receiving, from a code repository, code for shift-left testing, wherein the code comprises a plurality of code components” ([0057] Developments in a continuous testing environment occurs, and the developments happening on the left side of testing stages corresponds to the shift-left testing. Fig. 1B shows a method of an invention that receives code to evaluate, with steps 2-4 corresponding to an automation pipeline, as it performs the various tasks to verify software builds. [0065] Storage 142 has information of the content of a build, including code used for the build.);
Eizenman does not appear to disclose, but Hwang teaches ‘a non-transitory computer readable storage medium, including instructions stored thereon, which when read and executed by one or more computer processors, cause the one or more computer processors to perform steps comprising’ ([0087]-[0088] Computer readable storage media 915 can include storage devices of varying kinds, including electronic and magnetic storage devices. The media contains instructions for a program to be executed by a processor to perform the functions of the invention.):
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Eizenman, Hwang, Szerenyi, and Gupta before them, to include Hwang’s ‘non-transitory computer readable storage medium, including instructions stored thereon’ in Eizenman’s system performing ‘receiving, from a code repository, code for shift-left testing, wherein the code comprises a plurality of code components’. One would have been motivated to make such a combination to increase efficiency by containing a storage media to retain and store a program and digital information in a physical medium, as taught by Hwang [0058].
Regarding claim 16, Eizenman in view of Hwang, Szerenyi, and Gupta teaches the non-transitory computer readable storage medium of claim 15 as recited above. Eizenman also discloses the limitations present in dependent claim 2 above.
Regarding claim 17, Eizenman in view of Hwang, Szerenyi, and Gupta teaches the non-transitory computer readable storage medium of claim 15 as recited above. Eizenman also discloses the limitations present in dependent claim 3 above.
Regarding claim 20, Eizenman in view of Hwang, Szerenyi, and Gupta teaches the non-transitory computer readable storage medium of claim 15 as recited above. Eizenman also discloses the limitations present in dependent claim 6 above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TOMMY MARTINEZ whose telephone number is (703)756-5651. The examiner can normally be reached Monday thru Friday 8AM-4PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jorge L. Ortiz-Criado can be reached at (571) 272-7624 on Monday thru Friday 7AM-7PM ET. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.M./ Examiner, Art Unit 2496 /JORGE L ORTIZ CRIADO/Supervisory Patent Examiner, Art Unit 2496
Prosecution Insights