DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is filed in response to the Applicant’s Arguments and Remarks amendment dated on 07/14/2026. Claims 1, 12, 13, 15, and 18 are currently amended, Claim 11 is cancelled, Claim 21 is new, and Claims 1-10 and 12-21 are pending.
In view of Applicant’s Amendments and Remarks, the 35 USC 101 rejections are withdrawn.
In response to applicant’s argument that there is no teaching, suggestion, or motivation to modify the Sen and Kadirvel references, the examiner recognizes that obviousness may be established by modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, further support is given from Kadirvel for motivation to modify the teachings of Sen, on pages 28 and 29 of this action.
Applicant’s arguments with respect to the prior art rejections have been considered but are moot in view of the new grounds of rejection presented herein.
Examiner’s Notes
Examiner cites columns, paragraphs, figures and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-8 and 12-21 are rejected under 35 U.S.C. 102(a)(2) as being unpatentable by Sen et al. (U.S. Publication No. 2025/0077399, hereinafter Sen).
Regarding Claim 1:
Sen discloses, “An apparatus comprising: at least one processing device comprising a processor coupled to a memory; the at least one processing device being configured:” ([0002]; “an apparatus that may include a memory, a network interface configured to receive a request to test a software program stored in the memory from a user device, the request comprising a description of requirements of the software program, and a processor coupled to the memory and the network interface, the processor configured to generate a plurality of testing elements”.)
“to generate a first data structure by parsing a request to generate a given test scenario for testing of an information technology asset;” ([0056]; “The request may include an identifier of the model 224, such as a unique ID assigned by the host platform 220, a payload of data (e.g., to be input to the model during execution), and the like.”; [0057]; “In some embodiments, the data payload may be a format that cannot be input to the model 224 nor read by a computer processor. For example, the data payload may be text, image, audio, etc. In response, the AI engine 222 may convert the data payload into a format readable by the model 224, such as a vector or other encoding.”; [0081]; “According to various embodiments, the host platform 520 also hosts a generative artificial intelligence (GenAI) model 524 capable of generating a software test based on inputs received via the user interface 512. For example, the GenAI model 524 may be trained on a large corpus of software tests and software test documentation and can receive a description of a software test, such as the requirements of the test, and generate a software test document (e.g., the software test 530) that includes a set of instructions to be performed, expected results, and content areas that can be left blank for insertion by the user during testing.”; [0055]; “Referring to FIG. 2, a software application 210 may request execution of the model 224 by submitting a request to the host platform 220.”; [0039]; "The example embodiments are directed to a platform that can generate/design software tests, automation scripts for testing software, source code, software patches, and the like … Furthermore, the GenAI model may also generate new software tests based on descriptions thereof, such as a description of the requirements of the software test."; [0052]; “Referring to FIG. 1B, the software test case 150 includes a diagram with a plurality of boxes/cells identifying the actions to be taken.”; (Abstract); "An example operation may include one or more of receiving a description of a plurality of testing elements for testing a software program...".)
(Examiner’s Note: The payload data in a request is parsed by the AI engine by the use of conversion, to generate the test case of the first data structure. The testing is intended for testing a software program. Because a software program is a type of information technology asset, it can then be said a given test scenario is used for testing of an information technology asset. Therefore, the reference teaches the same limitation as the examined claim.)
“to process the first data structure utilizing a machine learning model to generate a second data structure, the second data structure comprising (i) a given sequence of test steps for the given test scenario” ([0046]; “Furthermore, the GenAI model described herein can be integrated within a larger artificial intelligence system that includes machine learning”; [0063]; “The GenAI model 322 may be executed on training data … the training data may include software tests”; [0081]; “… the host platform 520 also hosts a generative artificial intelligence (GenAI) model 524 capable of generating a software test”; [0082]; “FIG. 5B illustrates a process 540 of generating a sequence of steps based on generative AI according to example embodiments.”; (Figure 5B); Software Tests (element 526) is input to the GenAI Model (element 524), where it is labeled to “Generate Steps”. The following “Steps” from the GenAI model are labeled on the output arrow to produce the test case with test steps, displayed on the user interface (element 512).)
“and (ii) a specification of one or more test bed characteristics to be utilized for executing the given test scenario;” ([0077]; “FIGS. 5A-5D illustrate a process of generating a software test”; [0082]: "FIG. 5B illustrates a process 540 of generating a sequence of steps based on generative AI according to example embodiments. Referring to FIG. 5B, in this example, a user may submit a list of requirements or features of the software program that the user wishes to test"; [0084]; “the steps may be determined by the GenAI model 524 based on historical software tests that have had similar requirements”; [0080]; “the host platform 520 also includes a script database 528 that stores automation scripts that enable the software tests to be performed/executed”.)
“to map the given sequence of test steps in the second data structure to respective application programming interface calls of a test automation framework, each of the application programming interface calls being associated with a functional code test unit of a test code database of the test automation framework;” ([0047]; “FIG. 1A illustrates a GenAI computing environment 100 that includes a system for generating test cases according to example embodiments. In the example of FIG. 1A, the system includes a test execution service 130 that hosts a software testing environment and can execute software tests stored in a test repository 132. In the example embodiments, the test execution service 130 can execute a test in any of multiple different frameworks/different programming languages”; [0048]; “According to various embodiments, the test execution service 130 is coupled to a framework connector 140 (software program), which manages the execution of the software tests via a plurality of different frameworks”; [0049]; “Further, the GenAI model 120 may communicate with the test execution service 130 directly via API calls or the like. In this example, the GenAI model 120 may include a data store 122 with a repository of software tests, automation scripts, source code, vulnerability code, software patches, and the like.”; [0065]; “In this example, an executable script 326 is developed and configured to read data from a database 324 and input the data to the GenAI model 322 while the GenAI model is running/executing via the AI engine 321.”; [0097]; “the sequence of steps (step definition) may include mappings between each step/scenario within the software test and the code function to be executed by the script. The step definition can be thought of as the automation script”.)
(Examiner’s Note: The reference teaches that the sequence of test steps is mapped to respective application programming interface (API) calls within the software test. The API call is associated with functional code test unit through the test execution service 130 element shown in Figure 1A communicating with the GenAI model 120 element, where the model may communicate with the data store 122 containing automation scripts used as functional code tests. As shown in Figure 1A, the test execution service 130 is connected to the framework connector 140, where test automation is managed and may execute test code in its respective framework.)
“to configure a test bed with the one or more test bed characteristics specified in the second data structure, wherein configuring the test bed comprises running one or more workloads on the test bed;” (The host platform is mapped as the test bed. [0080]: "the host platform 520 also includes a script database 528 that stores automation scripts that enable the software tests to be performed/executed in an automated manner." [0085]: "FIG. 5C illustrates a process 550 of generating a new software test from the steps … the host platform 520 may generate a document, an XML file, a JSON, etc., describing the steps"; [0086]: “Referring again to FIG. 5C, the new software test 552 may include a description 554 of the overall theme or purpose of the test”; [0088]: “FIG. 5D illustrates a process 560 of executing the new software test 552”.)
“and to execute the given test scenario on the test bed, utilizing the mapped application programming interface calls of the test automation framework, in conjunction with running the one or more workloads on the test bed” ([0049]; In Fig 6A, processes are executed in host platform 620, where the test bed is mapped to the host platform. [0091]: “FIG. 6A illustrates a computing environment 600 of a host platform 620 that generates an automation script 630”; [0092]: “the host platform 620 also hosts a generative artificial intelligence (GenAI) model 624 capable of generating an automation script for automating the execution of a software test”; [0098]: "FIG. 6C illustrates a process 650 of executing a software test via the testing software 622 based on the automation script 630 generated in FIG. 6A"; [0099]: “Here, the testing software 622 reads the step definition from the automation script, held in the repository of automation scripts 628. The testing software 622 can map the next step to be performed within the software test to the corresponding code module in the automation script and identify the code function(s) to execute and executes it.”; [0071]: " the GenAI model described herein may be trained based on custom-defined prompts designed to draw out specific attributes associated with a software test, automation script, vulnerability, or the like. These same prompts may be output during the live execution of the GenAI model".)
Regarding Claim 2:
Sen discloses, “The apparatus of claim 1 wherein the request to generate the given test scenario comprises a natural language description of a testing goal for the given test scenario.” ([0043]; “For example, the GenAI model may include libraries and deep learning frameworks that enable the GenAI model to create software tests, activation scripts, source code, etc., based on text inputs.”; [0044]; “By creating software tests from natural language descriptions, the GenAI model can relieve a user from generating such tests manually. Furthermore, the GenAI model described herein can learn activation scripts for executing the software tests “automatically” on the software program being tested.”)
Regarding Claim 3:
Sen discloses, “wherein generating the first data structure comprises selection of one or more test steps from a test management environment comprising a repository of one or more existing test scenarios and associated test steps.” ([0085]; “For example, FIG. 5C illustrates a process 550 of generating a new software test from the steps shown on the user interface in FIG. 5B. Referring to FIG. 5C, a user may press a submit button 549 on the user interface 512 or otherwise accept the steps shown on the user interface for inclusion in a software test.”; [0047]; "the system includes a test execution service 130 that hosts a software testing environment and can execute software tests stored in a test repository 132. In the example embodiments, the test execution service 130 can execute a test in any of multiple different frameworks/different programming languages"; [0049]; "the GenAl model 120 may include a data store 122 with a repository of software tests, automation scripts, source code, vulnerability code, software patches, and the like."; [0099]; “Here, the testing software 622 reads the step definition from the automation script, held in the repository of automation scripts 628.”)
Regarding Claim 4:
Sen discloses, “The apparatus of claim 3 wherein generating the first data structure comprises selecting at least one of one or more existing test scenarios from a repository of the test management environment” ([0086]; “Referring again to FIG. 5C, the new software test 552 may include a description 554 of the overall theme or purpose of the test, which is also generated by the GenAI model 524 and added/appended to the new software test 552 as a label. The description 554 can be searched by a search engine or other software program when searching for software tests to perform.”; [0087]; Here, the host system may store the new software test 552 paired with the description 554 in the software test repository 526.”; [0089]; “As noted, the description 554 added to the new software test 552 may be used to search the software test repository 526 for and select a software test based on a search term input from the user interface 570.”)
Regarding Claim 5:
Sen discloses, “The apparatus of claim 4 wherein processing the first data structure utilizing the machine learning model comprises utilizing the selected at least one existing test scenario for adapting the machine learning model to a testing context of the selected at least one existing test scenario.” ([0064]; "the training process may use executional results that have already been generated/output by the GenAl model 322 in a live environment (including any customer feedback, etc.)”; [0059]; “In some embodiments, the software application 210 may display a user interface that enables a user to provide feedback from the output provided by the model 224. For example, a user may input a confirmation that the test case generated by a GenAI model is correct or includes incorrect content. This information may be added to the results of execution and stored within a log 225. The log 225 may include an identifier of the input, an identifier of the output, an identifier of the model used, and feedback from the recipient. This information may be used to subsequently retrain the model.”; [0085]; “For example, FIG. 5C illustrates a process 550 of generating a new software test from the steps shown on the user interface in FIG. 5B. Referring to FIG. 5C, a user may … otherwise accept the steps shown on the user interface for inclusion in a software test.”; [0086]; “In some embodiments, the GenAI model 524 may generate other attributes associated with the new software test 552, including an activation script for executing the new software test 552 in an automated manner, a description/label for the new software test 552, and the like.”)
Regarding Claim 6:
Sen discloses, “The apparatus of claim 1 wherein the machine learning model comprises a large language model.” ([0043]; “According to various embodiments, the GenAI model may be a large language model (LLM), such as a multimodal large language model.”)
Regarding Claim 7:
Sen discloses, “The apparatus of claim 1 wherein processing the first data structure utilizing the machine learning model comprises generating two or more different sequences of test steps as alternatives for the given test scenario.” ([0082]; “FIG. 5B illustrates a process 540 of generating a sequence of steps based on generative AI according to example embodiments. Referring to FIG. 5B, in this example, a user may submit a list of requirements or features of the software program that the user wishes to test.”; [0084]; “In response, the GenAI model 524 may generate a sequence of steps ... Here, the steps may be determined by the GenAI model 524 based on historical software tests that have had similar requirements that the GenAI model has learned from. In this example, the steps … are displayed on the user interface 512, where the user of the user device can view the steps and provide feedback or accept the steps. Each step is another step during the test of the software program.”; [0085]; “For example, FIG. 5C illustrates a process 550 of generating a new software test from the steps shown on the user interface in FIG. 5B. Referring to FIG. 5C, a user may press a submit button 549 on the user interface 512 or otherwise accept the steps shown on the user interface for inclusion in a software test. In response, the system described herein inputs the steps on the user interface 512 into a new software test 552, which includes the steps shown on the user interface … as well as the order in which the sequence of steps are in.”)
(Examiner’s note: The reference teaches that multiple differing sequence of test steps can be produced for the given test scenario based on software tests. A new set of test steps may be generated through the user interface, where a user has the option to press a submit button and provide feedback to train the GenAI model. When test steps are generated, it is based on generative AI (paragraph [0082]). The new set of steps can base its generation on retrained information if the user provides feedback for the model to use.)
Regarding Claim 8,
The limitation, “The apparatus of claim 7 wherein generating the second data structure comprises: presenting the two or more different sequences of test steps to a source of the request to generate the given test scenario; selecting one of the two or more different sequences of test steps based at least in part on feedback received from the source of the request to generate the given test scenario; and adding the selected one of the two or more different sequences of test steps as the given sequence of test steps in the second data structure.” ([0084]; “In response, the GenAI model 524 may generate a sequence of steps ... In this example, the steps, including the step 542, the step 544, the step 546, and the step 548, are displayed on the user interface 512, where the user of the user device can view the steps and provide feedback or accept the steps. Each step is another step during the test of the software program.”; [0085]; “For example, FIG. 5C illustrates a process 550 of generating a new software test from the steps shown on the user interface in FIG. 5B. Referring to FIG. 5C, a user may press a submit button 549 on the user interface 512 or otherwise accept the steps shown on the user interface for inclusion in a software test. In response, the system described herein inputs the steps on the user interface 512 into a new software test 552, which includes the steps shown on the user interface, including the step 542, the step 544, the step 546, and the step 548, as well as the order in which the sequence of steps are in. For example, the host platform 520 may generate a document, an XML file, a JSON, etc., describing the steps.”; [0086]; “In some embodiments, the GenAI model 524 may generate other attributes associated with the new software test 552, including an activation script for executing the new software test 552 in an automated manner, a description/label for the new software test 552, and the like.”)
The second data structure is generated by the GenAI model. The steps are then presented on an interface to the user, being the source of the request. FIG. 5C illustrates a process 550 of generating a new software test 552 from the steps shown on the user interface 512 in FIG. 5B. The user has the ability to select the current sequence of steps with the submit button 549 of Fig. 5C, or generate a second different sequence of steps based on user feedback. Once the user accepts the sequence of steps they are satisfied with, a new test scenario is generated as a software test containing the selected sequence of steps, and put into a repository of software tests. The repository of software tests 526 may be used to associate an activation script for executing the given sequence of test steps held within the software test.
Regarding Claim 12:
The limitation, “The apparatus of claim 1 wherein the one or more test bed characteristics comprises at least one of a hardware and a software configuration for the test bed to be utilized for executing the given test scenario.” ([0047]; “In the example of FIG. 1A, the system includes a test execution service 130 that hosts a software testing environment and can execute software tests stored in a test repository 132. In the example embodiments, the test execution service 130 can execute a test in any of multiple different frameworks/different programming languages, including PYTHON®, JAVA®, C++, and the like”; [0048]; “According to various embodiments, the test execution service 130 is coupled to a framework connector 140 (software program), which manages the execution of the software tests via a plurality of different frameworks ... Each framework may include a collection of tools and software libraries necessary for the framework.”; [0153]; “As will be appreciated by one skilled in the art, aspects of the present application may be embodied as a system, method, or computer program product. Accordingly, aspects of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.), or an embodiment combining software and hardware aspects that may all generally be referred to herein as a ‘circuit,’ ‘module’ or ‘system.’”; [0012]; “A further example embodiment provides an apparatus that may include a memory configured to store a log file, a display, and a processor coupled to the memory and the display, the processor configured to execute tests on a software application stored in the memory and running on the apparatus via a test environment of a test platform and log results of the tests in the log file…”)
Regarding Claim 13:
The limitation, “The apparatus of claim 1 wherein the one or more test bed characteristics specifies one or more workloads to run on the test bed during execution of the given test scenario.” ([0047]; “FIG. 1A illustrates a GenAI computing environment 100 that includes a system for generating test cases … the system includes a test execution service 130 that hosts a software testing environment and can execute software tests stored in a test repository 132.”; [0051]; “For example, the user interface 110 may be included in a front-end of the software application shown on a user device 102 screen that accesses the host platform and the test execution service via a computer network such as the Internet. The user interface 110 may include controls that can be used to build software tests, perform software testing, develop automation scripts, and the like”. [0071]: “the GenAI model described herein may be trained based on custom-defined prompts designed to draw out specific attributes associated with a software test … a user may input a software test description, such as the requirements that the software test will be expected to perform when testing a software program”; [0070]: “the GUI menu options include options for adding features such as neural networks, machine learning models, AI models, data sources, conversion processes (e.g., vectorization, encoding, etc.), analytics, etc. The user can continue to add features to the model … the user can save the model for subsequent training/testing”.)
Regarding Claim 14:
Sen discloses, “The apparatus of claim 1 wherein at least a given one of the application programming interface calls of the test automation framework associated with a given functional code test unit comprises at least one of a validation and a verification of a given one of the test steps to be performed at least one of prior to and subsequent to execution of the given functional code test unit.” ([0133]; “In some embodiments, the generating may include generating a plurality of test components of the software test, including a test specification, a test execution, a test recording, and a test verification based on the execution of the GenAI model. In some embodiments, the generating may include generating a respective requirement for each component within the software test based on the execution of the large language model on the received descriptions.”; [0048]; “According to various embodiments, the test execution service 130 is coupled to a framework connector 140 (software program), which manages the execution of the software tests … Each framework may create test scripts by developing required code or running commands in the test environment of the respective framework.”; [0049]; “Further, the GenAI model 120 may communicate with the test execution service 130 directly via API calls or the like.”)
(Examiner’s note: Each test component may be generated with the respective requirement of verification for each test step based on the execution of the GenAI model. The GenAI model may use an application programming interface call associated with the test execution service that may comprise of a functional code test unit from test scripts.)
Regarding Claim 15:
Sen discloses, “A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device:” ([0007]; “a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of receiving a description of a plurality of testing elements for testing a software program and storing the software test within a storage device, generating an automation script for automating execution of the software test based on execution of a generative artificial intelligence model (GenAI) model on the plurality of testing elements and a repository of automation scripts, attaching the automation script to the software test within the storage device, and in response to a request to execute the software test, executing the plurality of testing elements within the software test based on the attached automation script.”)
“to generate a first data structure by parsing a request to generate a given test scenario for testing of an information technology asset;” ([0056]; “The request may include an identifier of the model 224, such as a unique ID assigned by the host platform 220, a payload of data (e.g., to be input to the model during execution), and the like.”; [0057]; “In some embodiments, the data payload may be a format that cannot be input to the model 224 nor read by a computer processor. For example, the data payload may be text, image, audio, etc. In response, the AI engine 222 may convert the data payload into a format readable by the model 224, such as a vector or other encoding.”; [0081]; “According to various embodiments, the host platform 520 also hosts a generative artificial intelligence (GenAI) model 524 capable of generating a software test based on inputs received via the user interface 512. For example, the GenAI model 524 may be trained on a large corpus of software tests and software test documentation and can receive a description of a software test, such as the requirements of the test, and generate a software test document (e.g., the software test 530) that includes a set of instructions to be performed, expected results, and content areas that can be left blank for insertion by the user during testing.”; [0055]; “Referring to FIG. 2, a software application 210 may request execution of the model 224 by submitting a request to the host platform 220.”; [0039]; "The example embodiments are directed to a platform that can generate/design software tests, automation scripts for testing software, source code, software patches, and the like … Furthermore, the GenAI model may also generate new software tests based on descriptions thereof, such as a description of the requirements of the software test."; [0052]; “Referring to FIG. 1B, the software test case 150 includes a diagram with a plurality of boxes/cells identifying the actions to be taken.”; (Abstract); "An example operation may include one or more of receiving a description of a plurality of testing elements for testing a software program...".)
(Examiner’s note: The payload data in a request is parsed by the AI engine by the use of conversion, to generate the test case of the first data structure. The testing is intended for testing a software program. Because a software program is a type of information technology asset, it can then be said a given test scenario is used for testing of an information technology asset. Therefore, the reference teaches the same limitation as the examined claim.)
“to process the first data structure utilizing a machine learning model to generate a second data structure, the second data structure comprising (i) a given sequence of test steps for the given test scenario” ([0046]; “Furthermore, the GenAI model described herein can be integrated within a larger artificial intelligence system that includes machine learning”; [0063]; “The GenAI model 322 may be executed on training data … the training data may include software tests”; [0081]; “… the host platform 520 also hosts a generative artificial intelligence (GenAI) model 524 capable of generating a software test”; [0082]; “FIG. 5B illustrates a process 540 of generating a sequence of steps based on generative AI according to example embodiments.”; (Figure 5B); Software Tests (element 526) is input to the GenAI Model (element 524), where it is labeled to “Generate Steps”. The following “Steps” from the GenAI model are labeled on the output arrow to produce the test case with test steps, displayed on the user interface (element 512).)
“and (ii) a specification of one or more test bed characteristics to be utilized for executing the given test scenario;” ([0077]; “FIGS. 5A-5D illustrate a process of generating a software test”; [0082]: "FIG. 5B illustrates a process 540 of generating a sequence of steps based on generative AI according to example embodiments. Referring to FIG. 5B, in this example, a user may submit a list of requirements or features of the software program that the user wishes to test"; [0084]; “the steps may be determined by the GenAI model 524 based on historical software tests that have had similar requirements”; [0080]; “the host platform 520 also includes a script database 528 that stores automation scripts that enable the software tests to be performed/executed”.)
“to map the given sequence of test steps in the second data structure to respective application programming interface calls of a test automation framework, each of the application programming interface calls being associated with a functional code test unit of a test code database of the test automation framework;” ([0047]; “FIG. 1A illustrates a GenAI computing environment 100 that includes a system for generating test cases according to example embodiments. In the example of FIG. 1A, the system includes a test execution service 130 that hosts a software testing environment and can execute software tests stored in a test repository 132. In the example embodiments, the test execution service 130 can execute a test in any of multiple different frameworks/different programming languages”; [0048]; “According to various embodiments, the test execution service 130 is coupled to a framework connector 140 (software program), which manages the execution of the software tests via a plurality of different frameworks”; [0049]; “Further, the GenAI model 120 may communicate with the test execution service 130 directly via API calls or the like. In this example, the GenAI model 120 may include a data store 122 with a repository of software tests, automation scripts, source code, vulnerability code, software patches, and the like.”; [0065]; “In this example, an executable script 326 is developed and configured to read data from a database 324 and input the data to the GenAI model 322 while the GenAI model is running/executing via the AI engine 321.”; [0097]; “the sequence of steps (step definition) may include mappings between each step/scenario within the software test and the code function to be executed by the script. The step definition can be thought of as the automation script”.)
(Examiner’s note: The reference teaches that the sequence of test steps is mapped to respective application programming interface (API) calls within the software test. The API call is associated with functional code test unit through the test execution service 130 element shown in Figure 1A communicating with the GenAI model 120 element, where the model may communicate with the data store 122 containing automation scripts used as functional code tests. As shown in Figure 1A, the test execution service 130 is connected to the framework connector 140, where test automation is managed and may execute test code in its respective framework.)
“to configure a test bed with the one or more test bed characteristics specified in the second data structure, wherein configuring the test bed comprises running one or more workloads on the test bed;” (The host platform is mapped as the test bed. [0080]: "the host platform 520 also includes a script database 528 that stores automation scripts that enable the software tests to be performed/executed in an automated manner." [0085]: "FIG. 5C illustrates a process 550 of generating a new software test from the steps … the host platform 520 may generate a document, an XML file, a JSON, etc., describing the steps"; [0086]: “Referring again to FIG. 5C, the new software test 552 may include a description 554 of the overall theme or purpose of the test”; [0088]: “FIG. 5D illustrates a process 560 of executing the new software test 552”.)
“and to execute the given test scenario on the test bed, utilizing the mapped application programming interface calls of the test automation framework, in conjunction with running the one or more workloads on the test bed” ([0049]; [0091]: “FIG. 6A illustrates a computing environment 600 of a host platform 620 that generates an automation script 630”; [0092]: “the host platform 620 also hosts a generative artificial intelligence (GenAI) model 624 capable of generating an automation script for automating the execution of a software test”; [0098]: "FIG. 6C illustrates a process 650 of executing a software test via the testing software 622 based on the automation script 630 generated in FIG. 6A"; [0099]; “Here, the testing software 622 reads the step definition from the automation script, held in the repository of automation scripts 628. The testing software 622 can map the next step to be performed within the software test to the corresponding code module in the automation script and identify the code function(s) to execute and executes it.”; [0071]: " the GenAI model described herein may be trained based on custom-defined prompts designed to draw out specific attributes associated with a software test, automation script, vulnerability, or the like. These same prompts may be output during the live execution of the GenAI model".)
Regarding Claim 16:
Sen discloses, “The computer program product of claim 15 wherein the machine learning model comprises a large language model.” ([0043]; “According to various embodiments, the GenAI model may be a large language model (LLM), such as a multimodal large language model.”)
Regarding Claim 17:
Sen discloses, “The computer program product of claim 15 wherein generating the first data structure comprises selecting at least one of one or more existing test scenarios from a repository of a test management environment, and wherein processing the first data structure utilizing the machine learning model comprises utilizing the selected at least one existing test scenario for adapting the machine learning model to a testing context of the selected at least one existing test scenario.” ([0004]; “A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of receiving a request to test a software program from a user device, the request comprising a description of requirements of the software program, generating a plurality of testing elements based on execution of a generative artificial intelligence (GenAI) model on the description of the requirements of the software program and a repository of test cases, generating a test case for testing the software program where the test case comprises the plurality of testing elements generated by the GenAI model, and storing the test case within a storage device.”; [0011]; “A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of generating a large language model via a user interface, executing the large language model on a repository of software test cases and requirements of the software test cases to train the large language model to understand connections between test case components and test case requirements, receiving a description of features of a software program to be tested; and in response to receiving the description of the features, generating a software test case based on execution of the large language model on the received descriptions, and displaying the software test case via the user interface.”)
Regarding Claim 18:
Sen discloses, “A method comprising:” ([0003]; “Another example embodiment provides a method that includes one or more of receiving a request to test a software program from a user device, the request comprising a description of requirements of the software program, generating a plurality of testing elements based on execution of a generative artificial intelligence (GenAI) model on the description of the requirements of the software program and a repository of test cases, generating a test case for testing the software program where the test case comprises the plurality of testing elements generated by the GenAI model, and storing the test case within a storage device.”)
“generating a first data structure by parsing a request to generate a given test scenario for testing of an information technology asset;” ([0056]; “The request may include an identifier of the model 224, such as a unique ID assigned by the host platform 220, a payload of data (e.g., to be input to the model during execution), and the like.”; [0057]; “In some embodiments, the data payload may be a format that cannot be input to the model 224 nor read by a computer processor. For example, the data payload may be text, image, audio, etc. In response, the AI engine 222 may convert the data payload into a format readable by the model 224, such as a vector or other encoding.”; [0081]; “According to various embodiments, the host platform 520 also hosts a generative artificial intelligence (GenAI) model 524 capable of generating a software test based on inputs received via the user interface 512. For example, the GenAI model 524 may be trained on a large corpus of software tests and software test documentation and can receive a description of a software test, such as the requirements of the test, and generate a software test document (e.g., the software test 530) that includes a set of instructions to be performed, expected results, and content areas that can be left blank for insertion by the user during testing.”; [0055]; “Referring to FIG. 2, a software application 210 may request execution of the model 224 by submitting a request to the host platform 220.”; [0039]; "The example embodiments are directed to a platform that can generate/design software tests, automation scripts for testing software, source code, software patches, and the like … Furthermore, the GenAI model may also generate new software tests based on descriptions thereof, such as a description of the requirements of the software test."; [0052]; “Referring to FIG. 1B, the software test case 150 includes a diagram with a plurality of boxes/cells identifying the actions to be taken.”; (Abstract); "An example operation may include one or more of receiving a description of a plurality of testing elements for testing a software program...".)
(Examiner’s note: The payload data in a request is parsed by the AI engine by the use of conversion, to generate the test case of the first data structure. The testing is intended for testing a software program. Because a software program is a type of information technology asset, it can then be said a given test scenario is used for testing of an information technology asset. Therefore, the reference teaches the same limitation as the examined claim.)
“processing the first data structure utilizing a machine learning model to generate a second data structure, the second data structure comprising (i) a given sequence of test steps for the given test scenario;” ([0046]; “Furthermore, the GenAI model described herein can be integrated within a larger artificial intelligence system that includes machine learning”; [0063]; “The GenAI model 322 may be executed on training data … the training data may include software tests”; [0081]; “… the host platform 520 also hosts a generative artificial intelligence (GenAI) model 524 capable of generating a software test”; [0082]; “FIG. 5B illustrates a process 540 of generating a sequence of steps based on generative AI according to example embodiments.”; (Figure 5B); Software Tests (element 526) is input to the GenAI Model (element 524), where it is labeled to “Generate Steps”. The following “Steps” from the GenAI model are labeled on the output arrow to produce the test case with test steps, displayed on the user interface (element 512).)
“and (ii) a specification of one or more test bed characteristics to be utilized for executing the given test scenario;” ([0077]; “FIGS. 5A-5D illustrate a process of generating a software test”; [0082]: "FIG. 5B illustrates a process 540 of generating a sequence of steps based on generative AI according to example embodiments. Referring to FIG. 5B, in this example, a user may submit a list of requirements or features of the software program that the user wishes to test"; [0084]; “the steps may be determined by the GenAI model 524 based on historical software tests that have had similar requirements”; [0080]; “the host platform 520 also includes a script database 528 that stores automation scripts that enable the software tests to be performed/executed”.)
“mapping the given sequence of test steps in the second data structure to respective application programming interface calls of a test automation framework, each of the application programming interface calls being associated with a functional code test unit of a test code database of the test automation framework;” ([0047]; “FIG. 1A illustrates a GenAI computing environment 100 that includes a system for generating test cases according to example embodiments. In the example of FIG. 1A, the system includes a test execution service 130 that hosts a software testing environment and can execute software tests stored in a test repository 132. In the example embodiments, the test execution service 130 can execute a test in any of multiple different frameworks/different programming languages”; [0048]; “According to various embodiments, the test execution service 130 is coupled to a framework connector 140 (software program), which manages the execution of the software tests via a plurality of different frameworks”; [0049]; “Further, the GenAI model 120 may communicate with the test execution service 130 directly via API calls or the like. In this example, the GenAI model 120 may include a data store 122 with a repository of software tests, automation scripts, source code, vulnerability code, software patches, and the like.”; [0065]; “In this example, an executable script 326 is developed and configured to read data from a database 324 and input the data to the GenAI model 322 while the GenAI model is running/executing via the AI engine 321.”; [0097]; “the sequence of steps (step definition) may include mappings between each step/scenario within the software test and the code function to be executed by the script. The step definition can be thought of as the automation script”.)
(Examiner’s note: The reference teaches that the sequence of test steps is mapped to respective application programming interface (API) calls within the software test. The API call is associated with functional code test unit through the test execution service 130 element shown in Figure 1A communicating with the GenAI model 120 element, where the model may communicate with the data store 122 containing automation scripts used as functional code tests. As shown in Figure 1A, the test execution service 130 is connected to the framework connector 140, where test automation is managed and may execute test code in its respective framework.)
“configuring a test bed with the one or more test bed characteristics specified in the second data structure, wherein configuring the test bed comprises running one or more workloads on the test bed” (The host platform is mapped as the test bed. [0080]: "the host platform 520 also includes a script database 528 that stores automation scripts that enable the software tests to be performed/executed in an automated manner." [0085]: "FIG. 5C illustrates a process 550 of generating a new software test from the steps … the host platform 520 may generate a document, an XML file, a JSON, etc., describing the steps"; [0086]: “Referring again to FIG. 5C, the new software test 552 may include a description 554 of the overall theme or purpose of the test”; [0088]: “FIG. 5D illustrates a process 560 of executing the new software test 552”.)
“and executing the given test scenario on the test bed, utilizing the mapped application programming interface calls of the test automation framework, in conjunction with running the one or more workloads on the test bed;” ([0049]; [0091]: “FIG. 6A illustrates a computing environment 600 of a host platform 620 that generates an automation script 630”; [0092]: “the host platform 620 also hosts a generative artificial intelligence (GenAI) model 624 capable of generating an automation script for automating the execution of a software test”; [0098]: "FIG. 6C illustrates a process 650 of executing a software test via the testing software 622 based on the automation script 630 generated in FIG. 6A"; [0099]; “Here, the testing software 622 reads the step definition from the automation script, held in the repository of automation scripts 628. The testing software 622 can map the next step to be performed within the software test to the corresponding code module in the automation script and identify the code function(s) to execute and executes it.”; [0071]: " the GenAI model described herein may be trained based on custom-defined prompts designed to draw out specific attributes associated with a software test, automation script, vulnerability, or the like. These same prompts may be output during the live execution of the GenAI model".)
“wherein the method is performed by at least one processing device comprising a processor coupled to a memory.” ([0003]; “Another example embodiment provides a method that includes one or more of receiving a request to test a software program from a user device, the request comprising a description of requirements of the software program, generating a plurality of testing elements based on execution of a generative artificial intelligence (GenAI) model on the description of the requirements of the software program and a repository of test cases, generating a test case for testing the software program where the test case comprises the plurality of testing elements generated by the GenAI model, and storing the test case within a storage device.”)
(Examiner’s Note: The reference teaches that the method may be performed by the GenAI model, which processes the description of the test scenario from the user device to produce a software test, which then is put into storage.)
Regarding Claim 19:
Sen discloses, “The method of claim 18 wherein the machine learning model comprises a large language model.” ([0043]; “According to various embodiments, the GenAI model may be a large language model (LLM), such as a multimodal large language model.”)
Regarding Claim 20:
Sen discloses, “The method of claim 18 wherein generating the first data structure comprises selecting at least one of one or more existing test scenarios from a repository of a test management environment, and wherein processing the first data structure utilizing the machine learning model comprises utilizing the selected at least one existing test scenario for adapting the machine learning model to a testing context of the selected at least one existing test scenario.” ([0003]; “Another example embodiment provides a method that includes one or more of receiving a request to test a software program from a user device, the request comprising a description of requirements of the software program, generating a plurality of testing elements based on execution of a generative artificial intelligence (GenAI) model on the description of the requirements of the software program and a repository of test cases, generating a test case for testing the software program where the test case comprises the plurality of testing elements generated by the GenAI model, and storing the test case within a storage device.”; [0011]; “A further example embodiment provides a computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform one or more of generating a large language model via a user interface, executing the large language model on a repository of software test cases and requirements of the software test cases to train the large language model to understand connections between test case components and test case requirements, receiving a description of features of a software program to be tested; and in response to receiving the description of the features, generating a software test case based on execution of the large language model on the received descriptions, and displaying the software test case via the user interface.”)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 9 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Sen in view of Kadirvel et al. (U.S. Publication No. 2023/0350791 A1), hereinafter referred to as Kadirvel.
Regarding Claim 9,
Sen discloses, generating the second data structure as shown above with providing sequences of steps to the source of the request to generate the given test scenario (See FIG. 5B and paragraph [0084]; “In response, the GenAI model 524 may generate a sequence of steps ... Here, the steps may be determined by the GenAI model 524 based on historical software tests that have had similar requirements that the GenAI model has learned from. In this example, the steps … are displayed on the user interface 512, where the user of the user device can view the steps and provide feedback or accept the steps. Each step is another step during the test of the software program.”)
Sen does not but Kadirvel discloses, “determining a ranking of the two or more different sequences of test steps and providing the determined ranking of the two or more different sequences of test steps …” (Kadirvel [0020]; “In accordance with one or more embodiments, failure prediction model 114 can be trained using training data and a machine learning algorithm to determine a failure propensity score 116 for a test case using model input generated by model input generator 106 using execution history 102 and change history 104 information corresponding to the test case. The failure propensity score generated by failure prediction model 114 for each of a number of test cases can be used by test case prioritizer 118 to order (rank, sort, etc.) the test cases. Test case scheduler 120 can determine a testing schedule for the test cases in accordance with the order determined by test case prioritizer 118.”)
Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Sen by adopting the teaching Kadirvel to obtain better quality software with less time-consuming techniques (Kadirvel [0001], [0031]). With the common goal to improve the existing issue where “Software testing and certification can be time consuming and can cause delays in making a software application available to users” (Kadirvel [0001]), with a resulting solution where “testing can provide feedback which can be used to update the model to improve the model's accuracy and precision” (Kadirvel [0031]).
Regarding Claim 10,
Sen in view of Kadirvel teaches the apparatus of claim 9 as stated above.
Sen does not but Kadirvel further teaches, “wherein the ranking of the two or more different sequences of test steps is determined based at least in part on frequencies of use of test steps in the two or more different sequences of test steps in a set of one or more existing test scenarios of a test management environment.” (Kadirvel [0025]; “In accordance with one or more embodiments, change history encoder 110 can retrieve change history information from change history 104 for each test case for which a failure propensity score is to be determined by failure prediction model 114. By way of a non-limiting example, the change history information retrieved for a given test case can indicate each file that has been committed to during a determined time period. By way of a non-limiting example, the time period can be determined using a last-release date (e.g., date of the last release) associated with the test case and the build date of the test case.”; Kadirvel [0033]; “Turning to model training, embodiments of the present disclosure can map historical execution information with change history information, and generate training data using the mapping. In accordance with one or more embodiments of the present disclosure, the training data can include a code issue indicator determined using the historical execution information.”)
The historical execution information may be seen as the frequencies of use. The frequency of use for the programmer in this case would be the number of changes to the source code or the number of executions (Fig. 4, element 412; [0033]), as a result affecting the ranking of a test case negatively when the new resulting outcome caused by source code changes is not as expected [0025].)
Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Sen by adopting the teaching Kadirvel to obtain better quality software with less time-consuming techniques (Kadirvel [0001], [0031]). With the common goal to improve the existing issue where “Software testing and certification can be time consuming and can cause delays in making a software application available to users” (Kadirvel [0001]), with a resulting solution where “testing can provide feedback which can be used to update the model to improve the model's accuracy and precision” (Kadirvel [0031]).
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Sen in view of Rozen et al. (U.S. Publication No. 2023/0196018 A1, hereinafter Rozen).
Regarding Claim 21:
Sen discloses “The apparatus of claim 1 wherein processing the first data structure utilizing the machine learning model comprises utilizing ” ([0063]; “The GenAI model 322 may learn mappings/connections between requirements associated with a software test and the components included within the software test case and can thus create software test cases from a description of the requirements of the software test case”; [0049]; “a GenAI model 120 trained to generate software test cases from input data … the GenAI model 120 may communicate with the test execution service 130”; [0048]; “the test execution service 130 is coupled to a framework connector 140 (software program), which manages the execution of the software tests via a plurality of different frameworks … the framework connector can connect the test execution service 130 to the proper testing framework … based on a user request or in an automated way”; [0084]; “the GenAI model 524 may generate a sequence of steps … based on historical software tests that have had similar requirements that the GenAI model has learned from … the user of the user device can view the steps and provide feedback or accept the steps”;
[0072]; “Prompt engineering is the process of structuring sentences (prompts) so that the GenAI model understands them”; [0076]; “submitting prompts to the GenAI model … the ordering of the prompts and the follow-up questions may differ depending on the answers given during the previous prompt or prompts. The content within the prompts and the ordering of the prompts can cause the GenAI model 422 to generate software tests, automation scripts, source code, or the like”; [0071]; “According to various embodiments, the GenAI model described herein may be trained based on custom-defined prompts designed to draw out specific attributes associated with a software test, automation script, vulnerability, or the like”; [0081]; “the host platform 520 also hosts a generative artificial intelligence (GenAI) model 524 capable of generating a software test based on inputs received via the user interface 512”.).
Sen does not explicitly disclose however Rozen discloses, “few-shot prompting” ([0043]; “As noted above, a few-shot prompt is a structured text containing a descriptive introduction to the task, a few formatted examples, and a partial pattern that the model will tend to continue as another example”; [0042]; “FIG. 3 illustrates a typical few-shot prompt for toxic generation. The language model picks the pattern from the examples and tends to continue the text by generating a sequence related to the targeted community”.).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Sen to incorporate the teaching of few-shot prompting in Rozen to achieve the similar goal of improving the quality of a language model and the final output. Cited from the examined case’s specification, in lines 16-20 of page 14, “In some embodiments, a few-shot prompting approach is utilized for fine-tuning the LLM 307 … This allows the LLM 307 to focus on a desired result and increase the precision of the final output”. Similar to Rozen, in paragraph [0120]; “The method may further comprise fine-tuning the classification prompt for the language model … the selected combination having an optimal separation of probabilities”; [0112]; “the few-shot prompt is enriched at each bootstrapping iteration with the new example generated by the language model to generate increasingly robust examples”; [0005]; “Thus, it is desirable to provide or control a language model that produces synthetic texts, without toxicity”; [0106]; “to deal with toxicity, likely present in the data used to train the model, by working on output of the model (not the model) to produce a controlled output. The methods provide self-healing in text generation based on language models by toxic example generation”; [0030]; “self-healing using a large language model”.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Beza D Nigatu whose telephone number is (571)272-9643. The examiner can normally be reached Monday - Friday 7:30am-3:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hyung Sough can be reached at (571) 272-6799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BEZA D NIGATU/Examiner, Art Unit 2192
/S. Sough/SPE, Art Unit 2192
Prosecution Insights