Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This office action is in response to the application filed on 06/17/2024. Claims 1-13 and 15-21 are pending in this application. Claims 1 and 15 are independent claims. Claims 14 and 22-31 were cancelled.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. CN 202410233321, filed on February 29, 2024. Foreign priority information is presented in the application data sheet under 37 CFR 1.76.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1 -13 and 15-21 are rejected under 35 U.S.C. 101 because the claimed invention recites a judicial exception, is directed to that judicial exception, an abstract idea, it has not been integrated into a practical application, and the claims do not recite significantly more than the judicial exception.
Regarding claims 1 and 15, the limitations “segmenting the original SOW and obtaining a plurality of segmented SOWs;” and “generating target test cases of the software product to be tested according to the plurality of segmented SOWs, the product documentation and the target historical test cases” as drafted, are functions that, under its broadest reasonable interpretation, recite the abstract idea of a mental process. The limitations encompass a human mind carrying out the function through observation, evaluation judgment and/or opinion, or even with the aid of pen and paper. Thus, these limitations recite and fall within the “Mental Processes” grouping of abstract ideas under Prong 1.
Under Prong 2, this judicial exception is not integrated into a practical application. The additional elements “obtaining an original statement of work (SOW) of a target project and product information of a software product to be tested associated with the target project;” and “obtaining product documentation and target historical test cases of the software product to be tested according to the product information” do nothing more than add insignificant extra solution activity to the judicial exception of merely gathering data/information. Accordingly, the additional elements do not integrate the recited judicial exception into a practical application, and the claims are therefore directed to the judicial exception. See MPEP 2106.05(g).
Under Step 2B, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “obtaining an original statement of work (SOW) of a target project and product information of a software product to be tested associated with the target project;” and “obtaining product documentation and target historical test cases of the software product to be tested according to the product information,” the courts have identified merely gathering data/information is well-understood, routine and conventional activity. See MPEP 2106.05(d). Therefore, mere data gathering does not amount to significantly more, thus, it cannot provide an inventive concept. Accordingly, claims 1 and 15 are not patent eligible under 35 USC 101.
Claim 15 recites further additional elements “at least one processor” and “a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor and the at least one processor is configured to.” These additional elements are recited at a high level of generality such that they amount to no more than mere instructions to apply the exception using generic computer, and/or generic computer components. See MPEP 2106.05(f). Therefore, the additional elements recited in claim 15 do not integrate the judicial exception into a practical application under prong 2, nor amount to significantly more under step 2B.
Regarding claims 2, 11, and 16, the “generating” limitations recite additional mental process under prong 1, & the “obtaining” is mere data gathering, which is neither a practical application under prong 2, nor amount to significantly more under step 2B for the same reasons given for the rejection of claim 1.
Regarding claims 3, 8, 12, and 17 the “obtaining” limitations recited in the claims are mere data gathering, which is neither a practical application under prong 2, nor amount to significantly more under step 2B for the same reasons given for the rejection of claim 1.
Regarding claims 4 and 18, the additional elements “extracting” and “obtaining” are mere data gathering, which is neither a practical application under prong 2, nor amount to significantly more under step 2B for the same reasons given for the rejection of claim 1.
Regarding claims 5 and 19, the “matching” and “determining” limitations recite additional mental process under prong 1.
Regarding claims 6, 13, and 20, the “determining” limitations recite additional mental process under prong 1, and “obtaining” is mere data gathering, which is neither a practical application under prong 2, nor amount to significantly more under step 2B for the same reasons given for the rejection of claim 1
Regarding claim 7, the “converting” limitation recites additional mental process under prong 1, and “obtaining” is mere data gathering, which is neither a practical application under prong 2, nor amount to significantly more under step 2B for the same reasons given for the rejection of claim 1.
Regarding claim 9, the “segmenting” limitation recites additional mental process under prong 1, and “extracting” and “obtaining” are mere data gathering, which is neither a practical application under prong 2, nor amount to significantly more under step 2B for the same reasons given for the rejection of claim 1.
Regarding claims 10 and 21, the “generating” and “determining” limitations recite additional mental process under prong 1, and “obtaining” is mere data gathering, which is neither a practical application under prong 2, nor amount to significantly more under step 2B for the same reasons given for the rejection of claim 1.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Khafizov et al. (US20230289282) in view of Hiaqun et al. (CN115757124).
Regarding claim 1, Khafizov teaches: product information of a software product to be tested associated with the target project; ([0023] “According to an exemplary embodiment, test repository 215 may store data/information in a manner that correlates or links a requirement text with these other exemplary types of data/information (e.g., acceptable requirement, a test case, a test script, a test result, a test value, etc.” [0052] “When it is determined that the confidence threshold value is satisfied (block 440—YES), network device 107 may generate a test case and/or a test script (block 445). For example, network device 107 may generate a procedure, which may include a pre-condition, test step(s), test data, and an expected result. The test case may include other information, description, and/or data (e.g., test identifier, test name, actual result, priority, type of test case, bug identifier, etc.). Additionally, or alternatively, network device 107 may generate a test script, as described herein.”) Khafizov teaches test related information which corresponds to the claimed product information because it provides information used to test the software product. The information used to generate the test case is associated with a project, therefore, corresponding to the claimed requirements that the product information is associated with the project.
obtaining product documentation and target historical test cases of the software product to be tested according to the product information; ([0009] “Typically, before tests cases are generated, a person will specify the requirements of the test. For example, the requirements may relate to functionality requirements, performance requirements, and/or another type of testable criterion.” [0010] “According to exemplary embodiments, a test case generation service is described.” [0040] “Referring to FIG. 4A, in block 405, network device 107 may receive requirements. For example, the requirements may relate to a desired test case and/or test script to be generated.” [0023] “Test repository 215 may store information relating to the test case generation service. For example, test repository 215 may store requirements, acceptable requirements, test cases, test scripts, test results, test values, test runtimes, and/or other data (e.g., test domains, test values, runtimes, lab profiles, deadlines, lab profiles, and the like). According to an exemplary embodiment, test repository 215 may store data/information in a manner that correlates or links a requirement text with these other exemplary types of data/information (e.g., acceptable requirement, a test case, a test script, a test result, a test value, etc.”) Khafizov teaches a test repository containing product documentation and historical test cases along with requirements and other test related information. The product documentation provides information about the product and its requirements, while the historical test cases correspond to test cases previously used for the software product. Applicant spec states [0038] “The product documentation may include a requirement background of the product, an implementation mode of requirements, a corresponding interface interaction, etc. The target historical test cases may refer to test cases used in previous tests of the software product to be tested.” Therefore, Khafizov teaches obtaining product specific documentation and target historical test cases based on information about the product, which corresponds with the claimed product documentation and target historical test cases.
the product documentation and the target historical test cases. ([0009] “For example, the requirements may relate to functionality requirements, performance requirements, and/or another type of testable criterion.” [0045] “In block 415, based on the result of the analysis, network device 107 may determine whether the requirements are acceptable. For example, network device 107 may determine the acceptability of the requirements based on the applied criteria.” [0052] “When it is determined that the confidence threshold value is satisfied (block 440—YES), network device 107 may generate a test case and/or a test script (block 445). For example, network device 107 may generate a procedure, which may include a pre-condition, test step(s), test data, and an expected result. The test case may include other information, description, and/or data (e.g., test identifier, test name, actual result, priority, type of test case, bug identifier, etc.). Additionally, or alternatively, network device 107 may generate a test script, as described herein.”) Khafizov teaches generating test cases according to the product documentation and other information such as test cases.
Khafizov fails to disclose: (1) A method for generating test cases, comprising: obtaining an original statement of work (SOW) of a target project (2) segmenting the original SOW and obtaining a plurality of segmented SOWs; (3) generating target test cases of the software product to be tested according to the plurality of segmented SOWs
However, Haiqun teaches: (1) A method for generating test cases, comprising: obtaining an original statement of work (SOW) of a target project ([0006] “This invention overcomes one of the shortcomings of the prior art by providing a test case generation method based on neural networks.” [0007-0008] “According to one aspect of this disclosure, a test case generation method based on neural networks is proposed, the method comprising: Collect historical project documents and use them as raw data” [0012] “In one possible implementation, the raw data includes requirements documents, design documents, and test cases.”) Haiqun teaches collecting project documents, including requirements documents as raw data for test case generation. The requirements documents provide the requirements of the project and therefore correspond to the claimed SOW, which the applicant spec describes as [0031] “a narrative description of a product or service to be provided by the project.” Therefore, Haiqun teaches obtaining the claimed SOW through its collection of project requirements documents.
(2) segmenting the original SOW and obtaining a plurality of segmented SOWs ([0047] “extracting keywords from the requirement document and the design document by using a word segmentation tool to obtain a keyword set; labeling, stepping and hot-reading coding processing are carried out on the test cases to obtain a test case label set; and constructing a training data set of the test case prediction model based on the keyword set and the test case label set. The keyword extraction may include keyword term extraction, test step extraction, and the like, to obtain a keyword sample and a test case step identifier.”) Haiqun teaches a test case generation method where the original data is broken down using a word segmentation tool. Applicant spec refers to the SOW as [0031] “a narrative description of a product or service to be provided by the project” which corresponds to the original data taught by Haiqun which consists of the requirement document and a person of ordinary skill in the art can see that as a statement of work (SOW). Furthermore, once the segmentation tool extracts keywords from the original data, a training data set is constructing which consists of the broken-down pieces of data that corresponds to a plurality of segmented SOWs.
(3) generating target test cases of the software product to be tested according to the plurality of segmented SOWs ([0014] “Use word segmentation tools to extract keywords from requirements documents and design documents to obtain a keyword set” [0048] “Keyword sets are obtained by extracting keywords from requirement documents and design documents using word segmentation tools; test cases are tagged, broken down into steps, and encoded using hot-read encoding to obtain test case tag sets; and training datasets for the test case prediction model are constructed based on the keyword sets and test case tag sets. Keyword extraction can include key term extraction, test step extraction, etc., to obtain keyword samples and test case step identifiers.” [0049] “Step S3: Input the training dataset into the neural network-based test case prediction model and output predicted test cases.”) Haiqun teaches a system where a requirement document is segmented to extract relevant information and generate test cases. This corresponds to generating target test cases according to the plurality of segmented SOWs because the segmented requirements provide the basis for producing and predicting the test cases.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of claimed invention to combine the teachings of Khafizov with Hiaqun. Doing so would result in a test case generation method that breaks down the original statement of work into sections and generates test cases based off the segments, the product information, and previous related test cases. Making this modification would result in improved coverage of each requirement, higher quality test cases, and traceability where each generated test case can be linked to a specific section of the SOW.
Regarding claim 15, it is an electronic device claim having similar limitations cited in the rejection of claim 1. Therefore, claim 16 is rejected under the same rationale as claim 1 above.
Claims 2 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Khafizov et al. (US20230289282) in view of Hiaqun et al. (CN115757124) and in further view of Raman et al. (US10073763B1).
Regarding claim 2, Khafizov fails to disclose: (1) generating a first prompt message corresponding to each segmented SOW according to each segmented SOW, the product documentation and the target historical test cases, wherein the first prompt message is used to instruct a first large model to perform a test case generation task (2) wherein the first prompt message is used to instruct a first large model to perform a test case generation task; generating a test case corresponding to each segmented SOW by processing the first prompt message using the first large model (3) generating a test case corresponding to each segmented SOW by processing the first prompt message using the first large model; and obtaining the target test cases according to test cases corresponding to the plurality of segmented SOWs
However, Raman teaches: (1) generating a first prompt message corresponding to each segmented SOW according to each segmented SOW, the product documentation and the target historical test cases, wherein the first prompt message is used to instruct a first large model to perform a test case generation task; ([0012] “Requirements documentation (e.g., business requirements, functional requirements, use cases, user stories, and so forth) captures, for example, information related to the business process(es) that will be supported by the software” [0012] “extracting terminologies from requirement documents; classifying the extracted terminologies into categories based on a corpus of known terms; and generating process maps based on the classified terminologies.” [0014] “determining a script template based on the processing and a selected automation tool; applying, based on the processing, the script template to the intended interaction and the correlated object to generate an automated testing script for the selected automating tool; and assigning the generated automated testing script to the one of the test cases.” [0037] “The functional scenarios may be used by the automation accelerator engine 122 to generate automation test scripts. The non-functional scenarios may be employed by a testing team(s) to generate test cases specific to their respective areas, such as performance, security, and architectural testing. In some implementations, the test scenario map builder 290 includes a non-functional attribute builder that scans the requirements document and extract requirements that are likely to have performance and scalability requirements.” [0051] “Based on the AI model, the script generator module 340, determines the action(s) to perform the determined intent to the correlated objects in the respective page of the UI being tested.” Raman teaches a system where a requirements document is extracted into sections. Additionally, Raman teaches a script generator module which is based off of an AI model that generates a testing script to instruct the AI model the intended interaction to perform. Furthermore, the test cases are generated based off of the requirement documentation, previous cases, other functional requirements of the product, and more. Applicant spec states [0077] “For example, the first prompt message is "generating 5 test cases by with reference to the target historical test cases and in combination with information of requirements to be implemented as described in the segmented SOW and the product documentation.” Which corresponds to the testing script taught by Raman that is generated to model the intended interaction with the AI model. A person of ordinary skill in the art can identify a statement of work as a business requirement document.
(2) generating a test case corresponding to each segmented SOW by processing the first prompt message using the first large model ([0037] “The functional scenarios may be used by the automation accelerator engine 122 to generate automation test scripts. The non-functional scenarios may be employed by a testing team(s) to generate test cases specific to their respective areas, such as performance, security, and architectural testing. In some implementations, the test scenario map builder 290 includes a non-functional attribute builder that scans the requirements document and extract requirements that are likely to have performance and scalability requirements.” [0051] “Based on the AI model, the script generator module 340, determines the action(s) to perform the determined intent to the correlated objects in the respective page of the UI being tested.”) Raman teaches a system where a requirements document is scanned, and data is extracted and categorized. Additionally, Raman teaches a script generator module based off of an AI model to generate a test script which corresponds to the first prompt message that utilizes a large model to generate a test case.
(3) obtaining the target test cases according to test cases corresponding to the plurality of segmented SOWs. ([0003] “Testing of software applications may include creating test cases based on requirements and then executing the test cases through, for example, a test script to detect defects. Test cases may be automated using commercial and open source tools to reduce execution time. For example, a regression test suite is a set of test cases, often written in the form of a script, designed to ensure that each of these functions remain accurate” [0026] “The test scenario and process map extractor engine 121 scans the requirements document and creates the high level test scenarios and process maps. The automation accelerator engine 122 analyzes manual test cases, extracts the intent, and converts the intent into executable automated scripts. The test suite analyzer engine 123 analyzes test suite and groups contextually similar test cases into clusters based on contextual distance.”) Raman teaches a system where a requirement document, which corresponds to an SOW being scanned in and broken down to generate test cases. Furthermore, a test suite analyzer engine tests the cases to ensure the accuracy of them according to the requirements of the software being tested. Doing so ensures that the test cases that are created are accurate and efficient and result in being the target test case.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of generating test cases based off of a software product’s data as taught by Khafizov and Hiaqun with the teachings of Raman to incorporate a large model like AI for generating test cases. Doing so would accelerate the process of generating test cases and expand the test coverage as well.
Regarding claim 16, it is an electronic device claim having similar limitations cited in the rejection of claim 2. Therefore, claim 16 is rejected under the same rationale as claim 2 above.
Claims 3, 4, 17, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Khafizov et al. (US20230289282) in view of Hiaqun et al. (CN115757124) and in further view of Jia et al. (US20220318275).
Regarding claim 3, Khafizov fails to disclose: (1) obtaining a knowledge graph, wherein the knowledge graph comprises relations among software products, product documentation and historical test cases (2) obtaining the product documentation and the target historical test cases by retrieving from the knowledge graph according to the product information.
However, Jia teaches: (1) obtaining a knowledge graph, wherein the knowledge graph comprises relations among software products, product documentation and historical test cases ([0019] “The knowledge graph is essentially a semantic network, a graph-based data structure composed of nodes and edges. In the knowledge graph, each node represents an entity that exists in the real world, and each edge is a relationship between the entities. Traditionally, the knowledge graph is a relational network obtained by connecting all different types of information together. The knowledge graph provides the ability to analyze problems from the perspective of “relationship.” [0064] “The knowledge graph displays the key information in the target search result and the relationship between the key information.”) The knowledge graph introduced by Jia consists of nodes and connects pieces of all different types of information together similar to the knowledge graph taught by the applicant.
(2) obtaining the product documentation and the target historical test cases by retrieving from the knowledge graph according to the product information. ([0024] “The query statement may be a text statement directly input by the user and used to obtain a search result, or may be a statement extracted from data such as an audio and an image uploaded by the user, which is not limited in the disclosure.” [0086] “The third determining module 440 is configured to determine a knowledge graph corresponding to the target search result based on a first structured data set corresponding to the target search result.”) Information that a user inputs is obtained to generate a search result, the data in this teaching by Jia corresponds to the product documentation and test cases in the knowledge graph.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Jia to the teachings of Khafizov and Haiqun to have a test case generation system that utilizes a structured graph to hold and connect all relevant pieces of information to ensure that all aspects of a product are covered and tested.
Regarding claim 4, Khafizov fails to disclose: (1) extracting a first keyword from the original SOW; (2) obtaining the product documentation and the target historical test cases by retrieving from the knowledge graph according to the product information and the first keyword.
However, Hiaqun teaches: (1) extracting a first keyword from the original SOW; ([0013] “extracting keywords from the requirement document and the design document by using a word segmentation tool to obtain a keyword set”) Hiaqun takes in a requirement document which corresponds to the SOW and extracts keywords from it by utilizing a word segmentation tool.
Furthermore, Jia teaches: (2) obtaining the product documentation and the target historical test cases by retrieving from the knowledge graph according to the product information and the first keyword. ([0004] “The disclosure provides a search method. The method includes: obtaining a query statement; determining a correlation between the query statement and a candidate result by matching the query statement with a first structured data set corresponding to the candidate result in a search database, in which the first structured data set is generated by performing information extraction on the candidate result by a structured information extraction model generated by training; and determining, based on the correlation, a target search result corresponding to the query statement.” [0024] “The query statement may be a text statement directly input by the user and used to obtain a search result, or may be a statement extracted from data such as an audio and an image uploaded by the user, which is not limited in the disclosure.” [0026] “Each piece of first structured data set is generated by performing information extraction on the corresponding candidate result by a structured information extraction model generated by training.” [0027] “In some embodiments, key character segmentations contained in the query statement can be obtained first, and then each key character segmentation can be matched with each piece of first structured data in the first structured data set corresponding to the candidate result, to determine the correlation between the query statement and the candidate result based on a matching degree between each key character segmentation and each piece of first structured data.”) Jia teaches a search system where an input test or data is obtained from the user. That data is then broken down into segments and key characters are extracted and matched accordingly. A knowledge graph is the structured system that holds the information of the data and the relationships between the data as well.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to add the teachings of Jia to the teachings of Khafizov and Hiaqun to achieve a method that consists of a knowledge graph to store product data because it provides a structured way to represent product information and relationships. This enables the generation of more accurate and comprehensive test cases based on the product’s features.
Regarding claim 17, it is an electronic device claim having similar limitations cited in the rejection of claim 3. Therefore, claim 17 is rejected under the same rationale as claim 3 above.
Regarding claim 18, it is an electronic device claim having similar limitations cited in the rejection of claim 4. Therefore, claim 18 is rejected under the same rationale as claim 4 above.
Claims 5 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Khafizov et al. (US20230289282) in view of Hiaqun et al. (CN115757124) in view of Jia et al. (US20220318275) in view of Vasavan et al. (US20230138180) and in further view of Xiaoting (CN116881122).
Regarding claim 5, Khafizov fails to disclose: (1) matching the product information with nodes in the knowledge graph to determine a first node that matches with the product information, wherein the first node represents the software product to be tested (2) determining, according to a first relation between the software product and the product documentation, a second node whose relation with the first node is the first relation from the knowledge graph, wherein the second node represents product documentation of the software product to be tested (3) determining, according to a second relation between the software product and a test case, a third node whose relation with the first node is the second relation from the knowledge graph, wherein the third node represents a historical test case of the software product to be tested (4) matching the first keyword with a second keyword of each historical test case to obtain a keyword match result between the first keyword and the second keyword of each historical test case (5) determining the target historical test cases from the historical test cases according to the keyword match result corresponding to each historical test case
However, Vasavan teaches: (1) matching the product information with nodes in the knowledge graph to determine a first node that matches with the product information, wherein the first node represents the software product to be tested ([0013] “The selection of the software product may include information identifying a software product to be tested. The test input data may identify inputs of a test case for the software product. For example, the test input data may include information identifying a functionality of the software product, a set of parameters under which the functionality is to be tested, and/or the like.” [0047] “For example, the machine learning system may identify a feature set (e.g., one or more features and/or feature values) by extracting the feature set from structured data, by performing natural language processing to extract the feature set from unstructured data, and/or by receiving input from an operator.” [0048] “As an example, a feature set for a set of observations may include a first feature of test data, a second feature of a software product, a third feature of test input data, and so on.”) Vasavan teaches that the selection of a software product may consist of information regarding the software product to be tested like the functionality of the product which corresponds to the product information as taught by the applicant. A person of ordinary skill in the art can identify that a knowledge graph is a structured data representation and therefore corresponds to the knowledge tree and the feature set corresponds to nodes where each feature set contains different types of values/data. In this example, the first feature set corresponds to the first node which represents the test data, in other words the software product since that is what is being tested.
(2) determining, according to a first relation between the software product and the product documentation, a second node whose relation with the first node is the first relation from the knowledge graph, wherein the second node represents product documentation of the software product to be tested ([0047] “In some implementations, the machine learning system may determine variables for a set of observations and/or variable values for a specific observation based on input received from the user device.” [0048] “As an example, a feature set for a set of observations may include a first feature of test data, a second feature of a software product, a third feature of test input data, and so on. As shown, for a first observation, the first feature may have a value of test data 1, the second feature may have a value of software product 1, the third feature may have a value of test input data 1, and so on. ”) The second feature set of a software product corresponds to the second node which represents the product documentation. The second feature set taught by Vasavan contains values and data specific to the software product. Product documentation, like requirements and more, would be found in this feature set since they are features and values of the software.
(5) determining the target historical test cases from the historical test cases according to the keyword match result corresponding to each historical test case ([0027] “Alternatively, and/or additionally, the testing system 115 may obtain the test case flow based on accessing a data structure storing test case flows associated with software products and/or functionalities of software products. In some implementations, the testing system 115 may determine the test case flow based on user input.) The testing system taught by Vasavan matches test cases from a data structure that are associated/similar to the software product that a test case is being generated for. Additionally, the testing system then determines a target test case is determined and can be determined based on user input.
Additionally, Xiaoting teaches: (3) determining, according to a second relation between the software product and a test case, a third node whose relation with the first node is the second relation from the knowledge graph, wherein the third node represents a historical test case of the software product to be tested ([0020] “acquiring a historical test case set and product rule information; the historical test case set comprises text information of historical test cases” [0112] “The historical test case set is a test case of a historical record, and comprises text information of the historical test case and mainly comprises the test case and text information of a test task corresponding to the test case.” [0113] “The product rule information is rule information of products in the software development process. Specifically, if the software of the mobile banking is developed, the product rule information includes, but is not limited to, all function information and guidance information corresponding to the functions in the software.” [0129] “In the embodiment, the test case knowledge graph is formed based on the historical test cases, the graph structure sample and the corresponding node information are obtained from the test case knowledge graph, and parameters of the case prediction model before training are adjusted until the case prediction model before training converges, so that the case prediction model after training is suitable for generating the test cases of the test tasks in the real environment”) Xiaoting teaches a system where a historical test case and the software product information is obtained. Additionally, Xiaoting teaches a graph structure, specifically a test case knowledge graph which consists of historical test cases and the corresponding node information.
Furthermore, Jia teaches: (4) matching the first keyword with a second keyword of each historical test case to obtain a keyword match result between the first keyword and the second keyword of each historical test case ([0027] “In some embodiments, key character segmentations contained in the query statement can be obtained first, and then each key character segmentation can be matched with each piece of first structured data in the first structured data set corresponding to the candidate result, to determine the correlation between the query statement and the candidate result based on a matching degree between each key character segmentation and each piece of first structured data.”) Jia teaches a system where once data is broken down into smaller sections, key words or characters are extracted and matched with the existing data set to determine a match.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Vasavan and Jia with the previous teachings to have a system that can generate test cases by matching all parts of the software product to one another and ensuring all aspects of the product get covered and tested accordingly.
Regarding claim 19, it is an electronic device claim having similar limitations cited in the rejection of claim 5. Therefore, claim 19 is rejected under the same rationale as claim 3 above.
Claims 6 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Khafizov et al. (US20230289282) in view of Hiaqun et al. (CN115757124) in view of Jia et al. (US20220318275) in view of Vasavan et al. (US20230138180) in view of Xiaoting (CN116881122) and in further view of Hengxin et al. (CN114546737).
Regarding claim 6, Khafizov fails to disclose: (1) obtaining a type label of each historical test case from the knowledge graph; (2) determining historical test cases whose keyword match results are matching and whose type labels are a preset label as the target historical test cases.
However, Hengxin teaches: (1) obtaining a type label of each historical test case from the knowledge graph; ([0007] “Optionally, in a second implementation manner of the first aspect of the present invention, the text classifier is obtained by training according to the following steps: acquiring a historical test case, and extracting text contents of the historical test case; performing word segmentation processing on the text content of the historical test case to obtain text word segmentation of the historical test case, and extracting second text features of the text word segmentation of the historical test case; labeling the second text features to obtain type labels corresponding to the second text features, wherein the type labels correspond to the test types one by one; inputting the second text feature into a preset initial classifier to obtain a prediction label of the second text feature; and adjusting the initial classifier according to the type label and the prediction label of the second text characteristic to obtain a text classifier.”) Hengxin teaches a system where first a historical test case is acquired and then a type label is obtained by determining the class/category of the dataset.
(2) determining historical test cases whose keyword match results are matching and whose type labels are a preset label as the target historical test cases. ([0008] “Optionally, in a third implementation manner of the first aspect of the present invention, the matching, according to the test type, the test case with all test templates in a preset test template library, and determining the test template corresponding to the test case includes: matching the test case with all test templates in a preset test template library according to the test types, wherein all test templates in the test template library have corresponding test types; judging whether a test template with the same test type as the test case exists in the test template library or not; and if so, determining the test template with the same test type as the test case as the test template corresponding to the test case.”) Hengxin teaches a testing method where he determines a preset label corresponding to the current test case’s type.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the previous teachings with the teachings of Hengxin to improve the relevance and accuracy of the historical test cases by further filtering/categorizing the results.
Regarding claim 20, it is an electronic device claim having similar limitations cited in the rejection of claim 6. Therefore, claim 20 is rejected under the same rationale as claim 6 above.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Khafizov et al. (US20230289282) in view of Hiaqun et al. (CN115757124) and in further view of Cragun et al. (US20140146053)
Khafizov does not teach: (1) in response to the original SOW containing an image, converting the image into a text description to obtain a new SOW; (2) obtaining the plurality of segmented SOWs by segmenting the new SOW.
However, Cragun teaches: (1) in response to the original SOW containing an image, converting the image into a text description to obtain a new SOW; ([0041] “The client devices 110-114 themselves may likewise provide content, such as in the form of electronic documents, which may include images, text, and the like.” [0042] “The alternative text generation engine may be initiated to generate alternative text for images in a web page or electronic document in response to receiving the web page or electronic document, for example.” [0015] “This automatic generation of textual descriptions of images may be performed for images that do not already have a textual description associated with them or may be used to replace an existing textual description with another automatically generated textual description. The automatically generated text is referred to herein as an “alternative text” representation for the image.”) Cragun teaches a system that generates alternative descriptions for images. A document is first uploaded and can consist of an image or text corresponding to an SOW which is a requirements document that can contain an image. Then a text generation engine converts the image into alternative text and obtaining and referred to as “alternative text” once generated.
Additionally, Haiqun teaches: (2) obtaining the plurality of segmented SOWs by segmenting the new SOW ([0003] “the original data such as the requirement document and the design document” [0031] “ The test case generation method based on the neural network has the advantages that historical project documents are collected and serve as original data; extracting keywords from the original data and labeling the test cases to obtain a training data set for constructing a test case prediction model” [0046] “ In an example, as shown in fig. 2, the obtaining a training data set for constructing a test case prediction model by performing keyword extraction and test case tagging on the raw data may include” [0047] “extracting keywords from the requirement document and the design document by using a word segmentation tool to obtain a keyword set; labeling, stepping and hot-reading coding processing are carried out on the test cases to obtain a test case label set; and constructing a training data set of the test case prediction model based on the keyword set and the test case label set. The keyword extraction may include keyword term extraction, test step extraction, and the like, to obtain a keyword sample and a test case step identifier.”) Haiqun teaches a test case generation method where the original data is broken down using a word segmentation tool. Applicant refers to the SOW as [0031] “a narrative description of a product or service to be provided by the project” which corresponds to the original data taught by Haiqun which consists of the requirement document and a person of ordinary skill in the art can see that as a statement of work (SOW). Furthermore, once the segmentation tool extracts keywords from the original data, a training data set is constructing which consists the broken-down pieces of data and that training data set is obtained in order to construct a test case.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Khafizov and Haiqun with the teachings of Cragun to create the claimed invention where if the SOW is an image, it gets converted into a text description and broken down into segments. Converting an image into text allows for an easier and cleaner search process. Additionally, breaking down the SOW into segments allows for a more precise extraction of specific requirements in order to generate test cases.
Claims 10 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Khafizov et al. (US20230289282) in view of Hiaqun et al. (CN115757124) in view of Jiacheng (CN115145812) and in further view of Jia et al. (US20220318275).
Khafizov does not teach: (1) generating a plurality of candidate test cases corresponding to each segmented SOW according to each segmented SOW, the product documentation, interface documentation , and target historical case information (2) obtaining a score for each candidate test case by performing a quality evaluation on each candidate test case; (3) determining a final test case for each segmented SOW from the plurality of candidate test cases according to the score of each candidate test case (4) obtaining the target test cases according to final test cases of the plurality of segmented SOWs.
However, Jiacheng teaches: (1) generating a plurality of candidate test cases corresponding to each segmented SOW according to each segmented SOW, the product documentation, interface documentation , and target historical case information ([0030] “The test case refers to the description of a specific software product for testing tasks, and the test scheme, method, technology and strategy are embodied. The content of the test cases may include test targets, test environments, input data, test steps, expected results, test scripts, etc., to ultimately form a document. Briefly, a test case may be considered a set of test inputs, execution conditions, and expected results tailored for a particular purpose to verify that the software under test meets predetermined requirements. The test case is one of methods for quantifying the test details, different types of software and different test cases.” [0046] “When the test cases corresponding to the interface metadata are generated, the parameters of different types of the interface metadata can be respectively processed, and then the processing results are combined to obtain the test cases corresponding to the interface metadata.”) Jiacheng teaches how test environments, input data, expected results, the description of software products, etc. all form a document that corresponds to the SOW which is a document that describes a project’s scope/requirements. Furthermore, test cases are generated according to that document.
(2) obtaining a score for each candidate test case by performing a quality evaluation on each candidate test case; ([0037] “Regarding how to scale the quality of test cases, the present disclosure may employ generic test case quality metrics. For example, the number of coverage lines/coverage of the test case to the tested content (such as the code of the tested software product) is used as a measurement standard of the test case; the larger the number of coverage lines/coverage, the more parts of the software product that the test case can test, the higher the quality of the test case. Based on this, the embodiment of the disclosure may preset a target coverage as a comparison criterion for determining whether the quality of the test case meets the requirement.”) Jiacheng teaches a system where the quality of test cases is evaluated, and the test case is rated for its target coverage.
Additionally, Jia teaches: (3) determining a final test case for each segmented SOW from the plurality of candidate test cases according to the score of each candidate test case ([0059] “Finally, according to the matching result corresponding to each piece of second structured data, the correlation between the query statement and the candidate result is determined.” [0060] ” In the embodiments of the disclosure, the second structured data in the query statement is matched with the first structured data of the same type, thereby shortening the matching time between the query statement and each candidate result, and further improving the efficiency of obtaining the target search result.” [0061] “In step S204, based on the correlation, a target search result corresponding to the query statement is determined.”) Jia teaches a final test case result being determined by matching the previous results that were obtained.
(4) obtaining the target test cases according to final test cases of the plurality of segmented SOWs. ([0074] “In the embodiments of the disclosure, each piece of the second structured data in the second structured data set corresponding to the query statement is matched with each piece of the first structured data corresponding to the candidate result, to obtain the target search result corresponding to the query statement. Finally, the target search result and the knowledge graph are displayed at the same time”) The structured data taught by Jia corresponds to the segmented SOW and that data is matched with the first set of results that correspond to the query statement to obtain the target search result.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Jiacheng and Jia to the previously taught teachings to obtain a method that scores the generated test cases to ensure the quality of test cases according to each segmented SOW to ensure all aspects of a product are being thoroughly tested.
Regarding claim 21, it is an electronic device claim having similar limitations cited in the rejection of claim 10. Therefore, claim 21 is rejected under the same rationale as claim 10 above.
Claims 11, 12, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Khafizov et al. (US20230289282) in view of Hiaqun et al. (CN115757124) in view of Jiacheng (CN115145812) in view of Jia et al. (US20220318275) and in further view of Xiaojia (CN113312261).
Regarding claim 11, Khafizov teaches: by processing the second prompt message using the second large model ([0010] “According to an exemplary embodiment, the test case generation service may include an automated test case generation based on a model, such as artificial intelligence (AI) and/or machine learning (ML) model, as described herein.” [0060] “The model profile may provide a threshold performance for a classification model to satisfy to be considered a candidate classification model for use in a test case generation procedure for a given requirement.”) Applicant states [0002] “The disclosure relates to a field of computer technology, especially to a field of artificial intelligence (Al) such as a large model and natural language processing, and more particular to method for generating test cases and an electronic device. [00192] “generate a second prompt message according to the plurality of candidate test cases and the original requirement information, in which the second prompt message is used to instruct a second large model to perform a test case quality evaluation task.” Khafizov teaches that a model such as AI is utilized for test case generation. Furthermore, a test case is generated based on a given requirement which corresponds to the second prompt message taught by the applicant which is generated through a model based off of test cases and the original requirements information.
Khafizov does not teach: (1) obtaining original requirement information corresponding to the plurality of candidate test cases (2) generating a second prompt message according to the plurality of candidate test cases and the original requirement information, wherein the second prompt message is used to instruct a second large model to perform a test case quality evaluation task (3) obtain the score for each candidate test case
However, Xiaojia teaches: (1) obtaining original requirement information corresponding to the plurality of candidate test cases ([0034] “step S10: acquiring case attribute information of the candidate test cases, and performing feature extraction on the case attribute information to acquire case feature information.” [0037] “It should be noted that the example attribute information includes an example number, a test module, a test title, a preset condition, input information, test operation information, expected output information, and the like, which is not limited in this embodiment.” [0043] “Further, in order to obtain multi-aspect case feature information, the obtaining of case attribute information of a candidate test case and feature extraction of the case attribute information are performed to obtain case feature information includes” [0044] “acquiring case attribute information of a candidate test case, extracting the case attribute information, acquiring input information, test operation information and expected output information of the candidate test case, performing content identification on the test operation information, determining associated case information, test scenario information, software function information and test tool information of the candidate test case according to a content identification result, searching import and export permission information corresponding to the candidate test case in a preset case permission table, and determining case characteristic information according to the input information, the expected output information, the associated case information, the test scenario information, the software function information, the test tool information and the import and export permission information.”) Applicant states [00132] “The original requirement information may be obtained from the segmented SOWs, from the product documentation, or by other means, which is not limited herein.” Xiaojia teaches a system where information is obtained of the test cases. Information can consist of expected outputs, titles, test operation information, etc. which corresponds to the applicant’s original requirement information which may include product documentation and how the product is expected to perform.
Additionally, Xiaojia teaches: (3) obtain the score for each candidate test case ([0008] “executing the candidate test case to obtain test information, and generating a dynamic evaluation score of the candidate test case according to the test information”) A score evaluation is generating and obtained from candidate test cases.
Furthermore, Jiacheng teaches: (2) generating a second prompt message according to the plurality of candidate test cases and the original requirement information, wherein the second prompt message is used to instruct a second large model to perform a test case quality evaluation task. ([0030] “Briefly, a test case may be considered a set of test inputs, execution conditions, and expected results tailored for a particular purpose to verify that the software under test meets predetermined requirements. The test case is one of methods for quantifying the test details, different types of software and different test cases.” [0104] ” It can be seen that, in the above case, if the actual coverage of the test case is greater than or equal to the preset target coverage, the value of L (y|f (X)) in the equation (4) is 0 or a positive number, which indicates that the quality of the test case meets the preset requirement; if the actual coverage of the test case is smaller than the preset target coverage, the value of L (Y|f (X)) in the equation (4) is a negative number, which indicates that the quality of the test case does not meet the preset requirement, and the larger the absolute value of L (Y|f (X)) is, the worse the quality of the test case is.”) Jiacheng teaches a system where values are generated once the second model performs a test case generation task to check the quality of the test case. Which corresponds to the second model that is for performing test case quality.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Jiacheng with Xiaojia and Khafizov to achieve a system that performs test case quality and scores the test cases again to ensure all requirements of the product are successfully covered.
Regarding claim 12, Khafizov teaches: obtaining reference requirement information and reference test cases of the software product to be tested ([0023] “Test repository 215 may store information relating to the test case generation service. For example, test repository 215 may store requirements, acceptable requirements, test cases, test scripts, test results, test values, test runtimes, and/or other data (e.g., test domains, test values, runtimes, lab profiles, deadlines, lab profiles, and the like). According to an exemplary embodiment, test repository 215 may store data/information in a manner that correlates or links a requirement text with these other exemplary types of data/information (e.g., acceptable requirement, a test case, a test script, a test result, a test value, etc.”) Applicant states [0038] “The product documentation may include a requirement background of the product, an implementation mode of requirements, a corresponding interface interaction, etc. The target historical test cases may refer to test cases used in previous tests of the software product to be tested.” Khafizov teaches a test repository which consists of multiple aspects of a product including the requirement information and other test cases. Additionally, the test repository may also store data/information that is relevant or connect the requirements text with other data/information.
Khafizov does not teach: obtaining the second large model by fine-tuning an original large model using the reference requirement information and the reference test cases
However, Jiacheng teaches: obtaining the second large model by fine-tuning an original large model using the reference requirement information and the reference test cases ([0093] “The manner in which test cases are generated is described above. After the test cases are generated, embodiments of the present disclosure may employ a discriminant model (also referred to as a discriminant) pair to determine the quality of the test cases and determine whether the quality of the test cases meets predetermined requirements.”) Jiacheng teaches that once test cases are generated, the system employs a model for performing a test case quality check to determine if the predetermined requirements are met. Applicant spec states [0130] “As a result, the second large model performs quality evaluation on the generated candidate test cases, which improves the accuracy of evaluation results and evaluation efficiency.” Which corresponds to the discriminant model taught.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the test case generation as previously taught by Khafizov to include the discriminant model of Jiacheng to achieve a system that would enable evaluation of the quality of generated test cases and therefore improving the reliability and effectiveness of the generated test case results.
Regarding claim 13, Khafizov teaches: determining the reference test cases from the plurality of historical test cases according to a generation mode label of each historical test case, and determining requirement information corresponding to the reference test cases as the reference requirement information. ([0023] “Test repository 215 may store information relating to the test case generation service. For example, test repository 215 may store requirements, acceptable requirements, test cases, test scripts, test results, test values, test runtimes, and/or other data” [0040] “Referring to FIG. 4A, in block 405, network device 107 may receive requirements. For example, the requirements may relate to a desired test case and/or test script to be generated.” [0044] “In block 410, network device 107 may analyze the requirements. For example, network device 107 may analyze multiple criteria that relate to the substance and form of the requirement text.”) Khafizov teaches a test repository that stores a plurality of test cases along with associated requirement information. The repository further stores the data such that requirement text is correlated/linked with the corresponding test case thereby providing requirement information corresponding to the stored test cases. For the desired test case, Khafizov teaches that there are corresponding requirements. A desired test case serves as the predefined test case against which testing is performed and therefore corresponds to the claimed reference test cases. A person of ordinary skill in the art would understand that the term “reference” simply denotes a selected historical test case. Additionally, in view of the specification, the recited “generation mode label” would have been understood as an identifier indicating the manner in which the historical test case was generated, either manually or automatically. Accordingly, determining the reference test case according to the generation mode label broadly encompasses selecting a historical test case based on its associated generation mode identifier.
However, Khafizov does not teach: (1) obtaining a knowledge graph, wherein the knowledge graph comprises relations among software products, product documentation and historical test case (2) obtaining a plurality of historical test cases of the software product to be tested from the knowledge graph according to the product information
However, Jia teaches: (1) obtaining a knowledge graph, wherein the knowledge graph comprises relations among software products, product documentation and historical test case ([0019] “The knowledge graph is essentially a semantic network, a graph-based data structure composed of nodes and edges. In the knowledge graph, each node represents an entity that exists in the real world, and each edge is a relationship between the entities. Traditionally, the knowledge graph is a relational network obtained by connecting all different types of information together. The knowledge graph provides the ability to analyze problems from the perspective of “relationship.” [0064] “The knowledge graph displays the key information in the target search result and the relationship between the key information.”) The knowledge graph introduced by Jia consists of nodes and connects pieces of all different types of information together similar to the knowledge graph taught by the applicant.
(2) obtaining a plurality of historical test cases of the software product to be tested from the knowledge graph according to the product information ([0024] “The query statement may be a text statement directly input by the user and used to obtain a search result, or may be a statement extracted from data such as an audio and an image uploaded by the user, which is not limited in the disclosure.”) Information that a user inputs is obtained to generate a search result, the data in this teaching corresponds to the product documentation and test cases.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine these teachings to create a system that generates test cases using the most relevant and up-to-date product information. By utilizing a large language model to perform this generation, the system can evaluate the quality of the generated test cases further improving their reliability and overall effectiveness.
Allowable Subject Matter
Claims 8-9 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 101, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
(US20240004911A1) Topic-based document segmentation: Teaches a system and method that segments a document into a set of sentences using a machine-learning model.
(US10073763B1) Touchless testing platform: Teaches a method and system for creating test cases through a machine learning algorithm.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Tubba Noor whose telephone number is 571-270-0803. The examiner can normally be reached on Monday-Friday from 8:00 AM to 5:00 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chat Do, can be reached at telephone number 571-272-3721. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TUBBA NOOR/Examiner, Art Unit 2193
/Chat C Do/ Supervisory Patent Examiner, Art Unit 2193
Prosecution Insights