DETAILED ACTION
This action is responsive to Remarks and Claim Amendments filed on September 08, 2026.
Claims 1-20 have been amended.
Claims 1-20 are pending and presented for examination.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner Notes
Examiner cites particular columns, paragraphs, figures and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
Response to Amendment
The objection to the specification for failure to include a Cross-Reference to Related Applications section is withdrawn in view of the amendment to the specification adding that section.
The objections to claims 1-20 for informalities are withdrawn in view of the amendments to the claims.
The rejection of claims 15, 16, and 19 under 35 U.S.C. 112(b) is withdrawn. Claim 15 now recites “a respective command tree map for each target command of a plurality of target commands in the target commands namespace”, which resolves the singular/plural ambiguity and the antecedent basis issue. Claim 16 now recites “a weight stored in the at least one respective child node of the respective command tree map” and “a full usage combination of the target command corresponding to the respective command tree map”, which resolves the antecedent basis issues. Claims 19 and 20 now recite “an information system” for the system under test as distinguished from “a command coverage assessment system”, which resolves the ambiguity as to which system is referenced. A new rejection of claim 15 under 35 U.S.C. 112(d) is set forth below.
The rejection of claims 1-20 under 35 U.S.C. 101 is withdrawn. The basis of the rejection of record was that the claims recited the abstract idea without reciting how the command tree map is built, traversed, structured in memory, queried, or transformed, such that the additional elements amounted to no more than obtaining data, applying the exception using generic computer components, and outputting a report. Amended claims 1 and 19-20 now recite that the command tree maps are created for each target command from full usage combinations, populated with weights, updated with hit ratios derived from parsing the test case log, and queried to identify a command combination having missing or weak coverage. Amended claims 1 and 19-20 further recite that the identified command combination is used to direct execution of an updated regression test case on the system, that the resulting further test case log is used to update the command tree maps with a further hit ratio, and that a determination is made from that further hit ratio and the assigned weight whether the identified coverage deficiency has been filled. The claims as a whole therefore no longer terminate in the output of a report. The analytical result governs a subsequent operation performed on the system under test, the result of that operation is written back into the claimed data structures, and the effect of that operation is verified against those same data structures. Under Step 2A Prong Two, these additional elements integrate the judicial exception into a practical application by reflecting the improvement to regression test coverage assessment set forth in the specification. See MPEP § 2106.05(a). Accordingly, the claims are eligible under 35 U.S.C. 101 and the analysis does not proceed to Step 2B.
The rejection of claims 1-10 and 17-20 under 35 U.S.C. 102(a)(1) as anticipated by Gopalakrishnan, and the rejection of claims 11-16 under 35 U.S.C. 103 over Gopalakrishnan in view of Lee-Smith, are withdrawn in view of the amendments to the claims. Lee-Smith is relied upon in a new ground of rejection of claim 14 set forth below. New grounds of rejection necessitated by the amendment are set forth below.
Response to Arguments
Applicant’s arguments filed September 8, 2026 with respect to the rejections of record have been fully considered and are persuasive in part.
Applicant’s arguments at pages 17-21 that Gopalakrishnan does not disclose obtaining target commands from one or more functional specifications, occurrence-based weights derived from functional specifications, zero weighting of full-usage combinations absent from the functional specifications, or normalized hit ratios, are persuasive. Gopalakrishnan extracts configurations from the software product under test rather than from a functional specification (Gopalakrishnan paragraph [0055]), and Gopalakrishnan does not disclose weighting. The rejection under 35 U.S.C. 102(a)(1) is accordingly withdrawn.
Applicant’s arguments at pages 22-23 that Lee-Smith does not cure these deficiencies are persuasive. Lee-Smith is no longer relied upon as to claims 11-13, 15, and 16; it is applied to claim 14 only, as set forth below.
Applicant’s arguments at pages 23-26 directed to the rejection under 35 U.S.C. 101 are persuasive to the extent set forth in the Response to Amendment above with respect to Step 2A Prong Two. Examiner notes for the record that Applicant’s argument that the amended claims “cannot reasonably be characterized as merely performing a mental process” by reason of the recited parsing of test case logs is not persuasive as to Step 2A Prong One. Parsing the test case log supplies the data upon which the recited comparison of weights against hit ratios operates; it does not perform that comparison, and it is therefore an additional element evaluated under Step 2A Prong Two rather than a limitation that removes the claim from the mental process grouping. See MPEP § 2106.05(g). The rejection is withdrawn on the basis of Step 2A Prong Two only.
Upon further search necessitated by the amendment, the amended subject matter is found to be unpatentable over the references newly applied below.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 15 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends.
Claim 15 was amended to depend from claim 1 and recites “creating a respective command tree map for each target command of a plurality of target commands in the target commands namespace, wherein each respective command tree map comprises full usage combinations of the target command, wherein each respective command tree map comprises a respective primary node and at least one respective child node.”
Claim 1, as amended, already recites “creating, for each target command of the plurality of target commands, a respective command tree map comprising full usage combinations of the target command, wherein each respective command tree map comprises the target command as an entrance, at least one primary node and at least one child node.”
The subject matter recited in claim 15 is therefore coextensive with subject matter already recited in claim 1, and claim 15 specifies no further limitation. Applicant is advised to cancel claim 15 or to amend it to recite subject matter that further limits claim 1. Claim 16, which depends from claim 15, is not rejected under 35 U.S.C. 112(d), as claim 16 recites the updating of a stored weight based on an occurrence of a full usage combination in the one or more functional specifications, which is a further limitation.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-5, 7-13, 15-17, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Gopalakrishnan et al. (US Pub. No. 2018/0107587 – hereinafter Gopalakrishnan) in view of Miranda et al., “An assessment of operational coverage as both an adequacy and a selection criterion for operational profile based testing,” Software Quality Journal, Vol. 26, No. 4, pp. 1571-1594, published online September 22, 2017 (hereinafter Miranda). With respect to claim 1 (Currently Amended), Gopalakrishnan teaches a method comprising:
executing at least one test case on a system (Gopalakrishnan discloses applying a test suite to a software product under test. See Gopalakrishnan paragraph [0027], (“the sequence of tests as defined by the Test Plan is applied to the code (Block 130)”). See paragraph [0035], (“After the line and function coverage are achieved, CCA is run to get the command coverage (Block 450)”). See paragraph [0045], (“the release engineering team runs the test suite on the software product code base”)) obtaining, by a command coverage assessment system, at least one test case log resulting from the execution of the at least one test case (Gopalakrishnan discloses a Command Coverage Analyzer (CCA), which reads on the claimed “command coverage assessment system”. See Gopalakrishnan paragraph [0022], (“The conditional command coverage analyzer (CCA) is a tool with an associated system and method that provide, among other things, a conditional coverage metric for software quality”). See paragraph [0049], (“a LUT or other storage mechanism may store a ‘golden log’ of results from test cases that have been firm to be correct, wrong, or other status of interest”). See paragraph [0050], (“configurations extracted from each test file in test suite database 610 are used to update (Block 630) the file, e.g. a LUT, created in Block 615. In one embodiment, the file updated is a golden log”). See paragraph [0051], (“a current set of results for a test suite applied to the software product under test are compared to a “golden” set of results”))
generating, by the command coverage assessment system, a command coverage report using one or more command tree maps, based on the at least one test case log (See Gopalakrishnan paragraph [0046], (“The CCA engine 570 takes these two inputs, and analyzes the overlap between the available permutations and combinations of configurations… and the tested set of configurations… The CCA engine 570 then generates a command coverage database 580”). See paragraph [0048], (“From command coverage database 580, command coverage reports 590 are generated”). See paragraph [0056], (“a report 760 is generated to identify the testing gaps based on the set of configurations”). See paragraph [0038], (“configuration extractor 520 forms the keys to a hash table data structure, which is created to keep track of occurrences of the software product configurations. In another embodiment, a tree structure is used, or a different type of data structure”). See paragraph [0049], (“a look up table (LUT) (Block 615) is created from this information, including the commands, options, arguments, and values that may be present in the software product under test. Other embodiments may use other mechanisms to store and interrelate this information, such as a graph, hash table, or tree”)), wherein generating the command coverage report comprises:
creating, for each target command of the plurality of target commands, a respective command tree map comprising full usage combinations of the target command, wherein each respective command tree map comprises the target command as an entrance, at least one primary node and at least one child node (See Gopalakrishnan paragraph [0022], (“The CCA may additionally extract configurations of a software product, produces permutations and combinations of these configurations”). See paragraph [0042], (“By testing numerous permutations and combinations configurations, such as various commands, options, arguments, and values, and validating the resultant architecture, desired coverage can be met”). See paragraph [0056], (“CCA engine 740 creates permutations and combinations of information within specification meta-data file 725”). See paragraph [0049], (“including the commands, options, arguments, and values that may be present in the software product under test. Other embodiments may use other mechanisms to store and interrelate this information, such as a graph, hash table, or tree”). Gopalakrishnan paragraph [0040] discloses the ordered structure of a command of the software product under test, (“a command such as “set_streaming_compression_configuration—dft_architecture open_core””), in which the command “set_streaming_compression_configuration” leads the command line and reads on “the target command as an entrance”, the option “—dft_architecture” occupies the middle position and reads on “at least one primary node”, and the value “open_core” occupies the last position and reads on “at least one child node”)
parsing, by a covered commands pool module, the at least one test case log to obtain covered commands and covered command counts for commands transmitted to the system during execution of the at least one test case (See Gopalakrishnan paragraph [0044], (“the test suite and test infrastructure information 540 are passed to a file list extractor 550… File list extractor 550, in one embodiment, uses a Tcl Parser to parse through the existing test suite”). See paragraph [0045], (“file list extractor 550 is provided the test suites as input and, using the parser algorithm, uses data mining on the test suite to identify each permutation and combination of configurations tested”). See paragraph [0049], (“the parser infrastructure of Block 620 determines the configurations tested in that file (Block 625)”). See paragraph [0038], (“a hash table data structure, which is created to keep track of occurrences of the software product configurations”))
wherein the method is implemented by at least one processing device comprising a processor coupled to a memory (See Gopalakrishnan paragraph [0066], (“The data processing system illustrated in FIG. 9 includes a bus or other internal communication means 940 for communicating information, and a processing unit 910 coupled to the bus 940 for processing information”). See paragraph [0067], (“a random access memory (RAM) or other volatile storage device 920 (referred to as memory), coupled to bus 940 for storing information and instructions to be executed by processor 910”))
Gopalakrishnan is silent to disclose; however, in an analogous art, Miranda teaches:
generating, by an adjustable target commands namespace module, a target commands namespace comprising a plurality of target commands obtained from one or more functional specifications for the system (Gopalakrishnan derives its namespace of commands by extraction from the software product itself. See Gopalakrishnan paragraph [0055], (“CCA 710 extracts configurations, such as commands, options, arguments, and values from software product under test 720”). Gopalakrishnan does not expressly disclose obtaining the target commands from a functional specification. Miranda expressly teaches deriving the list of the target operations of the system under test from the specification and user documentation. See Miranda section 3.2, page 1578, (“we express it as the list of operations that are expected to be invoked by users along with their associated occurrence probabilities. Ideally, it is developed during system specification with the participation of the system experts”), and (“after carefully reading the user manual for the version of the subject being investigated”). Miranda’s systems under test are command-line utilities. See section 3.1, page 1577, (“grep is a command-line utility that searches for lines matching a given regular expression in the provided file(s)”). See Table 4, page 1578, (“List of operations identified for grep”). Miranda further teaches that the list is adjustable. See section 3.3, page 1579, (“For deriving a customized operational profile, we randomly select an arbitrary number of operations”). Miranda’s specification-derived and customizable list of target operations of a command-line system reads on “a target commands namespace comprising a plurality of target commands obtained from one or more functional specifications for the system”)
assigning, to respective child nodes of each respective command tree map, weights based on occurrences of corresponding command combinations in the one or more functional specifications, wherein command combinations included in the full usage combinations and absent from the one or more functional specifications are assigned a zero weight (Gopalakrishnan treats every permutation and combination as an equally weighted covered-or-uncovered item and does not disclose weighting. Miranda expressly teaches assigning to each entity a weight derived from the number of occurrences of that entity in the specification-derived profile. See Miranda section 2, page 1574, (“Classify the entities according to their usage frequency with respect to the operational profile under testing”), and (“We rate the importance of entities based on their frequency of usage”). See section 2, page 1575, (“we opted for ordering the list of entities according to their frequency and assigning the first 1/3 entities to the high frequency group”), and (“we assigned the weights for the importance groups… in such a way that the medium group is three times more important than the low, and the high group, on its turn, is three times more important than the medium one”). Miranda Table 1, page 1576, records for each entity its count spectrum and its resulting importance group, reading on storing the assigned weight in the record for the respective entity. Miranda further expressly teaches assigning a zero weight to entities that are not relevant to the usage domain. See section 7, page 1589, (“we distinguished between relevant and not relevant coverage, which amounted to give entities either a 1 or 0 weight”), which reads on “command combinations included in the full usage combinations and absent from the one or more functional specifications are assigned a zero weight”)
normalizing the covered command counts to generate hit ratios and updating the respective command tree maps with the hit ratios (Gopalakrishnan tracks raw occurrences of configurations (paragraph [0038]) but does not expressly disclose normalizing those counts into a ratio stored back into the coverage structure. Miranda expressly teaches recording per-entity execution counts and converting them into a covered rate. See Miranda section 2, page 1575, (“Branch-count spectrum (BCS): for each conditional branch in a given program P, the spectrum indicates the number of times that branch was executed”), and Equation (1), defining “xi = the rate of covered entities from group i”. See section 4.1, page 1580, (“After each test case is run, we calculate:… the traditional coverage achieved… the operational coverage achieved (calculated according to (1))”). Miranda’s conversion of execution counts into covered rates, recomputed and stored after each execution, reads on “normalizing the covered command counts to generate hit ratios and updating the respective command tree maps with the hit ratios”)
outputting the command coverage report by querying the respective command tree maps to identify at least one command combination having missing coverage or weak coverage based at least in part on the weights and the hit ratios (Gopalakrishnan reports uncovered and partially covered commands (paragraphs [0056]–[0057]) but does so without reference to any weight or ratio. Miranda expressly teaches computing and reporting a coverage measure that is a function of both the assigned weights and the covered rates, and using that measure to identify which relevant entities remain uncovered. See Miranda section 2, page 1575, Equation (1), computing “Operational Coverage” as the weighted arithmetic mean of the rate of covered entities. See the worked example at page 1576, (“After TC1 is performed, two out of three low entities and two out of three medium entities are covered. No high entity is exercised by TC1”), in which the highest-weighted group contributes a zero term, identifying that the most important entities remain uncovered. See section 5.1, page 1582, (“the remaining test cases are ranked based on the list of uncovered entities and the weights assigned to the importance groups”). See section 8, page 1590, (“it would also provide the precise portions of code that are actually exercised by the program’s users and that are not covered by the test suite, guiding the test suite enhancement process towards the real usage of the program”). Under the broadest reasonable interpretation of “weak coverage”, which encompasses coverage that is insufficient relative to the importance of the entity, Miranda’s identification of highly weighted entities having a low covered rate reads on “identify at least one command combination having missing coverage or weak coverage based at least in part on the weights and the hit ratios”)
executing on the system at least one updated regression test case directed to the identified at least one command combination (Miranda expressly teaches selecting and running a further test case that targets the identified high-importance uncovered entities. See Miranda section 5.1, page 1582, (“the function getNextTestCase (line 3) selects the next test case based on the importance of the entities covered”), and (“The test case with highest rank is selected”). See section 4.1, page 1580, (“We carry out operational profile-based testing by selecting the next test case to be run (from the 10k set) according to the occurrence probabilities defined in the customized operational profile”). See section 5.1, page 1582, (“We run both test suites against the subject under testing”))
updating the respective command tree maps with at least one further hit ratio obtained from at least one further test case log resulting from the execution of the at least one updated regression test case (See Miranda section 4.1, page 1580, (“After each test case is run, we calculate:… the traditional coverage achieved… the operational coverage achieved (calculated according to (1))”). See the worked example at section 2, page 1576, (“After TC2 is performed, two additional entities, both high, are covered and the operational coverage will be 66.7%”), disclosing that the stored covered rates are revised from the execution data of each further test case)
determining, based at least in part on the at least one further hit ratio and the weight assigned to the identified at least one command combination, whether the missing coverage or weak coverage identified for that command combination has been filled (Miranda expressly teaches evaluating, from the recomputed weighted coverage value, whether the previously uncovered important entities have since been covered, and terminating when they have. See Miranda section 5.1, page 1582, (“TC1 is the third test case to be selected (with a rank of 7) and, at this point, Algorithm 2 stops because 100% coverage has been achieved”). See section 4.1, page 1580, (“we stop testing if none of them increases after a sequence of 10 test cases”). See section 8, page 1590, (“Operational coverage would acknowledge the former case by yielding 100% coverage. For the latter case, it would provide an assessment of the extent to which the derived test suite is adequate to that specific operational profile”))
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Miranda’s specification-derived, occurrence-based weighting and normalized coverage rates to Gopalakrishnan’s command coverage analyzer. Gopalakrishnan supplies the per-command usage combinations and the coverage report; Miranda supplies the weighting of those combinations by documented usage frequency and the weighted coverage rate. Gopalakrishnan itself identifies the need Miranda fills. See Gopalakrishnan paragraph [0009], (“existing metrics do not measures issues associated with customer usage… It would be desirable to have a quality metric that includes customer usage issues in the metrics for software quality testing”). Miranda articulates the same benefit. See Miranda section 2, page 1574, (“Software testing based on the operational profile ensures that testing resources are focused on the most frequently used operations and, thus, maximizes the reliability level that is achievable within the available testing time”). The combination is the application of a known technique to a known device ready for improvement to yield the predictable result of a command coverage report that distinguishes important under-tested command combinations from unimportant ones. See MPEP § 2143(I)(D).
With respect to claim 2 (Currently Amended), Gopalakrishnan further teaches wherein generating, by the command coverage assessment system, the command coverage report comprises: parsing, by a covered commands pool module, the at least one test case log, to obtain commands and corresponding covered command counts for command combinations transmitted to the system during the execution of the at least one test case, wherein the command coverage assessment system comprises the covered commands pool module (See Gopalakrishnan paragraph [0044], (“File list extractor 550, in one embodiment, uses a Tcl Parser to parse through the existing test suite… The results of this parse are stored in file list database 560, representing the configuration combinations that are present in each test case within the test suite”), which discloses counts obtained for configuration combinations. See paragraph [0045], (“uses data mining on the test suite to identify each permutation and combination of configurations tested”). See paragraph [0038], (“a hash table data structure, which is created to keep track of occurrences of the software product configurations”). Gopalakrishnan paragraph [0055] discloses that the file list extractor and the parsing engine are components of the CCA, (“CCA 710 extracts information from an existing test suite 730, if any is available, and stores the extracted information as a file list meta-data file 735”), which reads on “the command coverage assessment system comprises the covered commands pool module.”).
With respect to claim 3 (Currently Amended), Gopalakrishnan further teaches obtaining, by the covered commands pool module, a plurality of covered commands and a covered command count associated with each covered command of the respective plurality of covered commands, wherein the covered command count indicates a total occurrence of each covered command of the respective plurality of covered commands in the at least one test case log (See Gopalakrishnan paragraph [0038], (“a hash table data structure, which is created to keep track of occurrences of the software product configurations”). See paragraph [0039], (“By keeping track of occurrences of the EDA product configurations and determining permutations and combinations of those configurations, CCA applied to this example EDA product both identifies these types of situations as uncovered”). See paragraph [0050], (“configurations extracted from each test file in test suite database 610 are used to update (Block 630) the file… In one embodiment, the file updated is a golden log”). Gopalakrishnan’s tracking of the number of occurrences of each configuration across the parsed test data reads on “the covered command count indicates a total occurrence of each covered command of the respective plurality of covered commands in the at least one test case log.”).
With respect to claim 4 (Currently Amended), Gopalakrishnan is silent to disclose, however, in an analogous art, Miranda teaches updating a child node of a respective command tree map associated with the covered command with a hit ratio generated by normalizing the respective covered command count, wherein the plurality of covered commands comprises the covered command (Gopalakrishnan teaches tracking the raw number of occurrences of each configuration in the coverage data structure and updating that structure as test data is parsed. See Gopalakrishnan paragraph [0038], (“created to keep track of occurrences of the software product configurations”). See paragraph [0050], (“configurations extracted from each test file in test suite database 610 are used to update (Block 630) the file”). Gopalakrishnan’s stored values are raw occurrence counts, and Gopalakrishnan does not expressly disclose normalizing that count into a ratio written back to the node associated with the covered command.
Miranda expressly teaches normalizing the per-entity execution count into a covered rate and revising that rate for the respective entity after execution. See Miranda section 2, page 1575, (“for each conditional branch in a given program P, the spectrum indicates the number of times that branch was executed”), and Equation (1), defining “xi = the rate of covered entities from group i”. See section 4.1, page 1580, (“After each test case is run, we calculate:… the operational coverage achieved (calculated according to (1))”). Under the broadest reasonable interpretation of “hit ratio”, which encompasses any ratio derived from the number of times an entity is exercised during execution, Miranda’s normalized covered rate stored for the respective entity reads on “a hit ratio generated by normalizing the respective covered command count”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to normalize the covered command counts of Gopalakrishnan into hit ratios stored at the node associated with the covered command, as taught by Miranda, in order to express the coverage of each command combination in terms comparable across command combinations having different total occurrence counts. Gopalakrishnan already evaluates coverage in percentage terms at the aggregate level. See Gopalakrishnan paragraph [0035], (“command coverage is considered adequate when it is 100%… In another embodiment 90% coverage may be considered sufficient to ship the product”). Applying that same normalization at the per-command level, as taught by Miranda section 2, page 1575, Equation (1), is the use of a known technique to improve a similar device in the same way and yields the predictable result of a per-command coverage value that can be compared against the coverage of other command combinations. See MPEP § 2143(I)(C).
With respect to claim 5 (Currently Amended), Gopalakrishnan teaches outputting, by a commands coverage report module, an analysis report indicating whether target commands in a target commands namespace are covered in a covered commands pool associated with a covered commands pool module, wherein the covered commands pool comprises covered commands in the at least one test case log, wherein the command coverage assessment system comprises the commands coverage report module and the covered commands pool module (See Gopalakrishnan paragraph [0046], (“The CCA engine 570… analyzes the overlap between the available permutations and combinations of configurations, such as commands, options, arguments, and values, and the tested set of configurations, such as commands, options, arguments, and values, from the test suite”). See paragraph [0056], (“a report 760 is generated to identify the testing gaps based on the set of configurations. In one embodiment, configurations in first region 762 are uncovered commands, second region 764 are partially covered commands, and third region 766 are fully covered commands”). See paragraph [0057], (“Summary region 810, in one embodiment, gives a count for each of the number of uncovered commands (commands which are not represented any test in the test suite), partially covered commands… and fully covered commands”)).
With respect to claim 7 (Currently Amended), Gopalakrishnan teaches evaluating, by the command coverage assessment system, the at least one test case log to identify usage of at least one command combination used during the execution of the at least one test case (See Gopalakrishnan paragraph [0045], (“using the parser algorithm, uses data mining on the test suite to identify each permutation and combination of configurations tested”). See paragraph [0049], (“For each test in the test suite within test suite database 610, the parser infrastructure of Block 620 determines the configurations tested in that file (Block 625)”). See paragraph [0044], (“representing the configuration combinations that are present in each test case within the test suite”)).
With respect to claim 8 (Currently Amended), Gopalakrishnan teaches generating a data structure comprising the one or more command tree maps for command coverage analysis (See Gopalakrishnan paragraph [0038], (“configuration extractor 520 forms the keys to a hash table data structure, which is created to keep track of occurrences of the software product configurations. In another embodiment, a tree structure is used, or a different type of data structure”). See paragraph [0049], (“Other embodiments may use other mechanisms to store and interrelate this information, such as a graph, hash table, or tree”). See paragraph [0046], (“The CCA engine 570 then generates a command coverage database 580”)).
With respect to claim 9 (Currently Amended), Gopalakrishnan teaches instantiating at least one command tree map comprising at least one primary node and at least one child node, wherein the at least one primary node corresponds to a middle parameter of a command line and the at least one child node corresponds to a last parameter of the command line, and wherein a target command corresponding to the at least one command tree map forms an entrance of the at least one command tree map (See Gopalakrishnan paragraph [0049], (“a look up table (LUT) (Block 615) is created from this information, including the commands, options, arguments, and values that may be present in the software product under test. Other embodiments may use other mechanisms to store and interrelate this information, such as a graph, hash table, or tree”). See paragraph [0040], (“a command such as “set_streaming_compression_configuration—dft_architecture open_core””). In the disclosed command line, the command leads and reads on the target command forming “an entrance”; the option “—dft_architecture” occupies the middle position and reads on “the at least one primary node corresponds to a middle parameter of a command line”; and the value “open_core” occupies the last position and reads on “the at least one child node corresponds to a last parameter of the command line”. See also paragraph [0041], (“the input/output port count created in the core interface may be 12 or it may be 4, based on another option specifying a DFT_architecture value”), evidencing the dependent relationship between the option level and the value level).
With respect to claim 10 (Currently Amended), Gopalakrishnan is silent to disclose, however, in an analogous art, Miranda teaches generating, by an adjustable target commands namespace module, a target commands namespace comprising at least one target command, wherein the at least one test case comprises testing the at least one target command, and wherein the command coverage assessment system comprises the adjustable target commands namespace module, wherein the at least one target command is obtained from the one or more functional specifications for the system (Gopalakrishnan derives its namespace by extraction from the software product itself (paragraph [0055]) and does not expressly disclose obtaining the target commands from a functional specification, nor that each test case is created to test a particular target command of that namespace. Miranda expressly teaches both. See Miranda section 3.2, page 1578, (“we express it as the list of operations that are expected to be invoked by users”), and (“after carefully reading the user manual for the version of the subject being investigated”). See Table 4, page 1578, (“List of operations identified for grep”). See section 3.3, page 1579, (“Test pool. For each subject investigated, we created a test pool containing 10k test cases uniformly distributed among the operations in the subject’s operational profile”), and (“Operation matrix. Each test case in the test pool is created for a specific operation in the operational profile and this mapping (test case, operation) is stored in the operation matrix”), which reads on “the at least one test case comprises testing the at least one target command”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to derive the target commands namespace of Gopalakrishnan from the functional specification of the system, and to create each test case to test a target command of that namespace, as taught by Miranda, in order to scope the coverage analysis to the commands that the documentation holds out to users. Gopalakrishnan expressly identifies this need. See Gopalakrishnan paragraph [0009], (“existing metrics do not measures issues associated with customer usage… It would be desirable to have a quality metric that includes customer usage issues in the metrics for software quality testing, to proactively minimize bugs”). Miranda supplies the specification-derived namespace that meets that need. See Miranda section 3.2, page 1578. The combination is the use of a known technique to improve a similar device in the same way and yields the predictable result of a coverage namespace scoped to documented customer usage rather than to every configuration extractable from the product binary. See MPEP § 2143(I)(C).
With respect to claim 11 (Currently Amended), Gopalakrishnan is silent to disclose, however, in an analogous art, Miranda teaches obtaining from a target commands namespace, target command coverage information corresponding to command combinations obtained from the one or more functional specifications; assigning a weight to the target command coverage information; and storing the weight in a respective child node of a respective command tree map, wherein the target command coverage information is obtained from the one or more functional specifications for the system (Gopalakrishnan yields per-command coverage information from its namespace. See Gopalakrishnan paragraph [0046], (“The CCA engine 570… analyzes the overlap between the available permutations and combinations of configurations… and the tested set of configurations… from the test suite”). See paragraph [0055], (“specification meta-data file 725 representing the configurations available or possible in software product under test 720”). Gopalakrishnan does not, however, expressly disclose assigning a weight to that coverage information or storing such a weight at the node level. Miranda expressly teaches assigning each entity a weight derived from its frequency in the specification-derived profile and recording that weight for the respective entity. See Miranda section 2, page 1574, (“We rate the importance of entities based on their frequency of usage”). See section 2, page 1575, (“we assigned the weights for the importance groups… in such a way that the medium group is three times more important than the low, and the high group, on its turn, is three times more important than the medium one”). See Table 1, page 1576, recording for each entity its count spectrum and its assigned importance group. Under the broadest reasonable interpretation of “weight”, which encompasses any numerical value indicative of importance associated with the target command coverage information, Miranda’s per-entity importance value reads on “a weight” stored in the record for that entity).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to assign to the per-command coverage information of Gopalakrishnan a weight derived from the occurrence of the corresponding command combination in the functional specification, and to store that weight at the corresponding node of the tree-structured coverage data, as taught by Miranda, in order to distinguish the command combinations that matter most to users from those that do not. Miranda expressly states the reason. See Miranda section 1, page 1573, (“all entities are considered as having same relevance for the purpose of completing coverage. This assumption seems contradictory with the very idea of reliability according to which user’s functions should be exercised more or less frequently accordingly to user profiles”). Gopalakrishnan supplies the per-command coverage information and the node-structured store for it. See Gopalakrishnan paragraphs [0046] and [0049]. The combination is the use of a known technique to improve a similar device in the same way and yields the predictable result of a coverage structure in which each command combination carries its own importance value. See MPEP § 2143(I)(C).
With respect to claim 12 (Currently Amended), Gopalakrishnan is silent to disclose, however, in an analogous art, Miranda teaches wherein the respective child node comprises a hit ratio associated with a command executed during the execution of the at least one test case, wherein the hit ratio and the weight are stored in the respective child node (Gopalakrishnan reports coverage as a percentage at the aggregate level (paragraph [0035]) and tracks occurrences in the data structure (paragraph [0038]), but does not disclose a ratio carried at the node alongside a weight. Miranda expressly teaches maintaining, for each entity, both the execution-derived rate and the assigned importance value. See Miranda Table 1, page 1576, storing for each entity its “Count spectrum” and its “Importance group”. See Table 2, page 1576, (“Test cases and entities (from Table 1) they cover classified into different importance groups”), recording per entity the coverage achieved during test case execution alongside its importance group. See section 2, page 1575, Equation (1), (“xi = the rate of covered entities from group i”, “wi = the weight assigned to group i”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to store, at the same node that carries the weight, the hit ratio of the corresponding command obtained from the execution of the test case, as taught by Miranda, in order to make both values available at the node when the coverage of that command is evaluated. Miranda’s coverage equation requires both the weight and the covered rate for each entity. See Miranda section 2, page 1575, Equation (1), (“xi = the rate of covered entities from group i”, “wi = the weight assigned to group i”). See Miranda Tables 1 and 2, page 1576, maintaining both values per entity. The combination is the use of a known technique to improve a similar device in the same way and yields the predictable result of a node from which the coverage of the corresponding command can be evaluated without traversing a separate structure. See MPEP § 2143(I)(C).
With respect to claim 13 (Currently Amended), Gopalakrishnan is silent to disclose, however, in an analogous art, Miranda teaches wherein the hit ratio is a normalized covered command count, wherein the covered command count indicates a total occurrence of a covered command in the at least one test case log (Gopalakrishnan tracks occurrences of configurations in the coverage data structure (paragraph [0038]), which corresponds to the claimed total occurrence count, but does not disclose a hit ratio that is a normalized form of that count. Miranda expressly teaches both the occurrence count and its normalization. See Miranda section 2, page 1575, (“Statement-count spectrum (SCS): for each statement in a given program P, the spectrum indicates the number of times that statement was executed”). See section 2, page 1575, Equation (1) and (“When the weights are normalized such that they sum up to 1, Equation 1 can be reduced to”), disclosing the normalization of the counts into a rate).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to express the hit ratio as a normalized form of the covered command count that Gopalakrishnan already tracks, as taught by Miranda, in order to obtain a coverage value independent of the absolute number of occurrences recorded. Gopalakrishnan supplies the occurrence count. See Gopalakrishnan paragraph [0038], (“a hash table data structure, which is created to keep track of occurrences of the software product configurations”). Miranda supplies the normalization of that count. See Miranda section 2, page 1575, Equation (1), and (“When the weights are normalized such that they sum up to 1, Equation 1 can be reduced to”). The combination is the use of a known technique to improve a similar device in the same way, and normalizing a count into a ratio is a routine mathematical operation in coverage measurement yielding predictable results. See MPEP § 2143(I)(C).
With respect to claim 15 (Currently Amended), Gopalakrishnan teaches creating a respective command tree map for each target command of a plurality of target commands in the target commands namespace, wherein each respective command tree map comprises full usage combinations of the target command, wherein each respective command tree map comprises a respective primary node and at least one respective child node (See Gopalakrishnan paragraph [0056], (“CCA engine 740 creates permutations and combinations of information within specification meta-data file 725”). See paragraph [0022], (“The CCA may additionally extract configurations of a software product, produces permutations and combinations of these configurations”). See paragraph [0049], (“including the commands, options, arguments, and values that may be present in the software product under test. Other embodiments may use other mechanisms to store and interrelate this information, such as a graph, hash table, or tree”). See paragraph [0040], disclosing the command, option, and value levels of a command of the software product under test). Examiner notes that claim 15 is additionally rejected under 35 U.S.C. 112(d) as set forth above.
With respect to claim 16 (Currently Amended), Gopalakrishnan is silent to disclose, however, in an analogous art, Miranda teaches updating a weight stored in the at least one respective child node of the respective command tree map based on information associated with a full usage combination of the target command corresponding to the respective command tree map, wherein the information comprises an occurrence of the full usage combination in the one or more functional specifications (Gopalakrishnan updates the coverage data store collectively as configurations are extracted from each test file. See Gopalakrishnan paragraph [0050], (“configurations extracted from each test file in test suite database 610 are used to update (Block 630) the file, e.g. a LUT, created in Block 615”). Gopalakrishnan does not, however, disclose updating a weight at the node corresponding to a particular usage combination based on the occurrence of that combination in a functional specification. Miranda expressly teaches that the stored weight of each entity is re-derived from its occurrence frequency whenever the specification-derived profile changes. See Miranda section 3.3, page 1579, (“Importance groups. For each customized operational profile, we classified the program entities into different importance groups (high, medium, or low) based on their frequency of usage”). See section 2, page 1575, (“we opted for ordering the list of entities according to their frequency and assigning the first 1/3 entities to the high frequency group”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to update the weight stored at the child node of the command tree map when the occurrence of the corresponding full usage combination in the functional specification changes, as taught by Miranda, in order to keep the coverage priorities aligned with the current documented usage of the system. Miranda expressly teaches re-deriving the stored weight from occurrence frequency whenever the specification-derived profile changes. See Miranda section 3.3, page 1579, (“Importance groups. For each customized operational profile, we classified the program entities into different importance groups (high, medium, or low) based on their frequency of usage”). Gopalakrishnan already updates its coverage data store as inputs change. See Gopalakrishnan paragraph [0050]. The combination is the use of a known technique to improve a similar device in the same way and yields the predictable result of stored weights that remain current with the functional specification. See MPEP § 2143(I)(C).
With respect to claim 17 (Currently Amended), Gopalakrishnan teaches querying a respective command tree map associated with a command to determine whether a covered commands pool associated with a covered commands pool module comprises the command, wherein a target commands namespace associated with an adjustable target commands namespace module comprises the command, wherein the command coverage assessment system comprises the adjustable target commands namespace module and the covered commands pool module, and identifying, based on the querying, whether a command combination associated with the command has missing coverage or weak coverage (See Gopalakrishnan paragraph [0048], (“after parsing an entire test suite, a sequential scan of the coverage database, in one example a hash table, provides the command coverage statistics of the software product under test”), reading on querying the stored structure. See paragraph [0046], (“analyzes the overlap between the available permutations and combinations of configurations… and the tested set of configurations… from the test suite”). See paragraph [0056], (“configurations in first region 762 are uncovered commands, second region 764 are partially covered commands, and third region 766 are fully covered commands”). Gopalakrishnan’s uncovered commands read on “missing coverage” and its partially covered commands read on “weak coverage”. See paragraph [0057], (“partially covered commands (commands which are represented by one or more tests in the test suite, but not all configurations of the command are represented by a test in the test suite)”)).
Gopalakrishnan is silent to disclose, however, in an analogous art, Miranda teaches wherein the queried command tree map stores a weight and a hit ratio for the command (See Miranda Table 1, page 1576, storing for each entity its count spectrum and its assigned importance group, and Table 2, page 1576, recording the coverage achieved for each entity during test case execution. See section 2, page 1575, Equation (1), (“xi = the rate of covered entities from group i”, “wi = the weight assigned to group i”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to store the weight and the hit ratio for each command in the command tree map queried by Gopalakrishnan, as taught by Miranda, in order that the query distinguish command combinations that are important and under-tested from those that are unimportant. Gopalakrishnan supplies the queryable coverage structure and the uncovered and partially covered classification. See Gopalakrishnan paragraphs [0048] and [0056]. Miranda supplies the per-entity weight and rate that the query evaluates, and states the benefit. See Miranda section 8, page 1590, (“it would also provide the precise portions of code that are actually exercised by the program’s users and that are not covered by the test suite, guiding the test suite enhancement process towards the real usage of the program”). The combination is the use of a known technique to improve a similar device in the same way and yields the predictable result of a coverage query returning the command combinations most in need of additional testing. See MPEP § 2143(I)(C).
With respect to claim 19 (Currently Amended), claim 19 recites a system comprising at least one processing device configured to perform operations corresponding to the method of claim 1, and is rejected for the same reasons set forth with respect to claim 1. Gopalakrishnan further teaches at least one processing device comprising a processor coupled to a memory, the at least one processing device being configured to perform the recited operations (See Gopalakrishnan paragraph [0066], (“The data processing system illustrated in FIG. 9 includes a bus or other internal communication means 940 for communicating information, and a processing unit 910 coupled to the bus 940 for processing information”). See paragraph [0067], (“a random access memory (RAM) or other volatile storage device 920 (referred to as memory), coupled to bus 940 for storing information and instructions to be executed by processor 910”). See paragraph [0073], (“The control logic or software implementing the present invention can be stored in main memory 920, mass storage device 930, or other storage medium locally or remotely accessible to processor 910”)). The recitation of “an information system” for the system under test corresponds to Gopalakrishnan’s software product under test 720 and is taught for the same reasons set forth with respect to claim 1.
With respect to claim 20 (Currently Amended), claim 20 recites a computer program product comprising a non-transitory processor-readable storage medium having stored therein program code that causes at least one processing device to perform operations corresponding to the method of claim 1, and is rejected for the same reasons set forth with respect to claim 1. Gopalakrishnan further teaches a non-transitory processor-readable storage medium having stored therein program code of one or more software programs (See Gopalakrishnan paragraph [0074], (“This control logic or software may also be resident on an article of manufacture comprising a computer readable medium having computer readable program code embodied therein and being readable by the mass storage device 930 and for causing the processor 910 to operate in accordance with the methods and teachings herein”). See paragraph [0078], (“A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g. a computer). For example, a machine readable medium includes read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, or other storage media which may be used for temporary or permanent data storage”)).
Claims 6 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Gopalakrishnan et al. (US Pub. No. 2018/0107587 – hereinafter Gopalakrishnan) in view of Miranda et al., “An assessment of operational coverage as both an adequacy and a selection criterion for operational profile based testing,” Software Quality Journal, Vol. 26, No. 4, pp. 1571-1594, published online September 22, 2017 (hereinafter Miranda) and further in view of Takahashi (US Pub. No. 2010/0180258 – hereinafter Takahashi).
With respect to claim 6 (Currently Amended), Gopalakrishnan in view of Miranda is silent to disclose, however, in an analogous art, Takahashi teaches wherein the analysis report further comprises a coverage level associated with the covered commands and a coverage level requirement threshold, wherein the coverage level is determined based at least in part on the weights and the hit ratios stored in the respective command tree maps (Gopalakrishnan reports coverage against an adequacy criterion expressed as an unweighted percentage. See Gopalakrishnan paragraph [0035], (“command coverage is considered adequate when it is 100%… In another embodiment 90% coverage may be considered sufficient to ship the product”). Gopalakrishnan in view of Miranda does not expressly disclose an analysis report comprising a coverage level computed from the weights and the rates together with a requirement threshold against which that level is reported. Takahashi expressly teaches computing a weighted coverage level as the ratio of the aggregated weights of the executed entities to the aggregated weights of all entities, and reporting that level against a defined threshold. See Takahashi paragraph [0036], (“The WCC for the basic blocks executed during testing are added together. In this example, the WCC of the executed basic blocks (494) has a value of 575. The ratio of the WCC of the executed basic blocks to the maximum WCC is calculated, which in this example has a value of 92 percent”), which reads on “the coverage level is determined based at least in part on the weights and the hit ratios”. See paragraph [0039], (“the user defines their code coverage goal (510). For example, in one embodiment, the user may identify their goal that the code evaluation be limited to eighty percent of the maximum possible WCC”), which reads on “a coverage level requirement threshold”, and further, (“a percentage of maximum possible weight code coverage is calculated (516). A comparison (518) is made between the value defined at step (510) and the percentage value calculated at step (516). In one embodiment, a detailed report identifying all executed and non-executed basic block, and the WCC value for each basic block is generated”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to report the weighted coverage level and requirement threshold taught by Takahashi in the analysis report of Gopalakrishnan in view of Miranda. Gopalakrishnan already gates the ship decision on a coverage threshold (paragraph [0035]); Takahashi teaches expressing that threshold in weighted terms and states the reason. See Takahashi paragraph [0008], (“software test plans generally have code coverage objectives that fall short of full path coverage… A significant problem with this approach, however, is that there is generally no distinction made for which 20% of the code will go untested”). See paragraph [0049], (“this invention encourages the testing of code sections with high WCC values, and does not penalize the tester for failing to test code sections with low WCC values”). The combination yields the predictable result of an adequacy determination reflecting whether the important command combinations have been tested rather than merely how many have been tested. See MPEP § 2143(I)(A). With respect to claim 18 (Currently Amended), Gopalakrishnan in view of Miranda is silent to disclose, however, in an analogous art, Takahashi teaches determining whether a coverage level associated with the command meets a requirement threshold based at least in part on the weight and the hit ratio for the command (See Takahashi paragraph [0039], (“The weighted code coverage value per basic block is aggregated for all executed basic blocks (514), and a percentage of maximum possible weight code coverage is calculated (516). A comparison (518) is made between the value defined at step (510) and the percentage value calculated at step (516)… It is then determined if the comparison meets a threshold (520). If the comparison meets the threshold, then it is understood that a sufficient amount of basic blocks have been evaluated (522). Conversely, if the comparison does not meet the threshold, then further testing and evaluation of the software and the associated basic blocks is necessary (524)”). See paragraph [0011], (“The defined weighted code coverage goal is met when the aggregated score of the executed blocks at least meets the defined code coverage goal”)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to determine whether the coverage level of a command meets a requirement threshold on the basis of the weight and the hit ratio for that command, as taught by Takahashi, in order to decide whether further testing of that command is required. Takahashi expressly discloses that determination and its purpose. See Takahashi paragraph [0039], (“It is then determined if the comparison meets a threshold (520). If the comparison meets the threshold, then it is understood that a sufficient amount of basic blocks have been evaluated (522). Conversely, if the comparison does not meet the threshold, then further testing and evaluation of the software and the associated basic blocks is necessary (524)”). Gopalakrishnan already gates its testing-complete determination on a coverage threshold. See Gopalakrishnan paragraph [0035]. The combination is the use of a known technique to improve a similar device in the same way and yields the predictable result of a per-command adequacy determination reflecting the importance of that command. See MPEP § 2143(I)(C).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Gopalakrishnan et al. (US Pub. No. 2018/0107587 – hereinafter Gopalakrishnan) in view of Miranda et al., “An assessment of operational coverage as both an adequacy and a selection criterion for operational profile based testing,” Software Quality Journal, Vol. 26, No. 4, pp. 1571-1594, published online September 22, 2017 (hereinafter Miranda) and further in view of Lee-Smith et al. (US Pub. No. 2020/0104243 – hereinafter Lee-Smith – previously presented).
With respect to claim 14 (Currently Amended), Gopalakrishnan in view of Miranda is silent to disclose, however, in an analogous art, Lee-Smith teaches storing a sum of a plurality of weights in a primary node of the respective command tree map, wherein the plurality of weights is associated with a respective plurality of child nodes associated with the primary node, wherein the plurality of child nodes comprises the respective child node (Gopalakrishnan computes aggregate coverage statistics across the entire coverage database. See Gopalakrishnan paragraph [0048], (“a sequential scan of the coverage database, in one example a hash table, provides the command coverage statistics of the software product under test”). See paragraph [0057], (summary region with aggregate counts of uncovered, partially covered, and fully covered commands). Gopalakrishnan’s aggregation is at a single overall level, not a sum stored at an intermediate primary node where the summed values originate from the child nodes associated with that primary node.
Lee-Smith expressly teaches a tree having primary nodes with child nodes depending from them. See Lee-Smith paragraph [0021], (“the primary branches 214, which can represent the most fundamental aspects of the program or protocol… The tree can contain many children branches within the primary branches”). See paragraph [0024], (“branch node 262 is a parent node of leaf node 272 and nodes 274, 276”). Lee-Smith further expressly teaches computing, at the branch node, a value obtained by summing the values of the constituent child nodes, and retaining that value at the branch. See paragraph [0038], (“The process can begin with requesting the total number of completed tests within the branch (610)… The system can then determine the number of total accepted tests within the branch. The system can determine this number by requesting the number of incomplete accepted tests (620). By adding the number of incomplete accepted tests and completed tests, the system can find the number of total accepted tests within the branch”). See paragraph [0036], (“all of the branches can have some visual indicator as to the completion percentage—the percentage of accepted tests for which a completed test exists with respect to the total number of accepted tests—within the branch”). Under the broadest reasonable interpretation of “weights”, which encompasses any numerical value indicative of importance, magnitude, or coverage status associated with a node, Lee-Smith’s summation at the branch node of the values of its constituent child nodes reads on “storing a sum of a plurality of weights in a primary node… wherein the plurality of weights is associated with a respective plurality of child nodes associated with the primary node”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the branch-level summation technique of Lee-Smith to the weighted command tree map of Gopalakrishnan in view of Miranda, so that each primary node stores the sum of the weights of the child nodes depending from it, in order to permit assessment of coverage at the level of the primary node rather than only at the level of the individual child node or of the coverage database as a whole. Lee-Smith expressly identifies this benefit. See Lee-Smith paragraph [0037], (“The use of a heatmap or other visualization tool might allow for a quick appraisal of the thoroughness of the test cases in a program with a focus on what users are requesting, rather than every conceivable issue”). Gopalakrishnan supplies the tree-structured per-command coverage data and Miranda supplies the per-node weight; Lee-Smith supplies the known technique of aggregating child values at the parent node of such a tree. The combination is the use of a known technique to improve a similar device in the same way and yields the predictable result of a primary node from which the weighted coverage of the corresponding group of command combinations can be read directly. See MPEP § 2143(I)(C).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Avvari et al. (US Pub. No. 2004/0025088) discloses analyzing test coverage by profiling an application and a test suite into call trees, comparing the trees to identify gaps not covered by the test case call trees, creating new test cases to cover the identified gaps, executing those test cases, and profiling them to generate new test case call trees that are combined with the existing test case call trees. See Avvari paragraphs [0061]-[0064] and FIGS. 8A-8B.
Bnayahu et al. (US Pub. No. 2013/0067436) discloses associating portions of computer code with corresponding design specifications or requirements derived from a requirements document, applying a scoring function to yield a coverage score for each requirement, and calculating that score using a normalization of the number of links pointing to the requirement, for example ri = ci/Σ{cj}, the resulting ordered list being useable to generate functional tests for enhancing coverage. See Bnayahu paragraphs [0031]-[0033] and [0039].
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANIBAL RIVERACRUZ whose telephone number is (571)270-1200. The examiner can normally be reached Monday-Friday 9:30 AM-6:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hyung S Sough can be reached at 5712726799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANIBAL RIVERACRUZ/Primary Examiner, Art Unit 2192
