DETAILED ACTION
Remarks
Applicant presents a request for continued examination filed 13 July 2026 which was filed in response to the 23 April 2026 final Office action (the “Previous Action”).
Claims 1, 7-8 and 15 are amended.
Claims 1-20 are pending. Claims 1, 8 and 15 are the independent claims.
Any unpersuasive arguments are addressed in the “Response to Arguments/Amendments” section below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 13 July 2026 has been entered.
Response to Arguments/Amendments
Applicant submits that a terminal disclaimer has been filed and requests withdrawal of the double patenting rejections. (Remarks, p. 1 last par.)
Examiner respectfully points out that on terminal disclaimer has been filed. Nonetheless, the double patenting rejections have been withdrawn in view of Applicant’s claim amendments.
Applicant first argues with respect to claims 1, 8 and 15 that Goswani and Zhou do not individually teach or suggest certain features that they were not individually cited as teaching. (Remarks, p. 3 par 3 – p. 4 par. 1). These arguments are unpersuasive because one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Applicant also submits as part of this argument that Zhou updates the graph while iterating over every test to generate a complete coverage map as opposed to updating the graph for a subset of tests. (Remarks, p. 4 par. 2).
Examiner respectfully disagrees with this characterization of Zhou and points out that Zhou discloses that “if just one component or function is to be tested, a set of test cases suitable for testing that component” can be identified which “avoids wasting time running test cases not related to the selected components or functions” (Zhou at col. 3 ll. 17-23). And Zhou’s graph is “updated over time, as different test cases are executed.” (Zhou at col. 5 ll. 24-27).
Thus, because Zhou teaches executing only a subset of tests when only one component is tested and updating the graph as the tests are executed, Zhou does in fact teach updating a graph with information for only a subset of tests. Zhou’s subset may not be explicitly “change triggered” but Zhou was never cited as teaching a change triggered subset of tests, Goswani was. It would have been obvious to combine the references and arrive at what is claimed for the reasons set forth in the rejections below.
Applicant argues with respect to claims 1, 8 and 15 that the combination does not yield what is claimed because it does not supply a mechanism in which the selective updating and the call-graph-based subset selection are the “same mechanism” operating across successive cycles such that the very portion of the call graph updated by the executed test of one cycle is the portion that carries forward to inform the subset selection of the next, while the non-executed portion is deliberately left unchanged. (Remarks, p. 5 pars. 2-3).
Examiner respectfully points out in response, however, that none of the above is actual claim language. To the extent these features even appear in the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Every feature actually claimed is taught by or obvious in view of the cited combination of references for the reasons set forth below. Note too that this argument appears to be based on the assertion that Zhou’s graph is only updated when executing every test case. Examiner respectfully disagrees with this assertion for the reasons set forth above.
Applicant then argues with respect to claims 1, 8 and 15 that the rationale for combining the Goswani and Zhou speaks only to continual updating in the abstract, does not articulate why a person of ordinary skill would arrive at the “specific limitations recited” of selectively updating the executed portion, leaving the non-executed portion unchanged and reusing that partially-updated graph to drive the next change-triggered selection, and is based on hindsight. (Remarks p. 5 last par.).
Examiner respectfully disagrees.
First, “reusing the partially-updated graph to drive the next change-triggered selection” is not recited by the claim. Again, limitations from the specification are not read into the claims.
Second, Applicant’s argument mischaracterizes the rationale identified in the rejection. The rejection refers to “ensuring the graph is continually updated to reflect coverage of the executed test cases”, not merely continual updating “in the abstract.” In the examiner’s view, updating the executed portion of the graph and not the non-executed portion is performed by Zhou in order for Zhou’s graph to incorporate updated coverage when, as in Zhou, only a subset of tests is executed.
Third, any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made and does not include knowledge gleaned only from the applicant's disclosure, which is the case here, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971).
Applicant’s arguments with respect to claims 1, 8 and 15 are thus unpersuasive. Applicant’s arguments with respect to the remaining claims by virtue of their dependence from claims 1, 8 or 15 are unpersuasive for the same reasons.
Claim Objections
The Previous Action’s claim objections are withdrawn in view of Applicant’s claim amendments unless reproduced herein.
Claims 1-7 and 15-20 are objected to for the following informalities:
Claim 1 refers to “wherein the subset list of tests being a subset” at line 19, which appears to be a typographical error that should perhaps read -wherein the subset list of tests is a subset- instead.
Claims 2-7 are objected to via dependence from claim 1.
Claim 15 is objected to for the same reasons as claim 1.
Claims 16-20 are objected to via dependence from claim 8.
Double Patenting
The Previous Action’s double patenting rejections are withdrawn in view of Applicant’s amendments.
Claim Rejections - 35 USC § 112
The Previous Action’s § 112 rejections are withdrawn in view of Applicant’s amendments unless reproduced herein.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 7-11 and 14-18 are rejected under 35 U.S.C. 103 as being unpatentable over Griffin et al. (US 9,514,034) (art of record – hereinafter Griffin) in view of Goswami et al. (US 2022/0188215) (art of record – hereinafter Goswani), Farrier (US 2021/0263728) (art of record – hereinafter Farrier) and Zhou et al. (US 7,480,900) (art of record – hereinafter Zhou).
As to claim 1, Griffin discloses a method for automatically testing software code, comprising:
generating, by a remote server, a subset list of tests (e.g., Griffin, col. 3 ll. 11-23: the set of tests 115 may be selected from a larger set of potential tests 115; col. 3 l. 26-27 an ordered set of lists 125 that places the original set of tests 115 in the ordered sequence; Fig. 2 and associated text, col. 5 ll. 23-24: the test ordering service 220 may be implemented by one or more computing devices [remote servers]; col. 5 ll. 53-64: the test ordering service may generate the ordered set of tests 125 [subset list of tests] and return the ordered set of tests 125)
receiving, by the agent on the testing server from the remote server, the subset list of tests, (e.g., Griffin, Fig. 2 and associated text, col 5 ll. 53-64: the test ordering service may generate the ordered set of tests 125 [subset list of tests] and return the ordered set of tests 125. When test ordering service 220 returns the ordered set of tests 125 to the test execution system, the ordered set of tests 125 may be stored in the test ordering cache 140; col. 3 ll. 40-45: the test execution module [agent, or part of one] may perform the tests in the order specified in the ordered set of tests 125; col. 5 ll. 53-56: the test ordering service may generate the ordered set of tests 125 and return the ordered set of tests 125 to the test execution system 100) the received subset list of tests being a subset of the plurality of tests; (e.g., Griffin, col. 3 ll. 11-23: the set of tests 115 may be selected from a larger set of potential tests 115; col. 3 l. 26-27 an ordered set of lists 125 that places the original set of tests 115 in the ordered sequence)
ordering each test in the subset list of tests according to a likelihood of failure determined for each test in the subset of tests; (e.g., Griffin, col. 5 ll. 30-35: test ordering service 220 may use the functionality for test failure likelihood estimation 121 to generate ordered sequences of tests; col. 3 ll. at least a portion of the tests 115 may be ordered from most likely to fail to least likely to fail in the ordered set of tests 125) and
executing the ordered tests of the subset of tests by the agent in the testing server, (e.g., Griffin, Fig. 1 and associated text, col. 3 ll. 40-45: the test execution module [agent, on text execution system 100 (testing server)] may perform the tests in the order specified in the ordered set of tests 125)
Griffin does not explicitly disclose: generating an initial call graph for a plurality of methods within a first software to be tested in a first testing, the initial call graph including relationships between the methods and each of a plurality of tests in the first testing, each of the relationships associating one of the plurality of tests with one or more of the methods; detecting a test event initiated by a testing program and associated with the first testing of the first software at a testing server, the test event associated with a change to code in the first software and detected by an agent executing within the testing program at the testing server, the first testing initiated by the testing program and associated with the plurality of tests in the first testing of the first software; receiving one or more files associated with the change to the code of the first software; parsing the one or more files to identify one or more methods of the plurality of methods associated with the change to the code, the identified one or more methods associated with the changed code; generating a subset of tests in response to the detection of the test event by the agent and based on the identified one or more methods associated with the changed code, wherein the subset of tests being a subset a subset of the plurality of tests in the first testing of the first software, and wherein the subset of tests generated based on the initial call graph having the relationships between the plurality of tests and the identified one or more methods associated with the changed code, wherein the relationships of the initial call graph were previously updated based on results of a previously executed testing of the first software, such that a portion of the initial call graph relating to tests executed in the previously executed testing reflects the results of the previously executed testing while a portion of the initial call graph relating to tests not executed in the previously executed testing is unchanged; the subset list of tests to be performed in response to the test event; the subset of tests having fewer tests than the plurality of tests associated with the first testing initiated by the testing program; and automatically updating a portion of the initial call graph relating to the ordered tests based on results of the executed ordered tests, and not updating a portion of the initial graph related to the plurality of tests that are not included in the ordered tests.
However, in an analogous art, Goswami discloses:
generating an initial call graph for a plurality of methods within a first software to be tested in a first testing, the initial call graph including relationships between the methods and each of a plurality of tests in the first testing, each of the relationships associating one of the plurality of tests with one or more of the methods (e.g., Goswami, Fig. 4 and associated text, par. [0039]: a code analyzer generates a graph. A plurality of test cases are run to identify methods [plurality of methods] in the code base that are covered by such test case. Next at 530 the call graph is supplemented with test case nodes representing each of a plurality of available test cases [plurality of tests]. The test case nodes are coupled to method nodes by edges that correspond to coverage of the method by the test case corresponding to the connected test case node; par. [0024]: the anomaly handler 120 can run a complete test suite [running the complete test suite, i.e., running all test cases, is the first testing] in cases where the number of implicated test cases is above pre-defined threshold)
receiving one or more files associated with the change to the code of the first software; (e.g., Goswami, par. [0026]: the code change analyzer 110 can parse a commit. The code change analyzer 110 provides the files(s) that are modified. The code set of files are passed to the code analyzer 114 for further analysis. The code analyzer 113 can return the changes in terms of methods; par [0039]: a code base is parsed to identify methods having changes in a code base; par. [0028]: the source code parser takes source code as input data)
parsing the one or more files to identify one or more methods of the plurality of methods associated with the change to the code, the identified one or more methods associated with the changed code (see immediately above).
generating a subset of tests based on the identified one or more methods associated with the changed code, (see immediately below)
wherein the subset of tests being a subset of the plurality of tests in the first testing of the first software, and wherein the subset of tests generated based on the initial call graph the relationships between the plurality of test and the identified one or more methods associated with the changed code (e.g., Goswami, Fig.4 and associated text, par. [0039]: a call graph in which there are method nodes representing each method in a code base. The call graph is supplemented with test case nodes representing each of a plurality of available test cases [plurality of tests]. The test case nodes are coupled to method nodes by edges. The call graph is traversed to identify test cases [subset of tests] implicated by the identified methods having changes in the code base; par. [0011]: the subject matter reduces an amount of test cases to be executed; par. [0024]: the anomaly handler 120 can run a complete test suite in cases where the number of implicated test cases is above pre-defined threshold)
the subset of tests having fewer tests than the plurality of tests associated with the first testing initiated by the testing program (see immediately above).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the software to be tested, remote server and subset of tests in a list taught by Griffin by incorporating generating an initial call graph for a plurality of methods within the software to be tested in a first testing, the initial call graph including relationships between the methods and each of a plurality of tests in the first testing, each of the relationships associating one of the plurality of tests with one or more of the methods; receiving one or more files associated the code of the first software to be changed; parsing the one or more files to identify one or more methods of the plurality of methods associated with the change to the code, the identified one or more methods associated with the changed code, generating the subset of tests based on the identified one or more methods associated with the changed code, wherein the subset of tests being a subset a subset of the plurality of tests in the first testing of the first software, and wherein the subset of tests generated based on the initial call graph having the relationships between the plurality of tests and the identified one or more methods associated with the changed code, as taught by Goswami, as Goswami would provide the advantages of a means of executing only the tests implicated by methods having changes and a means avoiding the execution of unnecessary tests. (See Goswami, par. [0039], [0002]).
Further, in an analogous art, Farrier discloses:
detecting a test event initiated by a testing program and associated with the first testing of the first software at a testing server, the test event associated with a change to code in the first software and detected by an agent executing within the testing program at the testing server, the first testing initiated by the testing program and associated with the plurality of tests in the first testing of the first software; (e.g., Farrier, par. [0029]: software [a testing program] performs all of the steps of the present invention; par. [0031]: software described herein may be disclosed as operating on one computing device “(e.g., a dedicated server [testing server] or a workstation)”; par. [0050]: a CI server or other application [agent within a testing program] monitoring the code repository; par. [0046]: a server overseeing a code repository transmits a function call or other message [initiates a test event] to notify a system of the invention that a new commit has taken place [whatever portion of code receives this message being the agent]; par. [0034]: the term “commit” refers to a changes made to one or more software files; par. [0045]: a method selectively runs tests. Such methods in most cases eliminate the need to run most tests in the test suite [all tests in the test suite being the plurality of tests in the first testing])
generating, a subset list of tests in response to the detection of the test event by the agent; (e.g., Farrier, par. [0046]: the invention uses a combination of webhook data, API call or agent executing commands on the repository to get the commit metadata to determine which files or lines of code have changed. The system may then determine which tests to run based on the changes and send the list to execute the selected tests) and
the subset list of tests to be performed in response to the test event (see immediately above).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the list of tests and agent detecting test events and executing tests of Griffin/Goswami to include detecting a test event initiated by a testing program and associated with first testing of a first software at a testing server, the test event associated with a change to code in the first software and detected by an agent executing within the testing program at the testing server, the first testing initiated by the testing program and associated with the plurality of tests in the first testing of the first software; generating, a subset list of tests in response to the detection of the test event by the agent, and the subset list of tests to be performed in response to the test event, as taught by Farrier, as Farrier would provide the advantage of a means of automatically initiating testing of changed code whenever the code is changed (See Farrier, par. [0050])
Finally, in analogous art, Zhou discloses:
wherein the relationships of the initial call graph were previously updated based on results of a previously executed testing of the first software, such that a portion of the initial call graph relating to tests executed in the previously executed testing reflects the results of the previously executed testing while a portion of the initial call graph relating to tests not executed in the previously executed testing is unchanged (e.g., Zhou, col. 3 ll. 20-23: a set of test cases for testing that component can be identified. This avoids running test cases not related to the selected components; col. 3 ll. 40-45: information may be used to order test cases for execution; col. 5 ll. 5-27: as the test cases are exercised, the graph of FIG. 1 can be continually updated. For example, test case 104a may be executed, and each function, method, module or other element of the software that it covers “(e.g., source files 102b, 102c)” may be tagged. After test case 104a finishes executing, edges between test case 104a and the tested source files can be created. Further, the node for test case 104a can be updated with information such as the identities of source files 102b, 102c, etc. The nodes for source files 102b, 102c are updated to identify test case 104a [note that only the portions of the graph relating to the executed test are updated. The other portions are thus unchanged]. Then the tags are cleared and the next case can be executed. The graph can this be generated and/or updated over time, as different test cases are executed)
automatically updating a portion of the initial call graph relating to the ordered tests based on results of the executed ordered tests, and not updating a portion of the initial call graph related to the plurality of tests that are not included in the ordered tests (see immediately above, nodes and edges of executed test cases are updated, not nodes and edges of the test cases that are not).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the initial call graph of Griffin/Goswami, such that the relationships of the initial call graph were previously updated based on results of a previously executed testing of the first software, such that a portion of the initial call graph relating to tests executed in the previously executed testing reflects the results of the previously executed testing while a portion of the initial call graph relating to tests not executed in the previously executed testing is unchanged and the initial call graph is previously updated based on a previously executed testing of the first software and automatically updating a portion of the initial call graph relating to the ordered tests based on results of the executed ordered tests, and a portion of the initial call graph related to the plurality of tests that are not included in the ordered tests is not updated, as taught by Zhou, as Zhou would provide the advantage of a means of ensuring the graph reflects coverage of the executed subset of tests. (See Zhou, col. 5 ll. 4-7).
As to claim 2, Griffin/Goswami/Farrier/Zhou renders obvious the method of claim 1 (see rejection of claim 1 above), Griffin further discloses:
further comprising generating, by a score generator stored in memory and executed by a processor on an intelligence server, a likelihood of failure for each test, wherein the ordering is based on the likelihood of failure for each test (e.g., Griffin, col. 5 ll. 23-26: test ordering service 220 [a service being software, so it necessarily stored in memory and executed by a processor] may be implemented using one or more computing devices [intelligence servers] and accessible over a network; in FIG. 6; col. 5 ll. 30-35: test ordering service 220 may use the functionality for test failure likelihood estimation 121 to generate ordered sequences of tests; col. 6 ll. 55-60: to score each of the tests 115-115C, the functionality for test failure likelihood estimation 121 may use a plurality of test scoring plugins 150; col. 7 ll. 65-67: the tests may be sorted by their score for likelihood of failure to generate the ordered sequence).
As to claim 3, Griffin/Goswami/Farrier/Zhou renders obvious the method of claim 2 (see rejection of claim 1 above), Griffin further discloses:
wherein the score generator predicts the likelihood of failure based on historical data (e.g., Griffin, col. 6 ll. 55-60: to score each of the tests 115-115C, the functionality for test failure likelihood estimation 121 may use a plurality of test scoring plugins 150; col. 7 ll. 1-5: failure rate plugin 152 scores a test according to its failure rate, e.g., the failures in its portion of the test execution history; col. 7 ll. 65-67: the tests may be sorted by their score for likelihood of failure to generate the ordered sequence; col. 2 ll. 22-25: when the testing process is initiated, tests with an estimated likelihood of failure may be performed earlier [the score for likelihood of failure is a prediction because the tests have not been executed yet]).
As to claim 4, Griffin/Goswami/Farrier/Zhou renders obvious the method of claim 2 (see rejection of claim 1 above), Griffin further discloses:
wherein the score generator predicts the likelihood of failure based on source code change data (e.g., Griffin, col. 7 ll. 17-22: test scoring plugins 150 may include a source code modification plugin 154 that scores a test according to the age of any modification to its source code).
As to claim 7, Griffin/Goswami/Farrier/Zhou renders obvious the method of claim 1 (see rejection of claim 1 above), Griffin further discloses
wherein the duration of execution of the subset of tests in the test list is shorter than the duration of execution of the plurality of tests (e.g., Griffin, col. 3 ll. 11-23: the set of tests may be selected from a larger set of potential tests 115 [executing a subset of tests is necessarily of shorter duration than the full set]; col. 2 ll. 23-30: when the testing process is initiated using the ordered sequence, tests with a higher estimated likelihood of failure may be performed earlier. When one or more failures are encountered during the testing process the testing may be stopped [so execution of the ordered tests stops earlier, i.e., test execution is of shorted duration]).
Griffin/Goswami/Farrier/Zhou does not explicitly disclose wherein a new call graph is automatically generated after test completion.
However, in an analogous art, Zhou discloses:
wherein a new call graph is automatically generated after test completion (e.g., Zhou, col. 5 ll. 5-27: as the test cases are exercised, the graph of FIG. 1 can be continually updated [updating an existing graph being generating a new one]. For example, test case 104a may be executed, and each function, method, module or other element of the software that it covers “(e.g., source files 102b, 102c)” may be tagged. After test case 104a finishes executing, edges between test case 104a and the tested source files can be created. Further, the node for test case 104a can be updated with information such as the identities of source files 102b, 102c, etc. The nodes for source files 102b, 102c are updated to identify test case 104a. Then the tags are cleared and the next case can be executed. The graph can this be generated and/or updated over time, as different test cases are executed).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the call graph and first software of Griffin/Goswami such that a new call graph is automatically generated after test completion, as taught by Zhou, as Zhou would provide the advantage of a means of ensuring the graph reflect changes in the portions of software or test cases as the test cases are executed. (See Zhou, col. 5 ll. 4-7).
As to claim 8, it is a medium claim having limitations substantially the same as claim 1, which are obvious in view of the prior art for substantially the same reasons. .
Further limitations, disclosed by Griffin, include:
a non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for automatically testing software code, (e.g., Griffith, Fig. 6 and associated text) the method comprising: (see rejection of claim 1 above).
As to claim 9, it is a medium claim whose limitations are substantially the same as those of claim 2. Accordingly, it is rejected for substantially the same reasons.
As to claim 10, it is a medium claim whose limitations are substantially the same as those of claim 3. Accordingly, it is rejected for substantially the same reasons.
As to claim 11, it is a medium claim whose limitations are substantially the same as those of claim 4 Accordingly, it is rejected for substantially the same reasons.
As to claim 14, it is a medium claim whose limitations are substantially the same as those of claim 7. Accordingly, it is rejected for substantially the same reasons.
As to claim 15, it is a system claim having limitations substantially the same as claim 1, which are obvious in view of the prior art for substantially the same reasons.
Further limitations, disclosed by Griffin, include:
a system for automatically testing software code, comprising:
a testing server including a first memory and a first processor; (e.g., Griffin, Figs. 2, 6 and associated texts, col. 2 ll. 29-51: the test execution system 100 may comprise one or more computing devices [testing servers], any of which may be implemented by the example computing device 3000 in FIG. 6; col. 9 ll. 1-2: computing device 3000 may by a uniprocessor system including one processor 3010; col. 9 ll. 55-60: system memory 3020 may store program instructions as described above for implementing embodiments)
a remote server including a second memory and a second processor (e.g., Griffin, Figs. 2, 6 and associated texts col. 5 ll. 22-28: the test ordering service may be implemented by one or more computing devices [remote servers], any of which may be implemented by the example computing device 3000 in FIG. 6 [so a second memory and processor]. The ordering service 220 may be external to the test execution system, e.g., implemented using different computing devices and accessible to the test execution system 100 over a network) and
one or more modules stored in the first memory and executed by the first processor (e.g., Griffin, Fig. 6 and associated text) to (see rejection of claim 1 above).
As to claim 16, it is a system claim whose limitations are substantially the same as those of claim 2. Accordingly, it is rejected for substantially the same reasons.
As to claim 17, it is a system claim whose limitations are substantially the same as those of claim 3. Accordingly, it is rejected for substantially the same reasons.
As to claim 18, it is a system claim whose limitations are substantially the same as those of claim 4 Accordingly, it is rejected for substantially the same reasons.
Claims 5, 6, 12, 13, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Griffin (US 9,514,034) in view of Goswami (US 2022/0188215) in view of Farrier (US 2021/0263728) in view of Zhou (US 7,480,900) in further view of Jagannath (US 2020/0250078).
As to claim 5, Griffin/Goswami/Farrier/Zhou discloses the method of claim 4 (see rejection of claim 1 above), but does not explicitly disclose wherein the source code change data includes a source code change time, files that were added or modified to the source code, level of change made to the software, and who made the change to the source code.
However, in an analogous art, Jagannath discloses:
wherein the source code change data includes a source code change time, files that were added or modified to the source code, level of change made to the software, and who made the change to the source code (e.g., Jagannath, par. [0012]: changes to component such as source code; par. [0019]: a feature of a component has been altered; par. [0027]: the attributes of the altered feature 203, such as an author of the altered feature [who made the change to the source code], a nature of the altered feature “(such as a minor bug fix, newly added software feature, major revision, etc.)” [minor bug fix or major revision being a level of change, a newly added feature being a file that was added to the source code], a date/or time of the creation of the altered feature [source code change time]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the source code change data of Griffin/Goswami/Farrier/Zhou to include a source code change time, files that were added or modified to the source code, level of change made to the software, and who made the change to the source code, as taught by Jagannath, as Jagannath would provide the advantage of a means of correlating that data with testing failures. (See Jagannath, pars. [0033-0034]).
As to claim 6, Griffin/Goswami/Farrier/Zhou discloses the method of claim 1 (see rejection of claim 1 above), but does not explicitly disclose wherein ordering each test includes: training a score generator using training data associated with the source code being tested; and applying test data to the trained score generator to generate the likelihood of failure for each test in the test list.
However, in analogous art, Jagannath discloses
wherein ordering each test (see below) includes:
training a score generator using training data associated with the source code being tested; (e.g., Jagannath, par. [0031]: tests having higher probabilities of failure will be placed earlier in the dynamic test order than tests having lower probabilities of failure; par. [0012]: the probabilities [scores] may be generated based on machine-learned models that may be trained using results of prior testing outcomes) and
applying test data to the trained score generator to generate the likelihood of failure for each test in the test list (e.g., Jagannath, par. [0027]: apparatus 100 may use the attributes [test data] as input parameters to a machine-learned model of test procedures to generate, for each test procedure, a probability that the altered feature will fail; par. [0048]: test outcomes [also test data] may be used to re-train the machine learned model).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the ordering of each test of Griffin/Goswami/Farrier/Zhou to include training a score generator using training data associated with the source code being tested and applying test data to the trained score generator to generate the likelihood of failure for each test in the test list as taught by Jagannath, as Jagannath would provide the advantages of a means of ordering the tests via machine learning and a means to continuously re-train the machine learning process. (See Jagannath, pars. [0012], [0048])
As to claim 12, it is a medium claim whose limitations are substantially the same as those of claim 5. Accordingly, it is rejected for substantially the same reasons.
As to claim 13, it is a medium claim whose limitations are substantially the same as those of claim 6. Accordingly, it is rejected for substantially the same reasons.
As to claim 19, it is a system claim whose limitations are substantially the same as those of claim 5. Accordingly, it is rejected for substantially the same reasons.
As to claim 20, it is a system claim whose limitations are substantially the same as those of claim 6. Accordingly, it is rejected for substantially the same reasons.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TODD AGUILERA whose telephone number is (571)270-5186. The examiner can normally be reached M-F 11AM - 7:30PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hyung S Sough can be reached at (571)272-6799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TODD AGUILERA/Primary Examiner, Art Unit 2192
Prosecution Insights