DETAILED ACTION
Remarks
This Office action is in response to the communication filed 30 March 2026, which was filed in response to the 12 January 2026 non-final Office action (the “Previous Action”).
Claims 1, 8 and 15 are amended, as well as paragraphs [0030-0031] of the specification.
Claims 1-20 are pending. Claims 1, 8 and 15 are the independent claims.
Any unpersuasive arguments are addressed in the “Response to Arguments” section below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments/Amendments
With respect to the § 112 rejections, Applicant argues that claims 1, 8 and 15 have been amended to obviate the rejections.
Examiner respectfully disagrees and points out that claim 15 still includes ambiguous references to “the” memory and processor and the rejection on that basis is accordingly maintained.
With respect to the double patenting rejections, Applicant provides no argument but requests that it be held in abeyance “until such time claims are deemed allowable over the cited art.” (Remarks, p. 10 par 2).
The double patenting rejections are accordingly maintained.
With respect to the § 103 rejection of claim 1, Applicant makes various arguments.
First, Applicant argues that Goswani does not teach the call graph “including relationships between the methods and each of a plurality of tests in the first testing” because, according to Applicant, the call graph of Goswani “only includes the method nodes without showing any relationships between the methods and the tests.” (Remarks, p. 10 par. 2 – p. 11 par. 1).
Examiner respectfully disagrees. Goswani explicitly discloses that the call graph is “supplemented with test cases nodes” and that “the test case nodes are each coupled to one or more method nodes by unidirectional edges” in paragraph [0039]. These nodes and edges are also illustrated by Goswani in Figure 4.
Applicant then argues that Goswani does not teach or suggest “parsing the one or more files to identify methods associated with the changed code” because, according to Applicant, Goswani “parses the entire code base” as opposed to files associated with the change to the code. (Remarks. p. 11 pars. 4-5).
Examiner respectfully disagrees and submits that parsing the entire code base would still include parsing the files associated with the change since some files of the code base have changed. Goswani also discloses in paragraph [0026] that change analyzer 110 “can parse a commit” to provide “the file(s) that are modified”, which are then passed to code analyzer 114 for further analysis.
Applicant argues that Goswani does not teach generating the subset list of tests “in response to detection of the test.” (Remarks, p. 11 last par. – p. 12 par. 1).
Examiner respectfully submits that Goswani was not cited as teaching these features, Farrier was. One cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Applicant argues that Farrier does not teach or suggest detecting a test event initiated “by a testing program.” (Remarks, p. par. 4).
Examiner respectfully disagrees and submits that in Farrier, software performs all the steps of the invention, including the cited transmission of messages [initiation of test events]. (See Farrier, par. [0029]). That software is a testing program because the invention includes selectively running tests. (See Farrier at par. [0029]).
Applicant’s arguments are thus unpersuasive.
Applicant’s arguments with respect to the remaining claims by virtue of their dependence from claim 1, similarity with claim 1 or dependence from a similar claim are unpersuasive for the same reasons.
Information Disclosure Statement
Applicant appears to have filed the same information disclosure statement (IDS) on 29 January 2026 twice. Though one copy is marked to indicate that it has not been considered, the information therein actually has been considered via the other document.
Specification
The Previous Action’s objections to the specification are withdrawn in view of Applicant’s specification amendments.
Claim Objections
The Previous Action’s claim objections are withdrawn in view of Applicant’s claim amendments unless reproduced herein.
The objection to claim 15 for capitalizing certain claim language set forth at p. 4 item j of the Previous Action was in error and is withdrawn.
Claims 8-20 are objected to for the following informalities:
Claim 8 capitalizes the word “Generating” at line 4, which appears to be a typographical error.
Claims 9-14 are objected to via dependence from claim 8.
Claim 15 refers to “a the initial call graph” at lines 19-20, which appears to be a typographical error that should perhaps read -the initial call graph- instead.
Claims 16-20 are objected to via dependence from claim 15.
Claim Rejections - 35 USC § 112
The Previous Action’s § 112 rejections are withdrawn in view of Applicant’s amendments unless reproduced herein.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-7 and 15-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
As to claim 1, the claim refers to “the initial call graph having relationships between the plurality of tests and the change to the code of the first software” at lines 17-20. There is insufficient antecedent basis for these limitations in the claim, as the claim only previously refers to a call graph including relationships between each of a plurality of tests and “methods” within a first software. (See claim 1 at lines 3-5). It is accordingly unclear what graph lines 17-20 is referring to. For the purposes of examination, lines 17-20 will be construed as referring to the same graph of lines 3-5.
As to claims 2-7, the claim is dependent on claim 1 but does not cure the deficiencies of that claim. Accordingly, it is rejected for the same reasons.
As to claim 15, the claim refers to “the” memory and “the processor at line 4. However, the claim previously refers to “a” memory and processor at line 2 and a “second” memory and processor at line 3. It is accordingly unclear to which memory and processor “the” memory and processor of line 4 is referring. For the purposes of examination, “a memory and a processor” at line 2 of the claim will be construed as -a first memory and a first processor- and “the memory and executed by the processor” at line 4 will be construed as -the first and second memory and executed by the first and second processor-.
As to claims 16-20, the claims are dependent on claim 15 but do not cure the deficiencies of the claim. Accordingly, they are rejected for the same reasons.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1, 8, and 15 and are compared to claims 1 and 2 of Patent No. 12,399,806 in the following table:
Instant Application
Patent No. 12,399,806
Claim 1 – A method for automatically testing software code, comprising:
generating an initial call graph for a plurality of methods within a first software to be tested in a first testing, the call graph including relationships between the methods and each of a plurality of tests in the first testing
detecting a test event initiated by a testing program and associated with the first testing of the first software at a testing server, the test event associated with a change to code detected by an agent executing within the testing program at the testing server, the first testing initiated by the testing program and associated with the plurality of tests in the first testing of the first software,
receiving one or more files associated with the change to the code of the first software;
parsing the one or more files to identify methods associated with the changed code;
generating, by a remote server, a subset list of tests in response to the detection of the test event by the agent and based on the identified methods associated with the changed code, the subset list of tests being a subset of the plurality of tests in the first testing of the first software, the subset list of tests generated based on the initial call graph having relationships between the plurality of tests and the change to the code of the first software, the initial call graph previously updated based on a previously executed testing of the first software
receiving, by the agent on the testing server from the remote server, the subset list of tests to be performed in response to the test event, the received subset list of tests being a subset of the plurality of tests;
ordering each test in the subset list of tests according to a likelihood of failure determined for each test in the subset of tests;
executing the ordered tests of the subset of tests by the agent in the testing server the subset of tests having fewer tests than the plurality of tests associated with the first testing initiated by the testing program
automatically updating a portion of the first call graph relating to the ordered tests based on results of the executed ordered tests, and not updating a portion of the call graph related to the plurality of tests that are not included in the ordered tests.
Claim 1 – A method for automatically testing software code, comprising:
…a call graph…
Claim 2 – wherein the first call graph includes relationships between the plurality of tests and portions of the first software.
…detecting a test event initiated by a testing program and associated with testing a first software at a testing server, the test event detected by an agent separate from the testing program and executing within the testing program at the testing server, the testing event associated with a plurality of tests generated by the testing program for the first software…
…changed code within the first software…
…accessing one or more files associated with changed code…
…receiving…from the remote server, a list of tests to be performed in response to the test event, the…list of tests being a subset of the plurality of tests and generated at least in part from a first call graph…
Claim 2 – The method of claim 1, wherein the first call graph includes relationships between the plurality of tests and portions of the first software.
Claim 1 --
….receiving, by the agent on the testing server from the remote server, a list of tests to be performed in response to the test event, the received list of tests being a subset of the plurality of tests…
…the received list of tests being a subset of the plurality of tests…
…executing the tests in the list of tests by the agent in the testing server.
Claim 8 – A non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for automatically testing software code, the method comprising:
generating an initial call graph for a plurality of methods within a first software to be tested in a first testing, the call graph including relationships between the methods and each of a plurality of tests in the first testing
detecting a test event initiated by a testing program and associated with the first testing of the first software at a testing server, the test event associated with a change to code detected by an agent executing within the testing program at the testing server, the first testing initiated by the testing program and associated with the plurality of tests in the first testing of the first software,
receiving one or more files associated with the change to the code of the first software;
parsing the one or more files to identify methods associated with the changed code;
generating, by a remote server, a subset list of tests in response to the detection of the test event by the agent and based on the identified methods associated with the changed code, the subset list of tests being a subset of the plurality of tests in the first testing of the first software, the subset list of tests generated based on the initial call graph having relationships between the plurality of tests and portions of the first software, the initial call graph previously updated based on a previously executed testing of the first software
generating, by a remote server, a list of tests in response to the detection of the test event by the agent, the list of tests generated based on a first call graph having relationships between the plurality of tests and portions of the first software;
receiving, by the agent on the testing server from the remote server, the subset list of tests to be performed in response to the test event, the received subset list of tests being a subset of the plurality of tests;
ordering each test in the subset list of tests according to a likelihood of failure determined for each test in the subset of tests;
executing the ordered tests of the subset of tests by the agent in the testing server the subset of tests having fewer tests than the plurality of tests associated with the first testing initiated by the testing program
automatically updating a portion of the first call graph relating to the ordered tests based on results of the executed ordered tests, and not updating a portion of the call graph related to the plurality of tests that are not included in the ordered tests.
Claim 1 – A method for automatically testing software code, comprising:
…a call graph…
Claim 2 – wherein the first call graph includes relationships between the plurality of tests and portions of the first software.
detecting a test event initiated by a testing program and associated with testing a first software at a testing server, the test event detected by an agent separate from the testing program and executing within the testing program at the testing server, the testing event associated with a plurality of tests generated by the testing program for the first software…
…changed code within the first software…
…accessing one or more files associated with changed code…
…receiving…from the remote server, a list of tests to be performed in response to the test event, the…list of tests being a subset of the plurality of tests and generated at least in part from a first call graph…
Claim 2 – The method of claim 1, wherein the first call graph includes relationships between the plurality of tests and portions of the first software.
…a list of tests to be performed in response to the test event, the received list of tests…generated at least in part from a first call graph…
Claim 2 – The method of claim 1, wherein the first call graph includes relationships between the plurality of tests and portions of the first software.
Claim 1 --
….receiving, by the agent on the testing server from the remote server, a list of tests to be performed in response to the test event, the received list of tests being a subset of the plurality of tests…
…the received list of tests being a subset of the plurality of tests…
…executing the tests in the list of tests by the agent in the testing server.
Claim 15 -- A system for automatically testing software code, comprising:
a testing server including a memory and a processor;
a remote server including a second memory and a second processor;
one or more modules stored in the memory and executed by the processor to detect a test event initiated by a testing program and associated with first testing of a first software at the testing server, the test event detected by an agent executing within the testing program at the testing server, the first testing initiated by the testing program and associated with a plurality of tests in the first testing of the first software,
receive one or more files associated the code the first software to be changed,
parse the one or more files to identify methods associated with the changed code,
receive, by the agent on the testing server from remote server, subset list of tests to be performed in response to the test event,
the received subset list of tests generated by a remote server in response to the detection of the test event by the agent and based on the identified methods associated with the changed code,
the subset list of tests being a subset of the plurality of tests in the first testing of the first software, the subset list of tests generated based on a the initial call graph having relationships between the plurality of tests and portions of the first software, the received subset list of tests being a subset of the plurality of tests,
order each test in the subset list of tests according to a likelihood of failure determined for each test in the subset of tests,
execute the ordered tests of the subset of tests by the agent in the testing server, the subset of tests having fewer tests than the plurality of tests associated with the first testing initiated by the testing program,
automatically update a portion of the first call graph relating to the ordered tests and not updating a portion of the call graph related to the plurality of tests that are not included in the ordered tests.
Claim 1 – A method for automatically testing software code, comprising:
…a testing server…
…a remote server…
…detecting a test event initiated by a testing program and associated with testing a first software at a testing server, the test event detected by an agent separate from the testing program and executing within the testing program at the testing server, the testing event associated with a plurality of tests generated by the testing program for the first software…
…accessing one or more files associated with changed code…
…receiving…from the remote server, a list of tests to be performed in response to the test event, the…list of tests being a subset of the plurality of tests and generated at least in part from a first call graph…
……receiving…from the remote server, a list of tests to be performed in response to the test event, the…
…the test event detected by an agent…
…the list of tests being a subset of the plurality of tests and generated at least in part from a first call graph…
Claim 2 – The method of claim 1, wherein the first call graph includes relationships between the plurality of tests and portions of the first software.
…the received list of tests being a subset of the plurality of tests…
…executing the tests in the list of tests by the agent in the testing server.
It is also noted that the present application and the issued claims share the same assignee and inventors.
Claims 1-4, 7-11 and 14-18 are rejected on the grounds of nonstatutory obviousness-type double patenting as being unpatentable over claim 2 of issued patent no. 12,399,806 in view of Goswami et al. (US 2022/0188215) (art of record – hereinafter Goswami), Griffin et al. (US 9,514,034) (art of record – hereinafter Griffin) and Zhou et al. (US 7,480,900) (art of record – hereinafter Zhou).
As to claims 1, 8 and 15, although the conflicting claims are not identical, they are not patentably distinct.
Note with respect to claim 8 that while the issued claims do not explicitly disclose a non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform the method or a server including a memory and a processor, such features are inherent in the issued claims because they refer to execution within a testing program at a testing server.
Note too with respect to claim 15 that while the issued claims do disclose that the testing and remote server each comprise a memory and a processor, such features are inherent in the testing and remote server of the issued claims. Similarly, one or more modules stored in the memory and executed by the processor to perform the method are inherent in the claimed method because the method would be understood by persons of ordinary skill as describing the execution of software.
And note finally with respect to claims 1, 8 and 15 that while the issued claims do not explicitly disclose the list of tests is generated by a remote server, receiving the lists of tests from a remote server implies they are generated there, at least because they have been stored there.
Thus, the issued claims teach every element of the instant claims as set forth in the above table except an initial call graph for a plurality of methods within a first software, the call graph including relationships between the methods and each of a plurality of tests, an agent executing within the testing program, parsing the one or more files to identify methods associated with the changed code; generating a subset of tests based on the identified methods associated with the changed code; the subset of tests generated based on the initial call graph having relationships between the plurality of tests and the change to the code of the first software (as in claim 1, or between the plurality of tests and “portions of the first software” as in claim 8 and 15), the initial call graph previously updated based on a previously executed testing of the first software, ordering each test in the subset list of tests according to a likelihood of failure determined for each test in the subset of tests; ordered tests; and automatically updating a portion of the first call graph relating to the ordered tests based on results of the executed ordered tests, and not updating a portion of the call graph related to the plurality of tests that are not included in the ordered tests.
However, in an analogous art, Goswami discloses:
an initial call graph for a plurality of methods within a first software, the call graph including relationships between the methods and each of a plurality of tests,
parsing the one or more files to identify methods associated with the changed code;
generating a subset of tests based on the identified methods associated with the changed code; and
the subset of tests generated based on the initial call graph having relationships between the plurality of tests and the change to the code of the first software [as in claim 1, or between the plurality of tests and portions of the first software as in claim 8 and 15].
as set forth in the § 103 rejections below.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the call graph and method or system of the issued claims to include an initial call graph for a plurality of methods within a first software, including relationships between the methods and each of a plurality of tests; parsing the one or more files to identify methods associated with the changed code; generating a subset of tests based on the identified methods associated with the changed code, the subset list of tests generated based on the initial call graph having relationships between the plurality of tests and the change to the code of the first software or portions of the first software, as taught by Goswami, as Goswami would provide the advantages of a means of executing only the tests implicated by methods having changes and a means avoiding the execution of unnecessary tests. (See Goswami, par. [0039], [0002]).
Further, in analogous art, Griffin discloses:
an agent executing within the testing program
ordering each test in the subset list of tests according to a likelihood of failure determined for each test in the subset of tests;
ordered tests;
as set forth in the § 103 rejections below.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the agent of the issued claims such that it executes within the testing program, as taught by Griffin. An agent at a testing server having a testing program can either be within the testing program or separate from it, and as shown by Griffin, the agent can be located within the testing program and successfully perform its functions. (See M.P.E.P. § 2143(I)(E) and US 9,569,341 at col. 13 ll. 55-61).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the list of tests of the issued claims by reordering them according to a likelihood of failure in substitution of their original order and executing them in the reordered sequence as taught by Griffin, as Griffin would provide the advantage of a means of providing earlier testing feedback. (See Griffin, col. 2 ll. 25-30).
Finally, in an analogous art, Zhou discloses:
the call graph previously updated based on a previously executed testing of the first software and
automatically updating a portion of the first call graph relating to the ordered tests based on results of the executed ordered tests, and not updating a portion of the call graph related to the plurality of tests that are not included in the ordered tests
as set forth in the § 103 rejections below.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the call graph and first software of the issued claims to include call graph is previously updated based on a previously executed testing of the first software and updating only a portion of the call graph relating to the ordered tests based on results of the executed ordered tests, and not a portion of the call graph related to the plurality of tests that are not included in the ordered tests, as taught by Zhou, as Zhou would provide the advantage of a means of continually updating the graph to reflect changes in the portions of software or test cases as the test cases are executed. (See Zhou, col. 5 ll. 4-7).
As to claim 2, the issued claims in view of Goswami/Grifffin/Zhou disclose every element of claim 1 but do not explicitly disclose features of this claim.
However, Griffin does, as set forth in the § 103 rejections below.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the list of tests of the issued claims to include a score generator stored in memory and executed by a processor on an intelligence server generating a likelihood of failure for each test and ordering based on that likelihood, as taught by Griffin, as Griffin would provide the advantage of a means of providing earlier testing feedback. (See Griffin, col. 2 ll. 25-30).
As to claim 3, the issued claims in view of Goswami/Grifffin/Zhou disclose every element of claim 2 but do not explicitly disclose features of this claim.
However, Griffin does, as set forth in the § 103 rejections below.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the list of tests of the issued claims by a score generator predicts the likelihood of failure based on historical data, as taught by Griffin, as Griffin would provide the advantage of a means of optimizing the ordered sequence of tests. (See Griffin, col. 6 ll. 25-30).
As to claim 4, the issued claims in view of Goswami/Grifffin/Zhou disclose every element of claim 2 but do not explicitly disclose features of this claim.
However, Griffin does, as set forth in the § 103 rejections below.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the list of tests of the issued claims by including a score generator predicts the likelihood of failure based on source code change data, as taught by Griffin, as Griffin would provide the advantage of a means of taking into account the age of the code changes. (See Griffin, col. 7 ll. 17-22).
As to claim 7, the issued claims do not disclose that the duration of execution of the subset of tests in the test list is shorter than the duration of execution of the plurality of tests but these features are inherent in the issued claims because the subset of the issued claims is a subset of the plurality and a subset of tests will necessarily take less time to execute than executing a larger set of tests that includes that subset.
Thus, the issued claims disclose every feature of claim 7 except wherein a new call graph is automatically generated after test completion.
However, in an analogous art, Zhou discloses these features as set forth in the § 103 rejections below.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the call graph and first software of the issued claims such that a new call graph is automatically generated after test completion, as taught by Zhou, as Zhou would provide the advantage of a means of continually updating the graph to reflect changes in the portions of software or test cases as the test cases are executed. (See Zhou, col. 5 ll. 4-7).
As to claims 9-11 and 14, they are medium claims whose limitations are substantially the same as claims 2-4 and 7 and are rejected for substantially the same reasons.
As to claims 16-18 they are system claims whose limitations are substantially the same as claims 2-4 and are rejected for substantially the same reasons.
Claims 5, 6, 12, 13, 19 and 20 are rejected on the grounds of nonstatutory obviousness-type double patenting as being unpatentable over claim 2 of issued patent no. 12,399,806 in view of Goswami (US 2022/0188215), Griffin (US 9,514,034) and Zhou (US 7,480,900) in further view of Jagannath et al. (US 2020/0250078) (art of record – hereinafter Jagannath)
As to claim 5, the issued claims in view of Goswami/Griffin/Zhou render obvious every element of claim 4 but do not explicitly disclose features of this claim.
However, Jagannath does, as set forth in the § 103 rejections below.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the source code change data of the issued claims in view of Goswami/Griffin/Zhou to include a source code change time, files that were added or modified to the source code, level of change made to the software, and who made the change to the source code, as taught by Jagannath, as Jagannath would provide the advantage of a means of correlating that data with testing failures. (See Jagannath, pars. [0033-0034]).
As to claim 6, the issued claims in view of Goswami/Griffin/Zhou render obvious every element of claim 1 but do not explicitly disclose features of this claim.
However, Jagannath does, as set forth in the § 103 rejections below.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the ordering of each test of the issued claims in view of Goswami/Griffin/Zhou to include training a score generator using training data associated with the source code being tested and applying test data to the trained score generator to generate the likelihood of failure for each test in the test list as taught by Jagannath, as Jagannath would provide the advantages of a means of ordering the tests via machine learning and a means to continuously re-train the machine learning process. (See Jagannath, pars. [0012], [0048])
As to claims 12-13, they are medium claims whose limitations are substantially the same as claims 5-6 and are rejected for substantially the same reasons.
As to claims 19-20, they are system claims whose limitations are substantially the same as claims 5-6 and are rejected for substantially the same reasons.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 7-11 and 14-18 are rejected under 35 U.S.C. 103 as being unpatentable over Griffin (US 9,514,034) in view of Goswami (US 2022/0188215), Farrier et al. (US 2021/0263728) (art of record – hereinafter Farrier) and Zhou (US 7,480,900).
As to claim 1, Griffin discloses a method for automatically testing software code, comprising:
generating, by a remote server, a subset list of tests (e.g., Griffin, col. 3 ll. 11-23: the set of tests 115 may be selected from a larger set of potential tests 115; col. 3 l. 26-27 an ordered set of lists 125 that places the original set of tests 115 in the ordered sequence; Fig. 2 and associated text, col. 5 ll. 23-24: the test ordering service 220 may be implemented by one or more computing devices [remote servers]; col. 5 ll. 53-64: the test ordering service may generate the ordered set of tests 125 [subset list of tests] and return the ordered set of tests 125)
receiving, by the agent on the testing server from the remote server, the subset list of tests, (e.g., Griffin, Fig. 2 and associated text, col 5 ll. 53-64: the test ordering service may generate the ordered set of tests 125 [subset list of tests] and return the ordered set of tests 125. When test ordering service 220 returns the ordered set of tests 125 to the test execution system, the ordered set of tests 125 may be stored in the test ordering cache 140; col. 3 ll. 40-45: the test execution module [agent, or part of one] may perform the tests in the order specified in the ordered set of tests 125; col. 5 ll. 53-56: the test ordering service may generate the ordered set of tests 125 and return the ordered set of tests 125 to the test execution system 100) the received subset list of tests being a subset of the plurality of tests; (e.g., Griffin, col. 3 ll. 11-23: the set of tests 115 may be selected from a larger set of potential tests 115; col. 3 l. 26-27 an ordered set of lists 125 that places the original set of tests 115 in the ordered sequence)
ordering each test in the subset list of tests according to a likelihood of failure determined for each test in the subset of tests; (e.g., Griffin, col. 5 ll. 30-35: test ordering service 220 may use the functionality for test failure likelihood estimation 121 to generate ordered sequences of tests; col. 3 ll. at least a portion of the tests 115 may be ordered from most likely to fail to least likely to fail in the ordered set of tests 125) and
executing the ordered tests of the subset of tests by the agent in the testing server, (e.g., Griffin, Fig. 1 and associated text, col. 3 ll. 40-45: the test execution module [agent, on text execution system 100 (testing server)] may perform the tests in the order specified in the ordered set of tests 125)
Griffin does not explicitly disclose: generating an initial call graph for a plurality of methods within a software to be tested in a first testing, the call graph including relationships between the methods and each of a plurality of tests in the first testing; detecting a test event initiated by a testing program and associated with the first testing of the first software at a testing server, the test event associated with a change to code in the first software and detected by an agent executing within the testing program at the testing server, the first testing initiated by the testing program and associated with the plurality of tests in the first testing of the first software; receiving one or more files associated with the change to the code of the first software; parsing the one or more files to identify methods associated with the changed code; generating a subset of tests based on the identified methods associated with the changed code, the subset of tests being a subset a subset of the plurality of tests in the first testing of the first software, the subset of tests generated based on the initial call graph having relationships between the plurality of tests and the change to the code of the first software, the initial call graph previously updated based on a previously executed testing of the first software, the subset list of tests to be performed in response to the test event; the subset of tests having fewer tests than the plurality of tests associated with the first testing initiated by the testing program; and automatically updating a portion of the first call graph relating to the ordered tests based on results of the executed ordered tests, and not updating a portion of the call graph related to the plurality of tests that are not included in the ordered tests.
However, in an analogous art, Goswami discloses:
generating an initial call graph for a plurality of methods within a software to be tested in a first testing, the call graph including relationships between the methods and each of a plurality of tests in the first testing; (e.g., Goswami, Fig. 4 and associated text, par. [0039]: a code analyzer generates a graph. A plurality of test cases are run to identify methods [plurality of methods] in the code base that are covered by such test case. Next at 530 the call graph is supplemented with test case nodes representing each of a plurality of available test cases [plurality of tests]. The test case nodes are coupled to method nodes by edges that correspond to coverage of the method by the test case corresponding to the connected test case node; par. [0024]: the anomaly handler 120 can run a complete test suite [running the complete test suite, i.e., running all test cases, is the first testing] in cases where the number of implicated test cases is above pre-defined threshold)
receiving one or more files associated with the change to the code of the first software; (e.g., Goswami, par. [0026]: the code change analyzer 110 can parse a commit. The code change analyzer 110 provides the files(s) that are modified. The code set of files are passed to the code analyzer 114 for further analysis. The code analyzer 113 can return the changes in terms of methods; par [0039]: a code base is parsed to identify methods having changes in a code base; par. [0028]: the source code parser takes source code as input data)
parsing the one or more files to identify methods associated with the changed code (see immediately above).
generating a subset of tests based on the identified methods associated with the changed code, the subset of tests being a subset of the plurality of tests in the first testing of the first software, the subset of tests generated based on the initial call graph having relationships between the plurality of tests and the change to the code of the first software (e.g., Goswami, Fig.4 and associated text, par. [0039]: a call graph in which there are method nodes representing each method in a code base. The call graph is supplemented with test case nodes representing each of a plurality of available test cases [plurality of tests]. The test case nodes are coupled to method nodes by edges. The call graph is traversed to identify test cases [subset of tests] implicated by the identified methods having changes in the code base [if the call graph has relations between tests and methods, it has relationships between tests and changes because the methods have the changes]; par. [0011]: the subject matter reduces an amount of test cases to be executed; par. [0024]: the anomaly handler 120 can run a complete test suite [plurality of tests in the first testing] in cases where the number of implicated test cases is above pre-defined threshold)
the subset of tests having fewer tests than the plurality of tests associated with the first testing initiated by the testing program (see immediately above).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the software to be tested, remote server and subset of tests in a list taught by Griffin by incorporating generating an initial call graph for a plurality of methods within the software to be tested in a first testing, the call graph including relationships between the methods and each of a plurality of tests in the first testing; receiving one or more files associated the code of the first software to be changed; parsing the one or more files to identify methods associated with the changed code, generating the subset of tests based on the identified methods associated with the changed code, the subset of tests being a subset a subset of the plurality of tests in the first testing of the first software, the subset of tests generated based on the initial call graph having relationships between the plurality of tests and the change to the code of the first software; the subset of tests having fewer tests than the plurality of tests associated with the first testing initiated by the testing program, as taught by Goswami, as Goswami would provide the advantages of a means of executing only the tests implicated by methods having changes and a means avoiding the execution of unnecessary tests. (See Goswami, par. [0039], [0002]).
Further, in an analogous art, Farrier discloses:
detecting a test event initiated by a testing program and associated with the first testing of the first software at a testing server, the test event associated with a change to code in the first software and detected by an agent executing within the testing program at the testing server, the first testing initiated by the testing program and associated with the plurality of tests in the first testing of the first software; (e.g., Farrier, par. [0029]: software [a testing program] performs all of the steps of the present invention; par. [0031]: software described herein may be disclosed as operating on one computing device “(e.g., a dedicated server [testing server] or a workstation)”; par. [0050]: a CI server or other application [agent within a testing program] monitoring the code repository; par. [0046]: a server overseeing a code repository transmits a function call or other message [initiates a test event] to notify a system of the invention that a new commit has taken place [whatever portion of code receives this message being the agent]; par. [0034]: the term “commit” refers to a changes made to one or more software files; par. [0045]: a method selectively runs tests. Such methods in most cases eliminate the need to run most tests in the test suite [all tests in the test suite being the plurality of tests in the first testing])
generating, a subset list of tests in response to the detection of the test event by the agent; (e.g., Farrier, par. [0046]: the invention uses a combination of webhook data, API call or agent executing commands on the repository to get the commit metadata to determine which files or lines of code have changed. The system may then determine which tests to run based on the changes and send the list to execute the selected tests) and
the subset list of tests to be performed in response to the test event (see immediately above).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the list of tests and agent detecting test events and executing tests of Griffin/Goswami to include detecting a test event initiated by a testing program and associated with first testing of a first software at a testing server, the test event associated with a change to code in the first software and detected by an agent executing within the testing program at the testing server, the first testing initiated by the testing program and associated with the plurality of tests in the first testing of the first software; generating, a subset list of tests in response to the detection of the test event by the agent, and the subset list of tests to be performed in response to the test event, as taught by Farrier, as Farrier would provide the advantage of a means of automatically initiating testing whenever the code is changed (See Farrier, par. [0050])
Finally, in analogous art, Zhou discloses:
the initial call graph previously updated based on a previously executed testing of the first software (see immediately below).
automatically updating a portion of the first call graph relating to the ordered tests based on results of the executed ordered tests, and not updating a portion of the call graph related to the plurality of tests that are not included in the ordered tests (e.g., Zhou, col. 3 ll. 20-23: a set of test cases for testing that component can be identified. This avoids running test cases not related to the selected components; col. 3 ll. 40-45: information may be used to order test cases for execution; col. 5 ll. 5-27: as the test cases are exercised, the graph of FIG. 1 can be continually updated. For example, test case 104a may be executed, and each function, method, module or other element of the software that it covers “(e.g., source files 102b, 102c)” may be tagged. After test case 104a finishes executing, edges between test case 104a and the tested source files can be created. Further, the node for test case 104a can be updated with information such as the identities of source files 102b, 102c, etc. The nodes for source files 102b, 102c are updated to identify test case 104a [note that only the portions of the graph relating to the executed test are updated]. Then the tags are cleared and the next case can be executed. The graph can this be generated and/or updated over time, as different test cases are executed).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the initial call graph of Griffin/Goswami, such that the initial call graph previously updated based on a previously executed testing of the first software and automatically updating a portion of the first call graph relating to the ordered tests based on results of the executed ordered tests, and not updating a portion of the call graph related to the plurality of tests that are not included in the ordered tests, as taught by Zhou, as Zhou would provide the advantage of a means of ensuring the graph is continually updated to reflect coverage of the executed tests. (See Zhou, col. 5 ll. 4-7).
As to claim 2, Griffin/Goswami/Farrier/Zhou renders obvious the method of claim 1 (see rejection of claim 1 above), Griffin further discloses:
further comprising generating, by a score generator stored in memory and executed by a processor on an intelligence server, a likelihood of failure for each test, wherein the ordering is based on the likelihood of failure for each test (e.g., Griffin, col. 5 ll. 23-26: test ordering service 220 [a service being software, so it necessarily stored in memory and executed by a processor] may be implemented using one or more computing devices [intelligence servers] and accessible over a network; in FIG. 6; col. 5 ll. 30-35: test ordering service 220 may use the functionality for test failure likelihood estimation 121 to generate ordered sequences of tests; col. 6 ll. 55-60: to score each of the tests 115-115C, the functionality for test failure likelihood estimation 121 may use a plurality of test scoring plugins 150; col. 7 ll. 65-67: the tests may be sorted by their score for likelihood of failure to generate the ordered sequence).
As to claim 3, Griffin/Goswami/Farrier/Zhou renders obvious the method of claim 2 (see rejection of claim 1 above), Griffin further discloses:
wherein the score generator predicts the likelihood of failure based on historical data (e.g., Griffin, col. 6 ll. 55-60: to score each of the tests 115-115C, the functionality for test failure likelihood estimation 121 may use a plurality of test scoring plugins 150; col. 7 ll. 1-5: failure rate plugin 152 scores a test according to its failure rate, e.g., the failures in its portion of the test execution history; col. 7 ll. 65-67: the tests may be sorted by their score for likelihood of failure to generate the ordered sequence; col. 2 ll. 22-25: when the testing process is initiated, tests with an estimated likelihood of failure may be performed earlier [the score for likelihood of failure is a prediction because the tests have not been executed yet]).
As to claim 4, Griffin/Goswami/Farrier/Zhou renders obvious the method of claim 2 (see rejection of claim 1 above), Griffin further discloses:
wherein the score generator predicts the likelihood of failure based on source code change data (e.g., Griffin, col. 7 ll. 17-22: test scoring plugins 150 may include a source code modification plugin 154 that scores a test according to the age of any modification to its source code).
As to claim 7, Griffin/Goswami/Farrier/Zhou renders obvious the method of claim 1 (see rejection of claim 1 above), Griffin further discloses
the duration of execution of the subset of tests in the test list is shorter than the duration of execution of the plurality of tests (e.g., Griffin, col. 3 ll. 11-23: the set of tests may be selected from a larger set of potential tests 115 [executing a subset of tests is necessarily of longer duration than the full set]; col. 2 ll. 23-30: when the testing process is initiated using the ordered sequence, tests with a higher estimated likelihood of failure may be performed earlier. When one or more failures are encountered during the testing process the testing may be stopped [so execution of the ordered tests stops earlier, i.e., test execution is of shorted duration]).
Griffin/Goswami/Farrier/Zhou does not explicitly disclose wherein a new call graph is automatically generated after test completion.
However, in an analogous art, Zhou discloses:
wherein a new call graph is automatically generated after test completion (e.g., Zhou, col. 5 ll. 5-27: as the test cases are exercised, the graph of FIG. 1 can be continually updated [updating an existing graph being generating a new one]. For example, test case 104a may be executed, and each function, method, module or other element of the software that it covers “(e.g., source files 102b, 102c)” may be tagged. After test case 104a finishes executing, edges between test case 104a and the tested source files can be created. Further, the node for test case 104a can be updated with information such as the identities of source files 102b, 102c, etc. The nodes for source files 102b, 102c are updated to identify test case 104a. Then the tags are cleared and the next case can be executed. The graph can this be generated and/or updated over time, as different test cases are executed).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the call graph and first software of Griffin/Goswami such that a new call graph is automatically generated after test completion, as taught by Zhou, as Zhou would provide the advantage of a means of continually updating the graph to reflect changes in the portions of software or test cases as the test cases are executed. (See Zhou, col. 5 ll. 4-7).
As to claim 8, it is a medium claim having limitations substantially the same as claim 1, which are obvious in view of the prior art for substantially the same reasons.
Note that while claim 8 only refers to generating a subset list of tests based on the initiate call graph having relationships between the plurality of test and “portions of the first software” instead of “the change to the code of the first software” in claim 1, the same teachings of the prior art teach the features of claim 8 and it would have been obvious to modify the teachings of Griffin with these features for the same reasons.
Further limitations, disclosed by Griffin, include:
a non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for automatically testing software code, (e.g., Griffith, Fig. 6 and associated text) the method comprising: (see rejection of claim 1 above).
As to claim 9, it is a medium claim whose limitations are substantially the same as those of claim 2. Accordingly, it is rejected for substantially the same reasons.
As to claim 10, it is a medium claim whose limitations are substantially the same as those of claim 3. Accordingly, it is rejected for substantially the same reasons.
As to claim 11, it is a medium claim whose limitations are substantially the same as those of claim 4 Accordingly, it is rejected for substantially the same reasons.
As to claim 14, it is a medium claim whose limitations are substantially the same as those of claim 7. Accordingly, it is rejected for substantially the same reasons.
As to claim 15, it is a system claim having limitations substantially the same as claim 1, which are obvious in view of the prior art for substantially the same reasons.
Note that while claim 15 only refers to generating a subset list of tests based on the initiate call graph having relationships between the plurality of test and “portions of the first software” instead of “the change to the code of the first software” in claim 1, the same teachings of the prior art teach the features of claim 15 and it would have been obvious to modify the teachings of Griffin with these features for the same reasons.
Further limitations, disclosed by Griffin, include:
a system for automatically testing software code, comprising:
a testing server including a memory and a processor; (e.g., Griffin, Figs. 2, 6 and associated texts, col. 2 ll. 29-51: the test execution system 100 may comprise one or more computing devices [testing servers], any of which may be implemented by the example computing device 3000 in FIG. 6; col. 9 ll. 1-2: computing device 3000 may by a uniprocessor system including one processor 3010; col. 9 ll. 55-60: system memory 3020 may store program instructions as described above for implementing embodiments)
a remote server including a second memory and a second processor (e.g., Griffin, Figs. 2, 6 and associated texts col. 5 ll. 22-28: the test ordering service may be implemented by one or more computing devices [remote servers], any of which may be implemented by the example computing device 3000 in FIG. 6 [so a second memory and processor]. The ordering service 220 may be external to the test execution system, e.g., implemented using different computing devices and accessible to the test execution system 100 over a network) and
one or more modules stored in the memory and executed by the processor (e.g., Griffin, Fig. 6 and associated text) to (see rejection of claim 1 above).
As to claim 16, it is a system claim whose limitations are substantially the same as those of claim 2. Accordingly, it is rejected for substantially the same reasons.
As to claim 17, it is a system claim whose limitations are substantially the same as those of claim 3. Accordingly, it is rejected for substantially the same reasons.
As to claim 18, it is a system claim whose limitations are substantially the same as those of claim 4 Accordingly, it is rejected for substantially the same reasons.
Claims 5, 6, 12, 13, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Griffin (US 9,514,034) in view of Goswami (US 2022/0188215) in view of Farrier (US 2021/0263728) in view of Zhou (US 7,480,900) in further view of Jagannath (US 2020/0250078).
As to claim 5, Griffin/Goswami/Farrier/Zhou discloses the method of claim 4 (see rejection of claim 1 above), but does not explicitly disclose wherein the source code change data includes a source code change time, files that were added or modified to the source code, level of change made to the software, and who made the change to the source code.
However, in an analogous art, Jagannath discloses:
wherein the source code change data includes a source code change time, files that were added or modified to the source code, level of change made to the software, and who made the change to the source code (e.g., Jagannath, par. [0012]: changes to component such as source code; par. [0019]: a feature of a component has been altered; par. [0027]: the attributes of the altered feature 203, such as an author of the altered feature [who made the change to the source code], a nature of the altered feature “(such as a minor bug fix, newly added software feature, major revision, etc.)” [minor bug fix or major revision being a level of change, a newly added feature being a file that was added to the source code], a date/or time of the creation of the altered feature [source code change time]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the source code change data of Griffin/Goswami/Farrier/Zhou to include a source code change time, files that were added or modified to the source code, level of change made to the software, and who made the change to the source code, as taught by Jagannath, as Jagannath would provide the advantage of a means of correlating that data with testing failures. (See Jagannath, pars. [0033-0034]).
As to claim 6, Griffin/Goswami/Farrier/Zhou discloses the method of claim 1 (see rejection of claim 1 above), but does not explicitly disclose wherein ordering each test includes: training a score generator using training data associated with the source code being tested; and applying test data to the trained score generator to generate the likelihood of failure for each test in the test list.
However, in analogous art, Jagannath discloses
wherein ordering each test (see below) includes:
training a score generator using training data associated with the source code being tested; (e.g., Jagannath, par. [0031]: tests having higher probabilities of failure will be placed earlier in the dynamic test order than tests having lower probabilities of failure; par. [0012]: the probabilities [scores] may be generated based on machine-learned models that may be trained using results of prior testing outcomes) and
applying test data to the trained score generator to generate the likelihood of failure for each test in the test list (e.g., Jagannath, par. [0027]: apparatus 100 may use the attributes [test data] as input parameters to a machine-learned model of test procedures to generate, for each test procedure, a probability that the altered feature will fail; par. [0048]: test outcomes [also test data] may be used to re-train the machine learned model).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the ordering of each test of Griffin/Goswami/Farrier/Zhou to include training a score generator using training data associated with the source code being tested and applying test data to the trained score generator to generate the likelihood of failure for each test in the test list as taught by Jagannath, as Jagannath would provide the advantages of a means of ordering the tests via machine learning and a means to continuously re-train the machine learning process. (See Jagannath, pars. [0012], [0048])
As to claim 12, it is a medium claim whose limitations are substantially the same as those of claim 5. Accordingly, it is rejected for substantially the same reasons.
As to claim 13, it is a medium claim whose limitations are substantially the same as those of claim 6. Accordingly, it is rejected for substantially the same reasons.
As to claim 19, it is a system claim whose limitations are substantially the same as those of claim 5. Accordingly, it is rejected for substantially the same reasons.
As to claim 20, it is a system claim whose limitations are substantially the same as those of claim 6. Accordingly, it is rejected for substantially the same reasons.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TODD AGUILERA whose telephone number is (571)270-5186. The examiner can normally be reached M-F 11AM - 7:30PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hyung S Sough can be reached at (571)272-6799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TODD AGUILERA/Primary Examiner, Art Unit 2192
Prosecution Insights