DETAILED ACTION
Claims 1-20 are pending.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Notes
Examiner has cited particular columns and line numbers, paragraph numbers, or figures in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant, in preparing the responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 7, 9-12, 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Jain et al. (US-PAT-NO: 10,949,172 B1) hereinafter Jain, in further view of Wiemann (US-PAT-NO: 6,298,317 B1).
As per claim 1, Jain teaches a computer-implemented method, comprising: receiving, by a computer system, a request to integrate a source code change into a codebase associated with an application (see Jain [column 5, lines 33-37], “In operation, a system retrieves a changelist associated with a plurality of software projects generated using an analyzer engine associated with the system. See operation 102,” see Jain [column 2, lines 38-40] and “For example, the changelist may include a listing of parts of the module that are changed by respective teams T1, T2 . . . Tn. For instance, the changelist CL1 may correspond to team T1, CL2 corresponds to team T2, and so on,” where the teams request integrations of changes being made to the projects being worked on by multiple teams); determining a line of source code associated with a requested source code change (see Jain [column 2, lines 49-50], “The system identifies one or more changed lines with respect to changed code utilizing the changelist. See operation 104”); determining an application test suite associated with the application (see Jain [column 3, lines 42-44], “In computer software, code coverage refers to a measure used to describe the degree to which the source code of a program is tested by a particular test suite”); determining a test subset of the application test suite, based on the source code attribute of the line of source code (see Jain [column 2, lines 58-61], “In one embodiment, the system may perform a lexical analysis of code portions that are changed to identify names of test cases that were added specifically as part of a particular project”).
Jain does not teach determining, by the computer system, a source code attribute of the line of source code; mutating, by the computer system, the source code change into a mutated source code change; building, by the computer system, a mutated application by compiling one or more software classes including the mutated source code change; and executing, by the computer system, the first set of tests on the mutated application. However, Wiemann teaches determining, by the computer system, a source code attribute of the line of source code (see Wiemann [column 8, lines 55-56], “Given a single input vector where I(1)=1, I(2)=2, I(3)=3, I(4)=4, and I(5)=5, as illustrated at 157”); mutating, by the computer system, the source code change into a mutated source code change (see Wiemann [column 7, lines 27-30], “After the mutation type has been selected from a defined group of mutation types, as indicated at block 104, a mutation of the selected mutation type is inserted into the selected source code module, as indicated at block 106”); building, by the computer system, a mutated application by compiling one or more software classes including the mutated source code change (see Wiemann [column 7, lines 33-35], “The computer program containing the mutated source code module is then compiled, as indicated at block 110, creating a mutated computer program”); and executing, by the computer system, the first set of tests on the mutated application (see Wiemann [column 7, lines 36-43], “The mutated computer program is next simulated and verified with first functional simulator 26, as indicated at block 112. As described earlier, first functional simulator 26 performs a subtlety check on the mutated computer program. In one embodiment, subtlety check is implemented as a minimal suite of tests that executes basic operation of the mutated computer program”).
Jain and Wiemann are analogous are because they are in the same field of endeavor of software development. Therefore it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Jain’s teaching of determining incremental code coverage and/or incremental mutation coverage for individual software projects or teams within a common environment with Wiemann’s teaching of functionally testing a computer program to determine if a test suite properly tests the computer program to incorporate mutated code into testing for better more reliable testing results and determining the likelihood that undetected bugs still exist within a computer program.
As per claim 2, Jain modified with Wiemann teaches further comprising: determining that the source code change modifies a second line of source code within the codebase (see Jain [column 2, lines 49-50], “The system identifies one or more changed lines with respect to changed code utilizing the changelist. See operation 104”); determining a second test subset of the application test suite associated with the second line of source code, wherein the second test subset is different from the test subset (see Jain [column 2, lines 58-61], “In one embodiment, the system may perform a lexical analysis of code portions that are changed to identify names of test cases that were added specifically as part of a particular project”); and executing the second test subset on the mutated application (see Wiemann [column 7, lines 65-67], “Second functional simulator 28 performs a full regression test to detect the subtle mutation within the mutated computer program”).
As per claim 3, Jain modified with Wiemann teaches further comprising: initiating an operation to integrate the source code change into the codebase (see Jain [column 6, lines 57-59], “The build automation tools, such as Apache Maven, are used to store the reports as an artifact item in the build repository for later retrieval”), based at least in part on test results associated with executing the test subset (see Jain [column 6, lines 52-57], “The libraries provide results in a very large xml file. The xml file contains the instruction level instrumentation results for each line of a source file of a particular project and lists out exactly how many numbers of instructions at those lines were covered by unit tests for that particular project”).
As per claim 4, Jain modified with Wiemann teaches inputting the source code attribute of the line of source code into a trained machine learning model configured to output data identifying the test subset of the application test suite (see Wiemann [column 9-10, lines 66-5], “A first test vector 202 of I(1)=1, I(2)=2, I(3)=3, I(4)=4, and I(5)=5 applied to the computer program produces an output of O(1)=1, O(2)=2, O(3)=3, O(4)=4, and O(5) =5 from the mutated computer program, as illustrated at 208. The output when first test vector 202 is applied to the correct (non-mutated) computer program is O(1)=1, O(2)=2, O(3)=3, O(4)=4, and O(5)=5, as illustrated at 209”).
As per claim 7, Jain modified with Wiemann teaches wherein: mutating the source code change is based on a set of code mutation rules (see Wiemann [column 5, lines 16-19], “Syntactic mutation selector 58 selects one type of syntactic mutation from predefined set of syntactic mutation types 40 and passes the selected syntactic mutation type onto mutation generator 24”); and determining the test subset is based at least in part on the set of code mutation rules (see Wiemann [column 7, lines 46-52], “At this point, no further functional simulation is performed, and process control is passed back to block 102, where a new source code module and mutation type are selected. Thus, the first functional simulator performs the subtlety check to quickly filter out the non-subtle mutations from further testing”).
As per claims 9-12 and 15, these are the computer system, comprising: one or more processor; and one or more non-transitory computer-readable media storing computer-executable instruction, when executed by the one or more processors, cause the one or more processors to perform operations (see Jain [column 8, lines 64-67 and column 9, lines 1-11], “As used here, a “computer-readable medium” includes one or more of any suitable media for storing the executable instructions of a computer program such that the instruction execution machine, system, apparatus, or device may read (or fetch) the instructions from the computer readable medium and execute the instructions for carrying out the described methods. Suitable storage formats include one or more of an electronic, magnetic, optical, and electromagnetic format. A non-exhaustive list of conventional exemplary computer readable medium includes: a portable computer diskette; a RAM; a ROM; an erasable programmable read only memory (EPROM or flash memory); optical storage devices, including a portable compact disc (CD), a portable digital video disc (DVD), a high definition DVD (HD-DVD™), a BLU-RAY disc; and the like”) claims to method claims 1-4 and 7, respectively. Therefore, they are rejected for the same reasons as above.
As per claims 17 and 18, these are the non-transitory computer-readable media (see Jain [column 8, lines 64-67 and column 9, lines 1-11], “As used here, a “computer-readable medium” includes one or more of any suitable media for storing the executable instructions of a computer program such that the instruction execution machine, system, apparatus, or device may read (or fetch) the instructions from the computer readable medium and execute the instructions for carrying out the described methods. Suitable storage formats include one or more of an electronic, magnetic, optical, and electromagnetic format. A non-exhaustive list of conventional exemplary computer readable medium includes: a portable computer diskette; a RAM; a ROM; an erasable programmable read only memory (EPROM or flash memory); optical storage devices, including a portable compact disc (CD), a portable digital video disc (DVD), a high definition DVD (HD-DVD™), a BLU-RAY disc; and the like”) claims to method claims 1 and 4, respectively. Therefore, they are rejected for the same reasons as above.
Claims 5, 6, 8, 13, 14, 16, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Jain (US-PAT-NO: 10,949,172 B1) and Wiemann (US-PAT-NO: 6,298,317 B1), in further view of Lees et al. (US-PGPUB-NO: 2021/0141714 A1) hereinafter Lees.
As per claim 5, Jain modified with Wiemann does not teach wherein determining the test subset comprises: retrieving test results associated with a previous execution of the application test suite, on a previous mutated application associated with the line of source code; and determining, within the test results associated with a previous execution, one or more passing tests; and excluding the one or more passing tests from the test subset. However, Lees teaches wherein determining the test subset comprises: retrieving test results associated with a previous execution of the application test suite, on a previous mutated application associated with the line of source code (see Lees paragraph [0062], “A software test is run on each code path CP1 and CP2 corresponding to the higher order mutant HOM. Put another way, a higher order mutant HOM is created which contains both FOM 1 and FOM2, and then all of the software tests are run for the higher order mutant HOM”); and determining, within the test results associated with a previous execution, one or more passing tests (see Lees paragraph [0063], “The first order mutant FOM1 is deemed to have survived if all of the software tests are passed”); and excluding the one or more passing tests from the test subset (see Lee paragraph [0063] and FIG. 6, “Conversely, if the first order mutant FOM 1 code is killed, nothing has been learned for the first order mutant FOM 2, and so the tests are run for the first order mutant FOM 2,” where if the FOM survives (passes a test) the FOM isn’t retested (excluded) and if the FOM is killed (failed a test) the FOM is retested (included)).
Jain, Wiemann and Lees are analogous are because they are in the same field of endeavor of software development. Therefore it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Jain’s teaching of determining incremental code coverage and/or incremental mutation coverage for individual software projects or teams within a common environment and Wiemann’s teaching of functionally testing a computer program to determine if a test suite properly tests the computer program with Lees teaching of creating higher order mutant for mutation testing software to incorporate software test identifying respective code paths and associating mutated code with the identified code paths to better test code using mutation and testing in parallel.
As per claim 6, Jain modified with Wiemann and Lee teaches wherein determining the test subset comprises: accessing a mapping storing associations between a plurality of subsets of the application test suite, and associated source code attributes (see Lees paragraph [0027] and FIG. 4, “Such an associating unit may be configured to, for each first order mutant of a plurality of first order mutants, associate the first order mutant with the one or more identified code paths it interacts with,”).
As per claim 8, Jain modified with Wiemann and Lees teaches further comprising: receiving a mutation test confidence level associated with the request to integrate the source code change (“In at least one embodiment of the present disclosure, there is provided a computer-implemented method for classifying a measure of performance for mutation testing software (i.e., a classifying method). The classifying method may comprise creating a higher order mutant for mutation testing software according to the method discussed above (i.e., a creating method),” see Lees paragraph [0034], where the classification is interpreted as the confidence level); and determining the test subset based at least in part on the mutation test confidence level (“The method for classifying a measure of performance for mutation testing software may further comprise, for each of the set of software tests, executing the software test using the higher order mutant, then classifying a measure of performance of the software test based on the executed software test,” see Lees paragraph [0034], where the classification is interpreted as the confidence level).
As per claims 13, 14 and 16, these are the computer system, comprising: one or more processor; and one or more non-transitory computer-readable media storing computer-executable instruction, when executed by the one or more processors, cause the one or more processors to perform operations (see Jain [column 8, lines 64-67 and column 9, lines 1-11], “As used here, a “computer-readable medium” includes one or more of any suitable media for storing the executable instructions of a computer program such that the instruction execution machine, system, apparatus, or device may read (or fetch) the instructions from the computer readable medium and execute the instructions for carrying out the described methods. Suitable storage formats include one or more of an electronic, magnetic, optical, and electromagnetic format. A non-exhaustive list of conventional exemplary computer readable medium includes: a portable computer diskette; a RAM; a ROM; an erasable programmable read only memory (EPROM or flash memory); optical storage devices, including a portable compact disc (CD), a portable digital video disc (DVD), a high definition DVD (HD-DVD™), a BLU-RAY disc; and the like”) claims to method claims 5, 6 and 8, respectively. Therefore, they are rejected for the same reasons as above.
As per claims 19 and 20, these are the non-transitory computer-readable media (see Jain [column 8, lines 64-67 and column 9, lines 1-11], “As used here, a “computer-readable medium” includes one or more of any suitable media for storing the executable instructions of a computer program such that the instruction execution machine, system, apparatus, or device may read (or fetch) the instructions from the computer readable medium and execute the instructions for carrying out the described methods. Suitable storage formats include one or more of an electronic, magnetic, optical, and electromagnetic format. A non-exhaustive list of conventional exemplary computer readable medium includes: a portable computer diskette; a RAM; a ROM; an erasable programmable read only memory (EPROM or flash memory); optical storage devices, including a portable compact disc (CD), a portable digital video disc (DVD), a high definition DVD (HD-DVD™), a BLU-RAY disc; and the like”) claims to method claims 6 and 8, respectively. Therefore, they are rejected for the same reasons as above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
McClory et al. (US-PGPUB-NO: 2018/0321918 A1) teaches creating and managing a software application by identifying an application template and determining an application creation configuration based on the application template.
Broadbent et al. (US-PGPUB-NO: 2018/0074936 A1) teaches accessing a quality of plurality of changes to a plurality of source code and identifying one property associated with each of the plurality of changes within the received plurality of source code.
Crova et al. (US-PGPUB-NO: 2016/0048444 A1) teaches selecting one or more tests for a compiled software module and detecting one or more changes being made to the source code.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LENIN PAULINO whose telephone number is (571)270-1734. The examiner can normally be reached on Week 1: Mon-Thu 7:30am - 5:00pm Week 2: Mon-Thu 7:30am - 5:00pm and Fri 7:30am - 4:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bradley Teets can be reached on (571) 272-3338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LENIN PAULINO/Examiner, Art Unit 2197
Prosecution Insights