DETAILED ACTION
Claims 1-13 and 16-22 are presented for examination.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings received on 10 April 2023 are accepted.
Specification
Applicant is reminded of the proper language and format for an abstract of the disclosure.
The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words in length. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
The abstract of the disclosure is objected to because:
The abstract includes phrases which can be implied. Examiner suggests amending the abstract to read:
Automatic test parameter tuning in constrained random verification. In aspects, a method receives a first set of parameters for testing a design under test, performs a first regression (e.g., an overnight regression test) on a design under test using the first set of parameters, and analyzes the results of the first regression including determining a coverage percentage. The method then generates an optimized set of parameters based on the analysis of the results of the first regression and performs an additional regression on the design under test using the optimized set of parameters. In aspects, the method is repeated using the optimized set of parameters until a coverage percentage is reached, or in some implementations, full coverage may be reached. Some implementations of the method utilize black-box optimization through use of a Bayesian optimization algorithm.
A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Claim Rejections - 35 USC § 101 – Abstract Idea
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6, 12, 13, and 16-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
To determine if a claim is directed to patent ineligible subject matter, the Court has guided the Office to apply the Alice/Mayo test, which requires:
1. Determining if the claim falls within a statutory category;
2A. Determining if the claim is directed to a patent ineligible judicial exception consisting of a law of nature, a natural phenomenon, or abstract idea; and
2B. If the claim is directed to a judicial exception, determining if the claim recites limitations or elements that amount to significantly more than the judicial exception.
See MPEP §2106.
Step 2A is a two prong inquiry. MPEP §2106.04(II)(A). Under 2A(i), the first prong, examiners evaluate whether a law of nature, natural phenomenon, or abstract idea is set forth or described in the claim. Abstract ideas include mathematical concepts, certain methods of organizing human activity, and mental processes. MPEP §2106.04(a)(2). Under 2A(ii), the second prong, examiners determine whether any additional limitations integrates the judicial exception into a practical application. MPEP §2106.04(d).
Claim 1 step 2A(i):
The claim(s) recite:
1. A method comprising:
performing a first regression on a design under test using a first set of parameters, the design under test comprising a logical representation of a hardware system that includes a plurality of logical components for fabrication as an integrated circuit;
analyzing results of the first regression including determining a coverage percentage of the first regression;
generating, based on the analysis of the results of the first regression, an optimized set of parameters for a subsequent regression; and
performing the subsequent regression on the design under test using the optimized set of parameters.
Performing a first regression on a design under test of a logical representation encompasses evaluation and/or judgement which can be performed mentally and/or with the aid of pen and paper. Notably, Specification page 15 paragraph 30 states “the baseline, which is the human-generated tests with fixed parameters” which admits the tests can be human generated. The logical representation corresponds with an abstract representation of the hardware system and not an actual physical system.
Analyzing the results of the regression for a coverage percentage corresponds with additional recitation of evaluation and/or judgment which can be performed mentally.
Generating an optimized set of parameters based on the results is recitation of evaluation, judgment, or opinion which can be performed mentally and/or with the aid of pen and paper.
Performing the subsequent regression using the optimized set of parameters can be performed mentally and/or with the aid of pen and paper.
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 1 step 2A(ii):
This judicial exception is not integrated into a practical application because:
Claim(s) do not recite any “additional” limitations.
Claim 1 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
Claim(s) do not recite any “additional” limitations.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Claim 2 step 2A(i):
Dependent claims recite at least the identified judicially excepted subject matter of their parent claim(s).
The claim(s) recite:
2. The method as recited in claim 1, wherein analyzing the results of the first regression includes determining a point-in-time coverage percentage for the first regression.
Determining a coverage percentage at a particular point in time is a simple calculation capable of being performed mentally and/or with the aid of pen and paper.
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 2 step 2A(ii):
This judicial exception is not integrated into a practical application because:
Claim(s) do not recite any “additional” limitations.
Claim 2 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
Claim(s) do not recite any “additional” limitations.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Claim 3 step 2A(i):
Dependent claims recite at least the identified judicially excepted subject matter of their parent claim(s).
The claim(s) recite:
3. The method as recited in claim 1 further comprising:
repeating the analyzing, generating, and performing steps with the optimized set of parameters for subsequent regressions until total coverage is achieved; and
Repeating the above discussed mental process steps of analyzing, generating, and performing is recitation to repeat respective mental process steps. Determining that a total coverage has been achieved is a criteria capable of being determined mentally and/or with the aid of pen and paper.
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 3 step 2A(ii):
This judicial exception is not integrated into a practical application because:
The claim(s) recite:
storing, for each regression of the first regression and the subsequent regressions, an associated set of parameters and results of each regression to be accessed by future regressions.
Storing the results of the respective evaluations and/or calculations is extra solution activity in the form of a generic outputting of the result of the abstract idea. See MPEP §2106.05(g).
Claim 3 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
The claim(s) recite:
storing, for each regression of the first regression and the subsequent regressions, an associated set of parameters and results of each regression to be accessed by future regressions.
MPEP §2106.05(d) provides examples:
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information);
iv. Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015)
These data transmitting and storing examples are encompassed by the generic recitation of storing data recited by the claim. Accordingly, the claim recitation here is at least as abstract as the examples given in the MPEP.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Claim 4 step 2A(i):
Dependent claims recite at least the identified judicially excepted subject matter of their parent claim(s).
The claim(s) recite:
4. The method as recited in claim 1, wherein:
the optimized set of parameters comprises at least one hardware condition related to the logical representation of the hardware system of the design under test; or
the optimized set of parameters comprises at least one of a bus width, a data width, a register depth, a memory depth, a voltage level, a clock frequency, a timing variable, or a delay useful to optimize the design under test or one of the plurality of logical components of the design under test.
From this list of alternatives, at least a parameter of a hardware condition related to the logical representation is capable of being considered mentally and/or with the aid of pen and paper.
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 4 step 2A(ii):
This judicial exception is not integrated into a practical application because:
Claim(s) do not recite any “additional” limitations.
Claim 4 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
Claim(s) do not recite any “additional” limitations.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Claim 5 step 2A(i):
Dependent claims recite at least the identified judicially excepted subject matter of their parent claim(s).
The claim(s) recite:
5. The method as recited in claim 1, wherein generating the optimized parameter set comprises using a black-box optimizer.
Examiner interprets a “black-box optimizer” as an optimizer intended to perform optimization in conjunction with a black box, and not itself a black box. Accordingly, the black-box optimizer encompasses many different heuristics and mathematical algorithms capable of being used to perform optimization without explicit internal information on what they are optimizing. Because using a black box optimizer does not require the use of a black box itself, these optimizer heuristics and algorithms encompass several simplified optimizers capable of being considered mentally and/or performed with the aid of pen and paper. Accordingly, generating the optimized parameter set remains a step capable of being performed mentally and/or with the aid of pen and paper.
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 5 step 2A(ii):
This judicial exception is not integrated into a practical application because:
Claim(s) do not recite any “additional” limitations.
Claim 5 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
Claim(s) do not recite any “additional” limitations.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Claim 6 step 2A(i):
Dependent claims recite at least the identified judicially excepted subject matter of their parent claim(s).
The claim(s) recite:
6. The method as recited in claim 5, wherein generating the optimized parameter set further comprises: selecting, for one or more parameters of the optimized parameter set, a value from a uniformly distributed set of values using a random seed.
Selecting a random value from a uniform distribution using a random seed is an evaluation capable of being performed mentally and/or with the aid of pen, paper, and dice or other suitable randomizer.
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 6 step 2A(ii):
This judicial exception is not integrated into a practical application because:
Claim(s) do not recite any “additional” limitations.
Claim 6 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
Claim(s) do not recite any “additional” limitations.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Claim 12 step 2A(i):
Dependent claims recite at least the identified judicially excepted subject matter of their parent claim(s).
The claim(s) recite:
12. The method as recited in claim 1, further comprising:
performing, for each regression of the first regression and the subsequent regression, a parallel regression using a predetermined set of parameters.
Performing another regression on a design under test of a logical representation encompasses evaluation and/or judgement which can be performed mentally and/or with the aid of pen and paper.
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 12 step 2A(ii):
This judicial exception is not integrated into a practical application because:
Claim(s) do not recite any “additional” limitations.
Claim 12 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
Claim(s) do not recite any “additional” limitations.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Claim 13 step 2A(i):
Dependent claims recite at least the identified judicially excepted subject matter of their parent claim(s).
The claim(s) recite:
13. The method as recited in claim 1, further comprising:
prior to performing the first regression:
…
generating the first and any subsequent parameter sets based on at least the analyzed results of each previous regression using each of the one or more subsets of parameters.
Generating a parameter set based on previous results corresponds with mental processes in the form of evaluation, judgement, and/or opinion.
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 13 step 2A(ii):
This judicial exception is not integrated into a practical application because:
The claim(s) recite:
receiving one or more subsets of parameters, each of the one or more subsets of parameters comprising a number of parameters less than or equal to a total number of parameters in the first set of parameters;
receiving analyzed results of at least one previous regression using each of the one or more subsets of parameters; and
Receiving data is a recitation of data gathering at a high level of generality. Data gathering here is extra solution activity. See MPEP §2106.05(g).
Claim 13 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
The claim(s) recite:
receiving one or more subsets of parameters, each of the one or more subsets of parameters comprising a number of parameters less than or equal to a total number of parameters in the first set of parameters;
receiving analyzed results of at least one previous regression using each of the one or more subsets of parameters; and
MPEP §2106.05(d) provides examples:
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information);
iv. Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015)
These data gathering examples are encompassed by the generic recitation of data gathering recited by the claim. Accordingly, the claim recitation here is at least as abstract as the examples given in the MPEP.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Claim 16 step 2A(i):
Dependent claims recite at least the identified judicially excepted subject matter of their parent claim(s).
The claim(s) recite:
16. The method as recited in claim 1, wherein analyzing the results of the first regression includes determining an accumulated coverage percentage.
Determining an accumulated coverage percentage is an evaluation capable of being performed mentally and/or with the aid of pen and paper.
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 16 step 2A(ii):
This judicial exception is not integrated into a practical application because:
Claim(s) do not recite any “additional” limitations.
Claim 16 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
Claim(s) do not recite any “additional” limitations.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Claim 17 step 2A(i):
Dependent claims recite at least the identified judicially excepted subject matter of their parent claim(s).
The claim(s) recite:
17. The method as recited in claim 1, wherein each parameter of the optimized set of parameters has a predetermined domain.
The respective domain for each parameter does not prevent consideration of these parameters within the human mind.
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 17 step 2A(ii):
This judicial exception is not integrated into a practical application because:
Claim(s) do not recite any “additional” limitations.
Claim 17 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
Claim(s) do not recite any “additional” limitations.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Claim 18 step 2A(i):
Dependent claims recite at least the identified judicially excepted subject matter of their parent claim(s).
The claim(s) recite:
18. The method as recited in claim 1, wherein the design under test is a hardware device, the hardware device selected from the group consisting of: a processing unit, the processing unit including a machine-learning processor; memory; or a cache controller.
These listed alternatives for the design under test are listed in the alternative. While it may be impractical to perform regression testing on an entire system include each of a processing unit, ML processor, memory, and a cache controller, it remains practical to consider such components individually. In particular, somewhat simple versions of these components are considered practical to consider mentally with the aid of pen and paper.
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 18 step 2A(ii):
This judicial exception is not integrated into a practical application because:
Claim(s) do not recite any “additional” limitations.
Claim 18 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
Claim(s) do not recite any “additional” limitations.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Claim 19 step 2A(i):
Dependent claims recite at least the identified judicially excepted subject matter of their parent claim(s).
The claim(s) recite:
19. The method as recited in claim 1, wherein the design under test is an algorithm or software application.
Algorithms and software applications are capable of being evaluated mentally and/or with the aid of pen and paper.
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 19 step 2A(ii):
This judicial exception is not integrated into a practical application because:
Claim(s) do not recite any “additional” limitations.
Claim 19 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
Claim(s) do not recite any “additional” limitations.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Claim 20 step 2A(i):
Dependent claims recite at least the identified judicially excepted subject matter of their parent claim(s).
The claim(s) recite:
20. The method as recited in claim 1, further comprising: modifying the design under test as a result of the first regression, the subsequent regression, or another regression.
Modifying a design under test as a result of the regression(s) corresponds with mental design processes including evaluation, judgement, opinion, and/or observation.
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 20 step 2A(ii):
This judicial exception is not integrated into a practical application because:
Claim(s) do not recite any “additional” limitations.
Claim 20 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
Claim(s) do not recite any “additional” limitations.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Claim 21 step 2A(i):
Dependent claims recite at least the identified judicially excepted subject matter of their parent claim(s).
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 21 step 2A(ii):
This judicial exception is not integrated into a practical application because:
The claim(s) recite:
21. The method as recited in claim 1, further comprising: manufacturing an article according to the design under test.
Instructions recited at a high level of generality to “manufacture” a designed article are mere instructions to “apply” the result of the abstract idea. See MPEP §2106.05(h).
Claim 21 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
Limitations analyzed under MPEP §2106.05(h) in step 2A(ii) above are analyzed the same here under step 2B.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Claim 22 step 2A(i):
Dependent claims recite at least the identified judicially excepted subject matter of their parent claim(s).
The claim(s) recite:
22. The method as recited in claim 1, wherein the optimized set of parameters comprises at least one hardware condition related to the logical representation of the hardware system of the design under test.
Hardware conditions related to logical representations of the system are capable of being considered mentally and/or with the aid of pen and paper.
This falls within the mental process grouping of abstract ideas. See MPEP §2106.04(a)(2).
Claim 22 step 2A(ii):
This judicial exception is not integrated into a practical application because:
Claim(s) do not recite any “additional” limitations.
Claim 22 step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered individually and in combination, because:
Claim(s) do not recite any “additional” limitations.
When further considering the claims as a whole and as an ordered combination the claims fail to amount to significantly more than the judicially excepted abstract idea.
Subject Matter Eligibility Claims 7-11
Claim 7 recites “using a Bayesian optimization algorithm.” Examiner recognizes there are limits to what is practical to perform mentally in the human mind, even with the aid of pen and paper. See MPEP §2106.04(a)(2)(III)(A) (“A Claim With Limitation(s) That Cannot Practically be Performed in the Human Mind Does Not Recite a Mental Process.”). Here, Examiner finds using a “Bayesian optimization algorithm” goes beyond the limits of what can practically be performed within the human mind and therefore cannot reasonably be considered mental processes.
Furthermore, while the “Bayesian optimization algorithm” limitation does refer to using a category of mathematical algorithm, it does not itself recite a specific mathematical algorithm or specific calculation. The limitation uses mathematical concepts, but is not itself mathematics per se. Accordingly, claim 7 is not directed towards the judicial exception of mathematical concepts.
Therefore, Examiner finds claims 7-11 are subject matter eligible under §101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-5, 7, 8, 13, 16-18, 20, and 22
Claims 1-5, 7, 8, 13, 16-18, 20, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over US patent 7,181,376 B2 Fine, et al. (cited in IDS dated 10 April 2023) [herein “Fine”] in view of US patent 10,607,039 B1 Kinderman, et al. [herein “Kinderman”].
Claim 1 recites “1. A method comprising: performing a first regression on a design under test using a first set of parameters.” Fine column 3 lines 36-40 disclose:
Test generator 10 may produce new tests 18' which will be used by simulator 20 to stimulate and/or operate design under test 22. The simulation output of simulator 20 may be analyzed by coverage analysis tool 24 and the output may be provided to CDG engine 30.
The simulator stimulating or operating the DUT with a new test(s) corresponds to performing a regression on DUT using a set of parameters. Without loss of generality, the new tests are a first set of parameters.
Claim 1 further recites “the design under test comprising a logical representation of a hardware system that includes a plurality of logical components for fabrication as an integrated circuit.” Fine column 1 lines 33-38 discloses:
A simulator 20 then simulates a design under test (DUT) 22 using generated test-cases 18 and the behavior of DUT 22 is monitored using checking (e.g. "assertion") tools and other checking methods, such as final results comparisons, to make sure that it meets its specification.
Fine column 8 lines 21-23 disclose “The design under test 22 for the second experiment is the Storage Control Element (SCE) 80 of an IBM z-series system, shown in FIG. 7.” A storage control element (SCE) of an IBM z-series system is a hardware system of logical components for fabrication as an integrated circuit. The simulation simulating the design under test (DUT) shows the DUT is a logical representation of the hardware system such that it can be simulated.
Claim 1 further recites “analyzing results of the first regression including determining a coverage percentage of the first regression.” Fine column 3 lines 36-40 disclose:
Test generator 10 may produce new tests 18' which will be used by simulator 20 to stimulate and/or operate design under test 22. The simulation output of simulator 20 may be analyzed by coverage analysis tool 24 and the output may be provided to CDG engine 30.
The output of simulator being analyzed by coverage analysis tool corresponds with analyzing results of the first regression to determine a coverage.
Fine does not explicitly disclose using a percentage; however, in analogous art of simulation analysis of SoC, Kinderman column 12 lines 5-8 teaches “The abscissae in distribution progression 800 (X-axis) indicate an iteration step number 801, and the ordinates (Y-axis) indicate a target coverage 802 (e.g., percentage, %, of desired coverage).” A target coverage of a percentage is a coverage percentage.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Fine and Kinderman. One having ordinary skill in the art would have found motivation to use a percentage for a desired coverage into the system of coverage directed test generation for the advantageous purpose of quantifying progression over multiple iterations. See Kinderman column 12 lines 1-20.
Claim 1 further recites “generating, based on the analysis of the results of the first regression, an optimized set of parameters for a subsequent regression.” Fine column 3 lines 46-50 disclose:
may receive coverage information 25 and coverage reports 26 as input and may query Bayesian network 38 to determine possible directives 16' to send to test generator 10 to create a new set of tests 18' which may, in the next simulation
Using the coverage reports as input to determine new sets of tests for a next simulation corresponds with generating an optimized set of parameters for a subsequent regression. See further Fine column 6 lines 26-43.
Fine column 7 lines 19-21 disclose “to reach the desired coverage cases with many relatively short instruction sequences.” Reaching a desired coverage with relatively short instruction sequences correspond with the set of parameters for subsequent regressions being optimized.
Claim 1 further recites “and performing the subsequent regression on the design under test using the optimized set of parameters.” Fine column 7 lines 52-57 disclose:
Each sequence was generated by DBN 70 by solving the Most Probable Explanation (MPE) problem for the coverage task requested by CDG task manager 32. All 49 coverage cases of the training set plus three additional uncovered cases were reached using instruction sequences designed by DBN 70.
Generating the sequences and reaching the cases using the instruction sequences corresponds with performing respective subsequent regressions on the DUT using the optimized sets of parameters.
Claim 2 further recites “2. The method as recited in claim 1, wherein analyzing the results of the first regression includes determining a point-in-time coverage percentage for the first regression.” Fine does not explicitly disclose using a percentage; however, in analogous art of simulation analysis of SoC, Kinderman column 12 lines 5-8 teaches “The abscissae in distribution progression 800 (X-axis) indicate an iteration step number 801, and the ordinates (Y-axis) indicate a target coverage 802 (e.g., percentage, %, of desired coverage).” A target coverage of a percentage is a coverage percentage. The iteration step indicates a respective point in time.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Fine and Kinderman. One having ordinary skill in the art would have found motivation to use a percentage for a desired coverage into the system of coverage directed test generation for the advantageous purpose of quantifying progression over multiple iterations. See Kinderman column 12 lines 1-20.
Claim 3 further recites “3. The method as recited in claim 1 further comprising: repeating the analyzing, generating, and performing steps with the optimized set of parameters for subsequent regressions until total coverage is achieved.” Fine column 1 lines 41-45 disclose “Analysis of reports 26 allows verification team 14 to modify directives 16 to test generator 10 to overcome weaknesses in the implementation of coverage model 12, This process is repeated until the exit criteria in coverage model 12 are met.”
Fine column 7 lines 19-21 disclose “to reach the desired coverage cases with many relatively short instruction sequences.” Reaching a desired coverage correspond with reaching a total coverage.
Claim 3 further recites “and storing, for each regression of the first regression and the subsequent regressions, an associated set of parameters and results of each regression to be accessed by future regressions.” Fine column 3 lines 46-47 disclose “coverage information 25 and coverage reports 26.” Fine column 3 line 59 disclose “a history 42 of coverage data.”
But Fine does not explicitly disclose storing parameters and results; however, in analogous art of simulation analysis of SoC, Kinderman column 5 lines 56-64 teach:
a constrained optimization engine may use a regression coverage database storing data obtained through a campaign of SoC simulations. Each simulation reports measurements for at least one output parameter under a certain device configuration determined by selected input parameters. Based on data collected from the measurements and stored in the system performance database, the constrained optimization engine renders a statistical model of the system
Storing data from the simulations correspond to storing results of each regression. The input parameters correspond with the associated set of parameters.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Fine and Kinderman. One having ordinary skill in the art would have found motivation to use a coverage database with the system of coverage directed test generation for the advantageous purpose of keeping the history of coverage data in a manner capable of rending subsequent statistical modeling. See Kinderman column 5 lines 56 to column 6 line 7.
Claim 4 further recites “4. The method as recited in claim 1, wherein: the optimized set of parameters comprises at least one hardware condition related to the logical representation of the hardware system of the design under test; or the optimized set of parameters comprises at least one of a bus width, a data width, a register depth, a memory depth, a voltage level, a clock frequency, a timing variable, or a delay useful to optimize the design under test or one of the plurality of logical components of the design under test.” From the above list of alternatives Examiner is selecting “a bus width.”
Fine column 8 lines 21-23 disclose “The design under test 22 for the second experiment is the Storage Control Element (SCE) 80 of an IBM z-series system, shown in FIG. 7.”
But Fine does not explicitly disclose configuration variables of the DUT nor a bus width in particular; however, in analogous art of simulation analysis of SoC, Kinderman column 4 line 67 to column 5 line 7 teaches:
For example, some controllable properties of a system may include the depth of a first-in-first-out (FIFO) buffer, a memory size (e.g., in kilobytes KB, megabytes, MB, gigabytes GB, and the like) or the clock frequency at which certain circuit component operates, or a bus bandwidth. Other controllable properties may include a bandwidth of a port (input/output) in a bus or a memory interface, or a read/write weight ratio in the port.
A bus bandwidth corresponds with a bus width.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Fine and Kinderman. One having ordinary skill in the art would have found motivation to use configuration variables including a bus bandwidth into the system of coverage directed test generation for the advantageous purpose to “sets proper values for the controllable properties to optimize the design.” See Kinderman column 5 lines 1-9.
Claim 5 further recites “5. The method as recited in claim 1, wherein generating the optimized parameter set comprises using a black-box optimizer.” Fine column 9 lines 45-47 disclose “CDG engine 90 may operate with any test verification system 92 which has inputs 94 and outputs 96.” A test verification system which only has inputs and outputs corresponds with a black-box system. Accordingly, the CDG being able to operate with a test verification system with only inputs and outputs corresponds with the generated optimization of the parameter set using a black-box optimizer. The CDG here is a black-box optimizer because it is able to operate with the black-box test verification system.
Claim 7 further recites “7. The method as recited in claim 5, wherein generating the optimized parameter set further comprises: generating one or more parameters of the optimized parameter set using a Bayesian optimization algorithm based on: previous parameter sets used to perform previous regressions on the design under test; and analyzed results from the previous regressions using the previous parameter sets.” Fine column 3 lines 46-50 disclose:
may receive coverage information 25 and coverage reports 26 as input and may query Bayesian network 38 to determine possible directives 16' to send to test generator 10 to create a new set of tests 18' which may, in the next simulation
Querying the Bayesian network corresponds with using a Bayesian optimization algorithm. The coverage reports correspond with analyzed results of previous regressions. The coverage information corresponds with previous parameter sets used. See further Fine column 6 lines 26-43.
Claim 8 further recites “8. The method as recited in claim 7, wherein the Bayesian optimization algorithm comprises: a statistical model used to approximate results of a regression using a specific set of parameters.” Fine column 4 lines 46-60 disclose:
Typical types of queries that can be efficiently answered by the Bayesian network model are derived from applying the Bayes rule to yield posterior probabilities for the values of a node (or set of nodes) X, given some evidence ….
a statistical inference can be made in the form of either selecting the maximal a posteriori (MAP) probability, max p(XIE), or obtaining the most probable explanation (MPE), arg max p(XIE).
The Bayesian network model is a statistical model. The posterior probabilities and statistical inference probabilities are approximate results. The given evidence of values of other nodes is using a specific set of parameters.
Claim 8 further recites “and an acquisition function used to determine an optimized set of parameters for maximizing results of the regression.” Fine column 6 lines 26-31 disclose:
In the evaluation phase, CDG engine 30 may utilize the trained Bayesian network to determine directives for a desired coverage task. For example, CDG task manager 32 (FIG. 2) may utilize posterior probabilities and MAP and MPE queries and may utilize the coverage task attributes as evidence.
Using a maximal a posteriori (MAP) probability corresponds with determining a maximizing result for the optimized parameters. The determined directives correspond with the parameters.
Claim 13 further recites “13. The method as recited in claim 1, further comprising: prior to performing the first regression: receiving one or more subsets of parameters, each of the one or more subsets of parameters comprising a number of parameters less than or equal to a total number of parameters in the first set of parameters.” Fine column 1 lines 22-27 disclose “Members of the coverage model are called ‘coverage tasks’ and are considered part of the coverage model. These models are defined by a basic event and a set of parameters or attributes, where the list of coverage tasks comprises all permissible combinations of values for the attributes.”
Fine column 6 lines 1-7 disclose:
A designer of CDG Bayesian network 38 may start by identifying the ingredients (attributes) of directives 16' and of the coverage model 12. These attributes are dictated by the interface to random test generator 10 (FIG. 2) (e.g. directives 16'), to coverage analysis tool 24, and by the specification of coverage model 12. These ingredients may be used to define a first set of nodes in the graph.
The identified attributes of the directives and coverage model by the specification of the coverage model correspond with a total number of parameters.
Fine column 7 lines 37-39 disclose “The training set contained 385 randomly chosen different instructions out of 449 possible instructions.” The training set containing only 385 out of 449 is a subset less than a total number of parameters.
Claim 13 further recites “receiving analyzed results of at least one previous regression using each of the one or more subsets of parameters; and generating the first and any subsequent parameter sets based on at least the analyzed results of each previous regression using each of the one or more subsets of parameters.” Fine column 6 lines 26-31 discloses:
In the evaluation phase, CDG engine 30 may utilize the trained Bayesian network to determine directives for a desired coverage task. For example, CDG task manager 32 (FIG. 2) may utilize posterior probabilities and MAP and MPE queries and may utilize the coverage task attributes as evidence.
Using the trained Bayesian network corresponds with receiving previous analysis for the subsets of parameters and generating the first and subsequent parameter sets based on the previous training.
Claim 16 further recites “16. The method as recited in claim 1, wherein analyzing the results of the first regression includes determining an accumulated coverage percentage.” Fine column 3 lines 36-40 disclose:
Test generator 10 may produce new tests 18' which will be used by simulator 20 to stimulate and/or operate design under test 22. The simulation output of simulator 20 may be analyzed by coverage analysis tool 24 and the output may be provided to CDG engine 30.
The output of simulator being analyzed by coverage analysis tool corresponds with analyzing results of the first regression to determine a coverage.
Fine does not explicitly disclose using a percentage; however, in analogous art of simulation analysis of SoC, Kinderman column 12 lines 1-8 teaches “FIG. 8 illustrates a distribution progression 800 after multiple iterations…. The abscissae in distribution progression 800 (X-axis) indicate an iteration step number 801, and the ordinates (Y-axis) indicate a target coverage 802 (e.g., percentage, %, of desired coverage).” A target coverage of a percentage is a coverage percentage. The distribution progression corresponds with an accumulated coverage percentage over the multiple iterations.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Fine and Kinderman. One having ordinary skill in the art would have found motivation to use a percentage for a desired coverage into the system of coverage directed test generation for the advantageous purpose of quantifying progression over multiple iterations. See Kinderman column 12 lines 1-20.
Claim 17 further recites “17. The method as recited in claim 1, wherein each parameter of the optimized set of parameters has a predetermined domain.” Fine column 1 lines 22-27 disclose “Members of the coverage model are called ‘coverage tasks’ and are considered part of the coverage model. These models are defined by a basic event and a set of parameters or attributes, where the list of coverage tasks comprises all permissible combinations of values for the attributes.” The permissible combination of values corresponds with a predetermined domain for the respective attributes. The attributes correspond with the set of parameters subsequently used to determine the optimized directives.
Claim 18 further recites “18. The method as recited in claim 1, wherein the design under test is a hardware device, the hardware device selected from the group consisting of: a processing unit, the processing unit including a machine-learning processor; memory; or a cache controller.” From the above list of alternatives Examiner is selecting “memory.”
Fine column 8 lines 21-23 disclose “The design under test 22 for the second experiment is the Storage Control Element (SCE) 80 of an IBM z-series system, shown in FIG. 7.” Fine column 8 lines 32-34 disclose “The simulation environment in this experiment also comprised behavioral models for the eight CPUs that SCE 80 services and a behavioral model for a memory subsystem 82.” A storage control element with a behavioral model for a memory subsystem corresponds with a memory or cache controller.
See further, Fine column 6 lines 49-50 disclose “The first experiment modeled a subset 50 of the pipelines of NorthStar, an advanced PowerPC processor.”
Claim 20 further recites “20. The method as recited in claim 1, further comprising: modifying the design under test as a result of the first regression, the subsequent regression, or another regression.” Fine does not explicitly disclose modifying the design under test from the regression results; however, in analogous art of simulation analysis of SoC, Kinderman column 14 lines 1-3 teach “step 1004 includes modifying the machine-learning engine based on a predicted quality of the first configuration and an evaluated quality of the first configuration.” Modifying the machine-learning engine based on a predicted performance quality corresponds with modifying the design under test as a result of the regression results.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Fine and Kinderman. One having ordinary skill in the art would have found motivation to use design modification into the system of coverage directed test generation for the advantageous purpose to “sets proper values for the controllable properties to optimize the design” and to ensure a quality is above a selected threshold See Kinderman column 5 lines 1-9 and column 14 lines 1-38.
Claim 22 further recites “22. The method as recited in claim 1, wherein the optimized set of parameters comprises at least one hardware condition related to the logical representation of the hardware system of the design under test.” Fine column 8 lines 21-23 disclose “The design under test 22 for the second experiment is the Storage Control Element (SCE) 80 of an IBM z-series system, shown in FIG. 7.”
But Fine does not explicitly disclose hardware configuration variables of the DUT; however, in analogous art of simulation analysis of SoC, Kinderman column 4 line 67 to column 5 line 7 teaches:
For example, some controllable properties of a system may include the depth of a first-in-first-out (FIFO) buffer, a memory size (e.g., in kilobytes KB, megabytes, MB, gigabytes GB, and the like) or the clock frequency at which certain circuit component operates, or a bus bandwidth. Other controllable properties may include a bandwidth of a port (input/output) in a bus or a memory interface, or a read/write weight ratio in the port.
A bus bandwidth corresponds with a hardware condition.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Fine and Kinderman. One having ordinary skill in the art would have found motivation to use configuration variables including a bus bandwidth into the system of coverage directed test generation for the advantageous purpose to “sets proper values for the controllable properties to optimize the design.” See Kinderman column 5 lines 1-9.
Dependent Claim 6
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Fine and Kinderman as applied to claim 5 above, and further in view of US patent 11,755,799 B1 Ganai [herein “Ganai”].
Claim 6 further recites “6. The method as recited in claim 5, wherein generating the optimized parameter set further comprises: selecting, for one or more parameters of the optimized parameter set, a value from a uniformly distributed set of values using a random seed.” Fine column 8 lines 43-48 disclose:
Each directive contained a set of possible values that the directive may receive. Each value had a weight associated with it. When the value of a directive was needed, test generator 10 randomly chose the value from the set of possible values according to the weights of these values.
A uniform distribution would correspond with equal weights. But Fine does not explicitly disclose a uniform distribution; however, in analogous art of circuit design verification to meet coverage goals, Ganai column 2 lines 15-16 teaches “can generate constrained random stimuli during functional verification of a DUY.” Ganai column 2 lines 30-34 teach “random solutions can be generated by a constraint satisfaction problem solver. In some embodiments, the intended probability distribution can be a uniform or non-uniform probability distribution.” Generating solutions with a uniform distribution corresponds with selecting values for the parameters from a uniformly distribution using a random seed.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Fine, Kinderman, and Ganai. One having ordinary skill in the art would have found motivation to use a uniform probability distribution into the system of coverage directed test generation for the advantageous because using a uniform distribution is art recognized equivalent to using a non-uniform distribution. See Ganai column 2 lines 30-34 and MPEP §2144.06(II).
Dependent Claims 9-11
Claims 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over Fine and Kinderman as applied to claims 7 and 8 above, and further in view of Srinivas, N., et al. “Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting” IEEE Transactions on Info. Theory, vol. 58, no. 5 (2012) [herein “Srinivas”].
Claim 9 further recites “9. The method as recited in claim 8, wherein the acquisition function comprises: a mean coverage percentage for a specific value of a parameter based on the analyzed results from the previous regressions using the previous parameter sets; and an uncertainty coverage percentage for a specific value of a parameter based on an uncertainty of the statistical model using the analyzed results from the previous regressions using the previous parameter sets.” Fine does not explicitly disclose an acquisition based on a mean and uncertainty; however, in analogous art of optimizing experimental design with exploration/exploitation tradeoffs, Srinivas page 3253 right column algorithm 1 teaches:
PNG
media_image1.png
200
400
media_image1.png
Greyscale
μ
t
x
corresponds with a mean coverage for a specific value x.
σ
t
x
corresponds with an uncertainty for a specific value x. Performing the Bayesian update between time steps means subsequent acquisitions are based on analyzed results of previous regression iterations.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Fine, Kinderman, and Srinivas. One having ordinary skill in the art would have found motivation to use GP-UCB optimization into the system of coverage directed test generation for the advantageous purposes because “GP-UCB compares favorably with other heuristical GP optimization approaches” for applications requiring optimizing an unknown, noisy function that is expensive to evaluate. See Srinivas abstract.
Claim 10 further recites “10. The method as recited in claim 9, wherein the uncertainty coverage percentage is scaled by a negotiating constant comprising: an exploitation mode corresponding to a smaller negotiating constant; or an exploration mode corresponding to a larger negotiating constant.” Fine does not explicitly disclose an acquisition based on a mean and uncertainty; however, in analogous art of optimizing experimental design with exploration/exploitation tradeoffs, Srinivas page 3253 right column algorithm 1 teaches:
PNG
media_image1.png
200
400
media_image1.png
Greyscale
μ
t
x
corresponds with a mean coverage for a specific value x.
σ
t
x
corresponds with an uncertainty for a specific value x. Performing the Bayesian update between time steps means subsequent acquisitions are based on analyzed results of previous regression iterations.
β
t
corresponds with a scalar negotiating constant where a larger
β
t
“implicitly negotiates the exploration-exploitation tradeoff.” Srinivas page 3253 left column first paragraph. The exploration-exploitation tradeoff corresponds to exploitation and exploration modes.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Fine, Kinderman, and Srinivas. One having ordinary skill in the art would have found motivation to use GP-UCB optimization into the system of coverage directed test generation for the advantageous purposes because “GP-UCB compares favorably with other heuristical GP optimization approaches” for applications requiring optimizing an unknown, noisy function that is expensive to evaluate. See Srinivas abstract.
Claim 11 further recites “11. The method as recited in claim 7, wherein the Bayesian optimization algorithm is a Gaussian Process Bandits Bayesian optimization algorithm.” Fine does not explicitly disclose a Gaussian Process Bandits Bayesian optimization algorithm; however, in analogous art of optimizing experimental design with exploration/exploitation tradeoffs, Srinivas title discloses “Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting.” Srinivas page 3253 right column algorithm 1 teaches:
PNG
media_image1.png
200
400
media_image1.png
Greyscale
Performing the Bayesian update between time steps corresponds with being Bayesian. Accordingly, the GP-UCB algorithm is a Gaussian Process Bandits Bayesian optimization algorithm.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Fine, Kinderman, and Srinivas. One having ordinary skill in the art would have found motivation to use GP-UCB optimization into the system of coverage directed test generation for the advantageous purposes because “GP-UCB compares favorably with other heuristical GP optimization approaches” for applications requiring optimizing an unknown, noisy function that is expensive to evaluate. See Srinivas abstract.
Dependent Claim 12
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Fine and Kinderman as applied to claim 1 above, and further in view of US patent 10,699,046 B2 Green (cited in IDS dated 10 April 2023) [herein “Green”].
Claim 12 further recites “12. The method as recited in claim 1, further comprising: performing, for each regression of the first regression and the subsequent regression, a parallel regression using a predetermined set of parameters.” Fine does not explicitly disclose performing a parallel regression testing; however, in analogous art of coverage and verification of a design under test, Green column 13 lines 54-58 teaches “Distributed learning may be characterized as a process by which a learning test generator can be trained by employing several computing devices in parallel which work together to implement the learning process described.” Executing the test generator in parallel corresponds with performing at least one regression in parallel with respective parameters.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Fine, Kinderman, and Green. One having ordinary skill in the art would have found motivation to use parallel processing into the system of coverage directed test generation because:
The distributed learning process creates several advantages. Using distributed learning with multiple computing devices can enable the learning process to achieve greater accuracy on large data sets than if only a single computing device were employed to implement the learning process. Distributed learning enables the learning process to train itself without being limited by the computer speed, stability, or storage capability of a single computing device.
See Green column 14 lines 6-13.
Dependent Claim 19
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Fine and Kinderman as applied to claim 1 above, and further in view of Ziv, H. & Richardson, D. “Constructing Bayesian-network Models of Software Testing and Maintenance Uncertainties” IEEE Proceedings 13th Int’l Conf. on Software Maintenance, pp. 100-109 (1997) [herein “Ziv”].
Claim 19 further recites “19. The method as recited in claim 1, wherein the design under test is an algorithm or software application.” The design under test being an algorithm or software is an intended use recitation. See MPEP §2111.02.
From the above list of alternatives Examiner is selecting “software application.”
Fine does not explicitly disclose software design; however, in analogous art of software testing with Bayesian models, Ziv title teaches “Constructing Bayesian-network Models of Software Testing and Maintenance Uncertainties.” Software testing corresponds with a design under test of a software application.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Fine, Kinderman, and Ziv. One having ordinary skill in the art would have found motivation to use testing of software into the system of coverage directed test generation for the advantageous purpose “to either confirm, evaluate, or predict software uncertainties.” See Ziv page 101 left column fourth paragraph. Furthermore, Fine coverpage cites Ziv as a reference.
Dependent Claim 21
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Fine and Kinderman as applied to claim 1 above, and further in view of Vance, J. “Applications of Bayesian Networks to Coverage Directed Test Generation for the Verification of Digital Hardware Designs” Thesis, U. Pittsburgh (2010) (cited in IDS dated 10 April 2023) [herein “Vance”].
Claim 21 further recites “21. The method as recited in claim 1, further comprising: manufacturing an article according to the design under test.” Fine does not explicitly disclose manufacturing the design under test; however, in analogous art of coverage directed test generation (CDG), Vance page 9 first paragraph teaches “Most of today’s integrated circuit designs have a state space so large that it is not possible to exhaustively test all functions under all plausible scenarios on the final chip that is manufactured.” Manufacturing a final chip after the design has been tested corresponds with manufacturing an article according to the respective integrated circuit design under test.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Fine, Kinderman, and Vance. One having ordinary skill in the art would have found motivation to use manufacturing a final design into the system of coverage directed test generation for the advantageous purpose because “[b]ugs not found until the design is manufactured in silicon are often expensive to fix.” See Vance page 2 second paragraph.
Conclusion
Prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Takakis, Z., et al. “Dynamic Adjustment of Test-Sequence Duration for Increasing the Functional Coverage” IEEE 4th Int’l Verification & Security Workshop, pp. 61-66 (2019) doi: 10.1109/IVSW.2019.8854389
teaches
Constrained-random test sequences adjusted by feedback of observed quality of each applied test.
Optimizing test duration per-cycle.
Šimková, M. “Automation and Optimization of Coverage-driven Verification” Euromicro Conf. on Digital Sys. Design, pp. 87-94 (2015)
Coverage driven verification (CDV).
Genetic algorithm optimization of coverage.
Fournier, L., et al. “A Probabilistic Analysis of Coverage Methods” ACM Transactions on Design Automation of Electronic Sys., vol. 16, no. 4, article 38 (2011)
Review of commonly used sampling strategies for coverage.
Multi-armed bandit.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jay B Hann whose telephone number is (571)272-3330. The examiner can normally be reached M-F 10am-7pm EDT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Renee Chavez can be reached at (571) 270-1104. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Jay Hann/Primary Examiner, Art Unit 2186 6 June 2026
Prosecution Insights