DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-4, 6, 8-14, 16, 18-20 are pending for examination.
This Office Action is FINAL.
Priority
Acknowledgment is made of applicant's claim for foreign priority based on an application filed in IN on 12/21/2023 and 10/04/2024. It is noted, however, that applicant has not filed a certified copy of the IN202321087767 and IN202421075245 applications as required by 37 CFR 1.55.
Specification
The disclosure is objected to because of the following informalities:
[0024] line 6: change to "along with random [[testcase]] test cases"
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6, 9-14, 16, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (NPL: "LLM4DV: Using Large Language Models for Hardware Test Stimuli Generation," as previously cited, hereinafter “Zhang”) in view of Mishra (NPL: “System-Level Register and Sequence Verification with UVM and Embedded C/C++,” as previously cited), in view of FastBitLab (NPL: “Microcontroller Embedded C Programming Lecture 78 | Operators in ‘C’”).
Regarding Claim 1, Zhang discloses a method (Page 2, Fig. 1b; Page 3: LLM4DV framework and method), comprising:
generating, by a processor of an apparatus (Page 5, Section 4: "With Verilator and cocotb, we could efficiently simulate and test the modules in Python." One of ordinary skill in the art would understand that running LLM4DV and various programs requires a processor and a computing apparatus), one or more random test cases with an aid of a generative artificial intelligence (GAI)-assisted random verification program (GRVP) framework (Page 2: "We establish a benchmarking framework named LLM4DV (Large Language Model for Design verification… we propose and develop a framework that uses prompted LLMs to generate test stimuli for hardware designs." Page 2, Fig. 1b, Page 3, Fig. 2 and Pages 5-6, Sections 4.1-4.3: generates test stimuli using pure random, coverpoint type-based, and mixed coverpoint type-based sampling to expose uncovered bins. Page 9, B1: all three missed-bin sampling methods either are or eventually use random sampling. Thus, the generated test stimuli by LLM4DV are random test stimuli to expose the randomly-sampled uncovered bins); and
performing, by the processor (see above details on LLM4DV + required processor to run various programming languages and libraries), validation testing on a design of a processor using the one or more random test cases (Page 2, first bullet: "We design and construct three DUT modules: a Primitive Data Prefetcher Core, an Ibex Instruction Decoder, and an Ibex CPU." The DUT modules are designs of components of a processor (i.e., Primitive Data Prefetcher, Instruction Decoder) or a processor design itself (i.e., CPU). Page 2, third bullet and Page 8, algorithm 1: “We show that our framework with optimal improvement measures achieves 98.94%, 86.19%, and 5.61% coverage rates on these [DUT] modules." Coverage rates are achieved by performing design verification using the aforementioned (random-based) test stimuli on the DUT modules).
Zhang does not disclose:
wherein the generating of the one or more random test cases comprises utilizing a C or C++-based random verification program (CRVP) generator to generate one or more random verification programs in C, wherein the one or more random verification programs are instruction set architecture (ISA)-independent, and wherein the CRVP generator is configured to generate random expressions for arithmetic and logical operations.
However, Mishra teaches:
wherein the generating of the one or more random test cases comprises utilizing a C or C++-based random verification program (CRVP) generator to generate one or more random verification programs in C (Page 3: IDS-NG generates C/C++ random value tests, where users will run these tests on their own model of the chosen CPU for their SoC [design] (see Page 2). Validating SoC design containing a CPU model encompasses testing the CPU model along with the remaining SoC design), wherein the one or more random verification programs are instruction set architecture (ISA)-independent (C is ISA independent), and wherein the CRVP generator is configured to generate random expressions (Page 3: generates C/C++ random value tests)…
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Zhang and Mishra performing a simple substitution of one known element (Zhang: Page 2, Fig. 1b; Page 3-4, Section 3.1; Page 5, Section 4; Page 11, Section D.2: generate test stimuli in Python targeting random-sampled uncovered bins for Ibex Instruction Decoder and register verification) for another (Mishra: Page 3: C/C++ random value tests for register verification) to obtain predictable results (generating random tests for register coverage in a specific language).
Zhang in view of Mishra does not teach:
…for arithmetic and logical operations
However, FastBitLab teaches:
…for arithmetic and logical operations (Page 11: expression “12 + 3 – 4 / 2 < 3 + 1” with “+, -, /” as arithmetic operations and “<” as logical operations)
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Zhang, Mishra, and FastBitLab by implementing the arithmetic and logical operators taught by FastBitLab. One of ordinary skill in the art would be motivated to make this modification in order to verify operator precedence (FastBitLab: Page 11-12).
Regarding Claim 2, Zhang in view of Mishra, in further view of FastBitLab teaches the method of Claim 1, as referenced above, wherein the generating of the one or more random test cases comprises utilizing a retrieval augmented generator (RAG) in the GRVP framework (Zhang: Page 5, Section 4; Page 6, Section 4.2; Page 11, Section D.2: LLM4DV framework uses OpenAI’s GPT 3.5 pre-trained with RISC-V knowledge) to perform operations comprising:
receiving a template-based user input along with one or more random feature specifications (Zhang: Page 4, Section 3.2: receives coverage-feedback template, including iterative user query messages containing “a list of uncovered bins.” Page 4-5, Section 3.3; Page 9, Section B.1: the uncovered bins will later be randomly sampled to improve test stimuli generation);
receiving one or more application programming interfaces (APIs) (Zhang: Page 11, Section D.2: GPT 3.5 and Llama 2 are pre-trained with RISC-V knowledge, including ALU operation bins ADD, ADDI, XOR, etc. Pretrained encompasses receiving knowledge on ALU operations/API);
learning from documents and examples (Zhang: Page 11, Section D.2: GPT 3.5 pretrained with RISC-V knowledge. RISC-V is a known instruction set architecture (ISA), defined in documents/source code. Page 4-5, Section 3.2-3.3: also learns from previous responses (examples)); and
prompting with an aid of relevant chunks shared to a large language model (LLM) in the GRVP framework (Zhang: Page 2; Fig. 1b; Page 4, Section 3.1: "In each generation cycle, the prompt generator generates a prompt based on a template and the current coverage feedback from the coverage monitor. The LLM takes in the prompt and generates a natural language response." Template and current coverage feedback are relevant chunks shared to the LLM in the LLM4DV framework to create new responses).
Regarding Claim 3, Zhang in view of Mishra, in further view of FastBitLab teaches the method of Claim 2, as referenced above, wherein the generating of the one or more random test cases further comprises:
utilizing the LLM to generate one or more test cases along with documentation based on an input received from the RAG (Zhang: Page 2, Fig. 1b; Page 8, algorithm 1: LLM generates test stimulus input (“test case”), which is then inserted into the testbench to generate a coverage rate (documentation) based on the current coverage input received from the previous LLM response); and
generating the one or more random test cases in a predefined language based on the one or more test cases generated by the LLM (Zhang: Page 3, Fig. 2; Page 5, first paragraph; Page 5, Section 4: generates test stimuli using random sampling of previous responses (previous test stimuli) from one of the three sampling methods to reach uncovered bins. The test stimuli are performed in Python. Thus, random-bin test stimuli are generated in Python based on previous test stimuli generated by the LLM).
Regarding Claim 4, Zhang in view of Mishra, in further view of FastBitLab teaches the method of Claim 2, as referenced above, wherein the utilizing of the RAG further comprises utilizing the RAG to perform operations comprising:
utilizing the LLM to generate one or more test cases along with documentation based on an input received from the RAG (Zhang: Page 2, Fig. 1b; Page 8, algorithm 1: LLM generates test stimulus input (“test case”), which is then inserted into the testbench to generate a coverage rate (documentation) based on the current coverage input received from a previous LLM response);
integrating reference libraries (Zhang: Page 5, Section 4: uses Verilator and cocotb); and
learning examples with the APIs to enhance generation of random test programs (Zhang: Page 2, Fig. 1b; Pages 4-5, Section 3.3: learns from previous responses and randomly sampled uncovered bins to improve test stimuli generation with respect to exposing the uncovered bins. Page 11, Section D.2: when testing Ibex Instruction Decoder, the previous responses may be test inputs for ALU operation bins. The next generation cycle generates test stimuli using the “prompting improvements” (see Page 2, Fig. 1b). Therefore, the LLM learns from previous responses (examples) containing ALU operations (ALU API) to enhance generation of randomly-sampled test bin programs for higher coverage).
Regarding Claim 6, Zhang in view of Mishra, in further view of FastBitLab teaches the method of Claim 1, as referenced above, wherein the one or more random verification programs in C are used in validating the design of the processor (Mishra: Page 3: IDS-NG generates C/C++ random value tests, where users will run these tests on their own model of the chosen CPU for their SoC [design] (see Page 2). Validating SoC design containing a CPU model encompasses testing the CPU model along with the remaining SoC design).
Regarding Claim 9, Zhang in view of Mishra, in further view of FastBitLab teaches the method of Claim 2, as referenced above, wherein the generating of the one or more random test cases further comprises utilizing a Python-based random verification program (PyRVP) generator to generate the one or more APIs (Zhang: Page 5, Section 4: “With Verilator and cocotb, we could efficiently and test the modules in Python.” Page 8, Algorithm 1; Page 11, Section D.2: Ibex Instruction Decoder verification involves generating test stimuli to cover ALU operation bins, wherein the generated stimuli is inputted into a testbench to calculate the resulting coverage. Pages 4-5, Section 3.3; Page 6, Section 4.2: Ibex Instruction Decoder verification uses Mixed Coverpoint Type-based and Pure Random Sampling to improve stimuli generation. Therefore, the LLM generates test stimuli (verification program generator) in Python, where the test stimuli are generated ALU operations/ALU API later inputted into a testbench. The LLM is also a random verification program generator as missed-bin sampling randomly samples uncovered bins to allow the LLM generate test stimuli and prioritize said random bins).
Regarding Claim 10, Zhang in view of Mishra, in further view of FastBitLab teaches the method of Claim 9, as referenced above, wherein the generating of the one or more random test cases further comprises utilizing the PyRVP generator to generate one or more random verification programs in Assembly that are used in validating the design of the processor (Page 6, Section 4.2; Page 11, Section D.2: generates test stimuli for ALU operation bin coverage when verifying Ibex Instruction Decoder of RISC-V processor design. ALU operations are Assembly operations).
Regarding Claim 11, the method of Claim 1 performs the same steps as the apparatus of Claim 11, and Claim 11 is rejected using the same art and rationale set forth above in the rejection of Claim 1 by the teachings of Zhang in view of Mishra, in further view of FastBitLab.
Zhang further discloses an apparatus (Page 5, Section 4: LLM4DV uses Verilator, cocotb, Python, and GPT 3.5. One of ordinary skill in the art would understand that a computer is used to run LLM4DV), comprising:
a memory configured to store information, data and instructions (see above – one of ordinary skill in the art would understand that memory containing data and instructions are required to execute LLM4DV and algorithm 1 (page 8));
and a processor coupled to the memory and configured to perform operations (see above – understood that a processor executes the data and instructions).
Regarding Claim 12, Zhang teaches the apparatus of Claim 11 above. The method of Claim 2 performs the same steps as the apparatus of Claim 12, and Claim 12 is rejected using the same art and rationale set forth above in the rejection of Claim 2 by the teachings of Zhang in view of Mishra, in further view of FastBitLab.
Regarding Claim 13, Zhang teaches the apparatus of Claim 12 above. The method of Claim 3 performs the same steps as the apparatus of Claim 13, and Claim 13 is rejected using the same art and rationale set forth above in the rejection of Claim 3 by the teachings of Zhang in view of Mishra, in further view of FastBitLab.
Regarding Claim 14, Zhang teaches the apparatus of Claim 12 above. The method of Claim 4 performs the same steps as the apparatus of Claim 14, and Claim 14 is rejected using the same art and rationale set forth above in the rejection of Claim 4 by the teachings of Zhang in view of Mishra, in further view of FastBitLab.
Regarding Claim 16, Zhang teaches the apparatus of Claim 11 above. The method of Claim 6 performs the same steps as the apparatus of Claim 16, and Claim 16 is rejected using the same art and rationale set forth above in the rejection of Claim 6 by the teachings of Zhang in view of Mishra, in further view of FastBitLab.
Regarding Claim 19, Zhang teaches the apparatus of Claim 11 above. The method of Claim 9 performs the same steps as the apparatus of Claim 19, and Claim 19 is rejected using the same art and rationale set forth above in the rejection of Claim 9 by the teachings of Zhang in view of Mishra, in further view of FastBitLab.
Regarding Claim 20, Zhang teaches the apparatus of Claim 19 above. The method of Claim 10 performs the same steps as the apparatus of Claim 20, and Claim 20 is rejected using the same art and rationale set forth above in the rejection of Claim 10 by the teachings of Zhang in view of Mishra, in further view of FastBitLab.
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang, in view of Mishra, in view of FastBitLab, in in further view of Tseng (US 20090164861 A1, as previously cited), in further view of 123calculus (NPL: “Random Polynomial Generator,” as previously cited).
Regarding Claim 8, Zhang in view of Mishra teaches the method of Claim 1, as referenced above, wherein the utilizing of the CRVP generator to generate one or more C-random verification programs comprises utilizing the CRVP generator to perform operations (Mishra: Page 3: IDS-NG performs the operations cited below) comprising:
running application and operating system (OS) fragments on simulators (Mishra: Page 3: “we also ship with Zephyr, an open-source real-time operating system (RTOS), so that users can run applications in the simulation environment”);
collecting snapshots of input and output registers… (Mishra: Page 5: coverage results gathered during test simulation are reported, including results for reading and writing values from registers. Coverage results encompass collecting snapshots (Page 4, IDS-NG Sequence View: read register value at a point in time) of read/input and write/output registers to calculate said coverage results
PNG
media_image1.png
206
416
media_image1.png
Greyscale
);
creating random expressions… (Mishra: Page 3: create random value test); and calculating outputs using known values as inputs (Mishra: Pages 4-5: calculating coverage results using known/defined arguments, constants variables, etc. as inputs for a test sequence).
Zhang in view of Mishra does not teach:
collecting snapshots of… control and status registers (CSRs);
…an order of 3 or 4;
However, Tseng teaches:
collecting snapshots of… control and status registers (CSRs) ([0021]: polls internal registers by repeatedly reading a status bit in one of the design’s internal registers until the bit is set to determine when the test completes);
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Zhang, Mishra, FastBitLab, and Tseng by implementing the internal register polling taught by Tseng. One of ordinary skill in the art would be motivated to make this modification in order to determine when the test completes (Tseng: [0021]).
Zhang in view of Mishra, in view of FastBitLab, in further view of Tseng does not teach:
…an order of 3 or 4
However, 123calculus teaches:
creating random expressions of an order of 3 or 4 (Page 1: random polynomial generator of degree 3);
Furthermore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Zhang, Mishra, FastBitLab, Tseng, and 123calculus by implementing the random polynomial generator taught by 123calculus. One of ordinary skill in the art would be motivated to make this modification in order to create a random expression that uses exponential operators to test ALU operations (123calculus: Page 1).
Regarding Claim 18, Zhang in view of Mishra, in further view of FastBitLab teaches the apparatus of Claim 11 above. The method of Claim 8 performs the same steps as the apparatus of Claim 18, and Claim 18 is rejected using the same art and rationale set forth above in the rejection of Claim 8 by the teachings of Zhang in view of Mishra, in view of FastBitLab, in further view of Tseng, in further view of 123calculus.
Response to Arguments
Applicant’s arguments filed on 04/07/2026 have been fully considered but they are not persuasive.
On Pages 16-20, Applicant argues:
Page 16-17: “With respect to the rejection under 35 U.S.C. $103 based on Zhang in view of Mishra, the Examiner's rationale is based on an improper combination of teachings that lack both technical compatibility and a sufficient motivation to combine. The Examiner asserts that Mishra teaches utilizing a C or C++-based generator to produce random test APIs, and that such teaching may be substituted into Zhang. However, Mishra does not disclose a generator for creating verification programs. Instead, Mishra discloses predefined C/C++ test cases constructed using an API library for register access, which fundamentally differs from a CRVP generator that automatically synthesizes random verification programs. APIs merely serve as building blocks, whereas the claimed CRVP generator defines a higher-level program synthesis architecture. Moreover, the proposed substitution of Zhang's Python-based stimuli generation with Mishra's C/C++ test cases does not constitute a simple substitution of known elements. Zhang's framework is directed to coverage-driven stimuli generation, while Mishra's approach focuses on executing predefined test programs for register verification. These two systems operate at different abstraction layers and serve different technical purposes. Accordingly, one of ordinary skill in the art would not have been motivated to combine these teachings in the manner proposed by the Examiner.”
Page 18: “Accordingly, the Examiner's reliance on the general characteristic of C as a high- level language is insufficient to establish that Mishra teaches or suggests the claimed ISA-independent verification programs. The Examiner effectively equates language abstraction with architecture independence, which is technically incorrect and fails to account for execution semantics and verification methodology. The test programs in Mishra remain dependent on specific architectures and execution environments, and therefore do not disclose the ISA-independent characteristics claimed in the present application.”
Page 19: “With respect to the additional limitations introduced in dependent claims, including operations such as collecting snapshots of registers, generating random expressions, and calculating outputs, the Examiner relies on Tseng and 123calculus to fill these gaps. However, these references are from distinct and non-analogous technical domains. Tseng relates to testbench control mechanisms involving polling internal registers to determine test completion, which is unrelated to program generation, structured expression construction, or snapshot-based verification semantics. Similarly, 123calculus is directed to mathematical polynomial generation and does not pertain to hardware verification, processor testing, or program synthesis.”
Page 20: “Additionally, the cited references collectively teach away from the claimed invention. Zhang emphasizes lightweight, stimuli-based generation rather than structured program synthesis. Mishra focuses on deterministic, API-based test construction tied to specific architectures. Tseng is directed to execution monitoring rather than generation mechanisms, and 123calculus is unrelated to verification altogether. These teachings reinforce approaches that are architecture-dependent and low-level, in contrast to the present application, which introduces a program-level, ISA-independent, and generator- driven verification framework. Accordingly, the cited references not only fail to suggest the claimed invention, but also discourage the direction taken by the present application.”
Examiner respectfully disagrees with A-D.
Regarding A, under BRI, IDS-NG is a C/C++ (random value) test generator for hardware/CPU verification (Mishra: Page 3). Furthermore, Zhang discloses generating test stimuli to expose uncovered bins (Page 2, Fig. 1b, Page 3, Fig. 2 and Pages 5-6, Sections 4.1-4.3). Page 11, Section D.2 describes bins for testing an Ibex Instruction Decoder (e.g., ALU operations ADD, ADDI, XOR, LW; register ports; cross coverage ALU x port). However, it is unclear how the test stimuli are generated after the bins are randomly sampled. Substituting with IDS-NG allows random values to be generated to be used in, for example, the ALU operations and register read/write ports as taught by Zhang.
Regarding B, Mishra teaches the test cases are generated in C/C++. On Page 15 of the previous Office Action mailed 01/09/2026, Section “Prior Art of Record,” although not recited in the 103 rejections, Datta et al. (US 20040098575 A1) teaches C is an ISA higher level language ([0040]: "ISA independent higher level language like C"). Under BRI, it is unclear whether the “one or more random verification programs” must be ISA-independent at compile time as argued by the Applicant on Page 18. Therefore, the C/C++ test cases (which are also programs containing C/C++ source code) are ISA-independent upon generation.
Regarding C, in response to applicant's argument that Tseng and 123calculusis nonanalogous art, it has been held that a prior art reference must either be in the field of the inventor’s endeavor or, if not, then be reasonably pertinent to the particular problem with which the inventor was concerned, in order to be relied upon as a basis for rejection of the claimed invention. See In re Oetiker, 977 F.2d 1443, 24 USPQ2d 1443 (Fed. Cir. 1992).
In this case, both modified Zhang and Tseng focus on testing, where modified Zhang is directed towards generating and executing tests and Tseng is directed towards executing generated tests. Tseng teaches polling internal registers with the motivation of combining to determine when testing completes ([0019]; [0021]: test bench includes test case generator 102; internal registers to determine when test completes), which is related to test case generation.
Furthermore, as described in (A) above, Zhang discloses generating test stimuli for ALU, register, and cross coverage bins, and Mishra provides the test stimuli as random test values. 123calculus further provides mathematical operators by generating random polynomials to the third exponential power (x^3 == x ** 3), which are known to be used in ALU operations and register ports. All three references are analogous to generating mathematical expressions, which is also consistent with [0028] of the instant specification and Claims 1, 8 to be used as tests.
Regarding D, please see A-C above for motivation to combine, how the references are analogous to one another, and how the C/C++ tests generated in Mishra are ISA-independent.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CATHERINE MARIE NGUYEN whose telephone number is (571)272-6160. The examiner can normally be reached M-F 7:30 AM - 4:30 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ASHISH THOMAS can be reached at (571) 272-0631. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.M.N./Examiner, Art Unit 2114
/ASHISH THOMAS/Supervisory Patent Examiner, Art Unit 2114
Prosecution Insights