DETAILED ACTION
This action is responsive to application filed on December 20, 2023, which is a National Stage entry of PCT/CN2021/121797, International filed on September 29, 2021.
The preliminary amendments filed on December 20, 2023 has been acknowledged and considered.
Claims 1, 4-11, 17, 19, 23 and 25 have been amended. Claims 15-16, 21-22 and 24 have been canceled.
Claims 1-14, 17-20, 23 and 25 are pending and are presented to examination.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Examiner Notes
Examiner cites particular columns, paragraphs, figures and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
Drawings
The drawings filed on December 20, 2023 are acceptable for examination purposes.
Information Disclosure Statement
As required by M.P.E.P. 609, the applicant’s submission of the Information Disclosure Statement dated December 20, 2023 is acknowledged by the examiner and the cited references have been considered in the examination of the claims now pending.
Specification
The abstract of the disclosure is objected to because the abstract of the disclosure does not commence on a separate sheet in accordance with 37 CFR 1.52(b)(4) and 1.72(b). A new abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text.
A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Claim Objections
Claims 1-14 and 17-20 are objected to because of the following informalities: Claim 1 recites “An apparatus for performing unit tests on firmware code, the firmware code being suitable for interacting with a hardware device, the apparatus comprising interface circuitry and processing circuitry to:” in lines 1-3. Please amend the claim language as “An apparatus for performing unit tests on firmware code, the firmware code being suitable for interacting with a hardware device, the apparatus comprising interface circuitry and processing circuitry configured to:” or “An apparatus for performing unit tests on firmware code, the firmware code being suitable for interacting with a hardware device, the apparatus comprising interface circuitry and processing circuitry executed to:”. Claim 17 recites “wherein the processing circuitry is to obtain the timeline of changes of the transaction data by simulating the hardware device based on the one or more simulation parameters, storing transaction data encountered at one or more interfaces of the hardware device during simulation of the hardware device, and determining the timeline of changes of the transaction data based on the stored transaction data.”. Please amend the claim language as “wherein the processing circuitry is configured to obtain the timeline of changes of the transaction data by simulating the hardware device based on the one or more simulation parameters, storing transaction data encountered at one or more interfaces of the hardware device during simulation of the hardware device, and determining the timeline of changes of the transaction data based on the stored transaction data.” or wherein the processing circuitry is executed to obtain the timeline of changes of the transaction data by simulating the hardware device based on the one or more simulation parameters, storing transaction data encountered at one or more interfaces of the hardware device during simulation of the hardware device, and determining the timeline of changes of the transaction data based on the stored transaction data.”.
Claim 19 recites “An apparatus for preparing data suitable for performing unit tests on firmware code, the firmware code being suitable for interacting with a hardware device, the apparatus comprising interface circuitry and processing circuitry to:”. Please amend the claim language as “An apparatus for preparing data suitable for performing unit tests on firmware code, the firmware code being suitable for interacting with a hardware device, the apparatus comprising interface circuitry and processing circuitry configured to:” or “An apparatus for preparing data suitable for performing unit tests on firmware code, the firmware code being suitable for interacting with a hardware device, the apparatus comprising interface circuitry and processing circuitry executed to:” Appropriate correction is required. Dependent claims 2-14, 18 and 20 do not overcome the deficiency of the base claim and, therefore, are objected for the same reasons as the base claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 25 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because the specification does not explicitly exclude signals or electromagnetic waves, which are non-statutory subject matters, from the claimed “machine-readable medium”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-8, 10-14, 17-20, 23 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Zidenberg et al. (US Pat. No. 11,307,882 – hereinafter Zidenberg) in view of Engebretsen et al. (US Pub. No. 2014/0173348 – hereinafter Engebretsen).
With respect to claim 1 (currently amended), Zidenberg teaches an apparatus for performing unit tests [[on firmware code]], [[the firmware code being suitable for interacting with a hardware device]], the apparatus comprising interface circuitry and processing circuitry (see figure 6) to: obtain a timeline of changes of transaction data encountered at one or more interfaces of the hardware device during simulation of the hardware device, the simulation being based on one or more simulation parameters defined by one or more unit tests [[to be performed on the firmware code]]; and (See figures 2, 3A-3B, 4 (and related text) and column 1 line 59 – column 2 line 15, “To reduce the amount of time needed to run a benchmark test for a pre-silicon integrated circuit design, the techniques disclosed herein leverage the ability of virtual machines to easily restore execution from a saved state. A full benchmark test can be run on a virtual machine of a hardware system that contains the actual integrated circuit device, and snapshots of the system state are captured at different times during execution of the benchmark test. When a pre-silicon design of the integrated circuit device becomes available, the performance of the design can be assessed by emulating the design in an emulator, and running snippets of the benchmark test on a virtual machine of the emulator. Each snapshot of the benchmark test taken with the real hardware can be loaded onto the emulator, and performance checkpoints are collected by running each snapshot for a short duration of runtime. The performance of the pre-silicon design can then be extrapolated from the performance checkpoints. Thus, instead of running the full benchmark test on the emulator, only snippets (e.g., portions) of the benchmark test are executed, and significant time savings can be achieved when evaluating the performance of a pre-silicon integrated circuit design.”. See column 3 lines 40-58, “To reduce the runtime of benchmarking a pre-silicon integrated circuit design on an emulator, the benchmark test is first executed on computer system 100 implementing a version of the integrated circuit device in silicon. In most cases, computer system 100 may contain an older version or generation of the chip being designed. During execution of the benchmark test on the real silicon, snapshots of the system state are taken sequentially at different times during execution of the test. The snapshots can be taken, for example, at a periodic interval, and may include a dump of the system memory (e.g., memory image of the system's DRAM) and the state of the virtual machine (e.g., the values of processor and/or peripheral registers, counters, interrupts, etc.). Each snapshot can be stored with a snapshot identifier to indicate the relative order of the snapshots and which benchmark test the snapshot was taken from. These snapshots of the system state can be used as checkpoints to replay a portion of the benchmark test.”. See column 4 lines 48-61, “Running a full benchmark test on emulator 202 to evaluate the performance of a hardware design may take a significantly long time and can be impractical in many situations. Instead of running the full duration of a benchmark test, snapshots of the system state taken at different times during execution of the benchmark test on real silicon can be loaded onto emulator 202, and a snippet of the benchmark test can be run on emulator 202. The performance of the hardware design can be measured for each of the short runs, and be extrapolated to derive a performance score representing the performance of running the full benchmark test on the hardware design. Additional details of the benchmarking techniques are described further with reference to FIGS. 3-4.”). perform the one or more unit tests [[of the firmware code]] using the timeline of changes of transaction data of the simulated hardware device, the one or more unit tests being based on the one or more simulation parameters (See figures 2-4 (and related text) and column 4 line 62 – column 5 line 10, “FIG. 3A illustrates a flow diagram of a process 300 for taking snapshots of a performance test (e.g., benchmark test) for execution on an emulator, according to some implementations. Process 300 is executed on real silicon of an actual host system to determine the state of the system during execution of the performance test. The system state is captured at various times during execution of the performance test to generate a set of snapshots that can be replayed on an emulator to determine the performance of an integrated circuit design of an integrated circuit device being emulated on the emulator. The integrated circuit design can be, for example, a pre-silicon microprocessor design, a pre-silicon memory controller design, a pre-silicon network adapter device design, or a pre-silicon design of a combination of one or more hardware components in a host system.”. Examiner notes: executing test and capturing the snapshot and/or timeline, then using the snapshot (i.e., parameters) to verify performance).
PNG
media_image1.png
356
336
media_image1.png
Greyscale
PNG
media_image2.png
414
358
media_image2.png
Greyscale
Zidenberg performs test in a host/simulation device in relationship with hardware/silicon. Zidenberg is silent to disclose, however in an analogous art, Engebretsen teaches firmware code, the firmware code being suitable for interacting with a hardware device (See figures 1-2 and 4 (and related text) and abstract, “When testing or validating a hardware system, a script file representing a portion of the firmware may be used to test the system instead of using the firmware code itself. For example, the script file may be plurality of register commands that perform the same initialization sequence as the firmware. Before connecting the hardware system to firmware drivers, the script file may be used to debug the initialization sequence. Instead of generating this script file manually, a firmware testing tool may be used. While executing the firmware, the tool may record the different register access commands performed during the initialization process.”. See paragraph [0004], “The system and computer program product include executing firmware in a firmware testing tool. The firmware testing tool includes a simulated model of the hardware system, where, while executing the firmware, the firmware testing tool writes data to registers in the simulated model that force the simulated model into a desired state. The method, system, computer program product include recording values written to the registers while executing the firmware. The system and computer program product include converting the recorded values into a plurality of register commands to generate a script, where the script is configured to force a physical representation of the hardware system into the desired state.”. Furthermore, see paragraphs [0029], [0044]-[0045]). Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Zidenberg’s teaching, which set forth techniques for obtaining the performance of an integrated circuit design, by using firmware code being suitable for interacting with a hardware device as suggested by Engebretsen, as Engebretsen would provide a mechanism for testing or validating a hardware system using a firmware tool (see abstract and paragraph [0004]). With respect to claim 2 (original), Zidenberg is silent to disclose, however in an analogous art, Engebretsen teaches wherein the firmware code interacts with the hardware device via one or more registers of the hardware device (See abstract, figure 2 (and related text) and paragraphs [0004], [0012], [0027], [0031], “When testing or validating a hardware system, a script file representing a portion of the firmware may be used to test the system instead of using the firmware code itself. For example, the script file may be plurality of register commands that perform the same initialization sequence as the firmware. Before connecting the hardware system to firmware drivers, the script file may be used to debug the initialization sequence. Instead of generating this script file manually, a firmware testing tool may be used. While executing the firmware, the tool may record the different register access commands performed during the initialization process. The script file is then generated programmatically using these recorded commands without requiring input from the system designer. The generated script file may then be tested on the hardware system to determine whether the command sequence in the script file forces the hardware system into the desired state.”). Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Zidenberg’s teaching, which set forth techniques for obtaining the performance of an integrated circuit design, wherein the firmware code interacts with the hardware device via one or more registers of the hardware device as suggested by Engebretsen, as Engebretsen would provide an enhanced mechanism for testing or validating a hardware system using a firmware tool (see abstract and paragraph [0004]).
With respect to claim 3 (original), Zidenberg teaches wherein the timeline of changes of the transaction data represent changes of a state or content of the one or more registers of the hardware device over time, wherein the one or more unit tests are performed using the changes of the state or content of the one or more registers of the hardware device over time (See column 3 lines 41-58, “To reduce the runtime of benchmarking a pre-silicon integrated circuit design on an emulator, the benchmark test is first executed on computer system 100 implementing a version of the integrated circuit device in silicon. In most cases, computer system 100 may contain an older version or generation of the chip being designed. During execution of the benchmark test on the real silicon, snapshots of the system state are taken sequentially at different times during execution of the test. The snapshots can be taken, for example, at a periodic interval, and may include a dump of the system memory (e.g., memory image of the system's DRAM) and the state of the virtual machine (e.g., the values of processor and/or peripheral registers, counters, interrupts, etc.). Each snapshot can be stored with a snapshot identifier to indicate the relative order of the snapshots and which benchmark test the snapshot was taken from. These snapshots of the system state can be used as checkpoints to replay a portion of the benchmark test.”. See column 4 lines 19-34, “Emulated hardware 250 may execute emulation software such as QEMU that runs on emulator 202 to mimic one or more integrated circuit devices. For example, emulated hardware 250 may implement one or more of emulated CPU 252, emulated memory device 254, and/or emulated network device 256. Emulated hardware 250 can execute virtual machine monitor functions, and may interact with a kernel and the virtual machine running on emulator 202 to execute a guest OS and/or application(s) 260. Emulated hardware 250 can maintain processor and/or peripheral registers that are found in the actual hardware components. In some implementations, emulated hardware 250 can be used to emulate an integrated circuit design that has yet to be implemented in real silicon. Thus, emulated hardware 250 may provide a mechanism to test and modify new hardware designs.”. See column 5 lines 25-43, “At block 304, snapshots of the host system state are captured at different times during execution of the performance test on the host system. For example, the snapshots can be taken at periodic time intervals during the performance test. In some implementations, if the performance test has large variations in computational demands throughout the test, the snapshots can be taken more frequently (e.g., once every 200 milliseconds, 500 milliseconds, or 1 second, etc.) to ensure the variations in computational demands are captured in the snapshots, and that the computational demands between each snapshot stays fairly uniform. If the performance test has fairly uniform computational demands, then the snapshots can be taken more infrequently (e.g., once every second, 2 seconds, or 5 seconds, etc.). Each snapshot may include a state of the virtual machine (e.g., including processor register and counter values, peripheral device counters and register values, etc.), and a memory image of the host or system memory (e.g., a DRAM dump) at the time when the snapshot is taken.”. See column 7 lines 46-53, “At block 412, the virtual machine state of the snapshot is loaded onto the virtual machine running on the emulator. This may involve copying processor register values from the snapshot onto the registers of the virtual machine running on the emulator. In some implementations, this may also involve copying peripheral device registers and/or other information from the snapshot, which may be needed by the emulator to restart execution from a restore point.”. Furthermore, see column 8 lines 42-53 and figure 5 (and related text)). With respect to claim 4 (currently amended), Zidenberg teaches wherein the timeline of changes of the transaction data encountered at the one or more interfaces of the hardware device is an abstract representation of a behavior of the hardware device to be encountered during the one or more unit tests (See figure 5 (and related text), emulator image file. See column 5 lines 44-64, “For each of the captured snapshots, the memory image can be stored to a binary file at block 306, and the virtual machine state can be stored to an emulator file format such as QEMU copy on write (qcow) format at block 308. Each of these files may include a snapshot identifier (e.g., a sequence number, time stamp, etc.) that can be used to indicate the time and/or order in which the snapshots were taken, as well as a test identifier indicating the corresponding performance test. It should be noted that in some implementations, the memory image of the host or system memory is not stored together with the virtual machine state in the emulator file format, but is instead stored separately as a binary file. Loading a memory image from an emulator file onto an emulator can be time consuming. Storing the memory image separately allows the memory image to be loaded onto an emulator using a separate channel to speed up the process. In some implementations, each virtual machine state can be stored in the emulator image file format as an overlay that represents state changes from a base image or state changes from a preceding overlay of a preceding virtual machine state.”. See column 5 line 65 – column 6 line 19, “After the set of snapshots corresponding to a performance test has been captured and stored, the snapshots can be replayed on the host system with the real hardware to verify that the performance score of running the full performance test on the real hardware corresponds to the performance score extrapolated from running the snapshots on the real hardware. FIG. 3B illustrate the flow diagram of a process 350 for evaluating the snapshots on the real silicon. It should be noted that in some implementations, process 350 can be optional. At block 310, a snapshot including the virtual machine state and the memory image are loaded onto the host system that was used to capture the snapshots. At block 312, the virtual machine of the host system is executed for a reduced or shortened runtime duration to run a snippet of the performance test. For example, if the snapshots were taken at a 1 second interval, each snapshot can be run for 200 msec. of system time. At block 314, the performance of the host system is measured over the reduced or shortened runtime duration, and recorded as a checkpoint performance for the corresponding snapshot. The operations of blocks 310-314 are repeated for each snapshot of the performance test.”). With respect to claim 5 (currently amended), Zidenberg is silent to disclose, however in an analogous art, Engebretsen teaches wherein the one or more simulation parameters comprise at least one simulation parameter related to a setting of the hardware device (See abstract, “Before connecting the hardware system to firmware drivers, the script file may be used to debug the initialization sequence.”. See paragraph [0012], “When testing the hardware system using the FPGA, a testing apparatus may write values into registers (i.e., blocks of memory) in the FPGA to force the hardware system into a desired state. In one embodiment, the firmware associated with the hardware system is tasked with providing the data values that initialize the system. Because the FPGA may not be connected to firmware drivers when being tested, the system programmers generate a script file (also known as a register file or SCOM file) that has a plurality of register read and write commands that initialize the hardware system in the same manner as the firmware. The commands in the script file are then executed on the FPGA to force the FPGA into the desired state. Thus, the firmware initialization sequence is tested using the script file in lieu of using the firmware itself.”. Furthermore, see paragraphs [0026]-[0028], [0036]). Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Zidenberg’s teaching, which set forth techniques for obtaining the performance of an integrated circuit design, by utilizing simulation parameter related to a setting of the hardware device as suggested by Engebretsen, as Engebretsen would provide an enhanced mechanism for testing or validating a hardware system using a firmware tool (see abstract and paragraph [0004]). With respect to claim 6 (currently amended), Zidenberg is silent to disclose, however in an analogous art, Engebretsen teaches wherein the one or more simulation parameters comprise at least one simulation parameter related to a software setting controlling the behavior of the hardware device (See abstract, “Before connecting the hardware system to firmware drivers, the script file may be used to debug the initialization sequence.”. See paragraph [0012], “When testing the hardware system using the FPGA, a testing apparatus may write values into registers (i.e., blocks of memory) in the FPGA to force the hardware system into a desired state. In one embodiment, the firmware associated with the hardware system is tasked with providing the data values that initialize the system. Because the FPGA may not be connected to firmware drivers when being tested, the system programmers generate a script file (also known as a register file or SCOM file) that has a plurality of register read and write commands that initialize the hardware system in the same manner as the firmware. The commands in the script file are then executed on the FPGA to force the FPGA into the desired state. Thus, the firmware initialization sequence is tested using the script file in lieu of using the firmware itself.”. Furthermore, see paragraphs [0026]-[0028], [0036]). Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Zidenberg’s teaching, which set forth techniques for obtaining the performance of an integrated circuit design, by utilizing simulation parameter related to a software setting controlling the behavior of the hardware device as suggested by Engebretsen, as Engebretsen would provide an enhanced mechanism for testing or validating a hardware system using a firmware tool (see abstract and paragraph [0004]). With respect to claim 7 (currently amended), Zidenberg teaches wherein the hardware device is a memory controller for interfacing with one or more memory modules via one or more memory module slots, wherein the one or more simulation parameters comprise at least one simulation parameter related to a population of the one or more memory module slots (See column 3 lines 9-13, “Hardware resources 110 are implemented using actual integrated circuit device components such as microprocessors, memory controllers, semiconductor memories such as DRAMs, network adapter devices such as network interface cards, etc.”. See column 3 lines 41-58, “To reduce the runtime of benchmarking a pre-silicon integrated circuit design on an emulator, the benchmark test is first executed on computer system 100 implementing a version of the integrated circuit device in silicon. In most cases, computer system 100 may contain an older version or generation of the chip being designed. During execution of the benchmark test on the real silicon, snapshots of the system state are taken sequentially at different times during execution of the test. The snapshots can be taken, for example, at a periodic interval, and may include a dump of the system memory (e.g., memory image of the system's DRAM) and the state of the virtual machine (e.g., the values of processor and/or peripheral registers, counters, interrupts, etc.). Each snapshot can be stored with a snapshot identifier to indicate the relative order of the snapshots and which benchmark test the snapshot was taken from. These snapshots of the system state can be used as checkpoints to replay a portion of the benchmark test.”. See column 5 lines 5-10, “The integrated circuit design can be, for example, a pre-silicon microprocessor design, a pre-silicon memory controller design, a pre-silicon network adapter device design, or a pre-silicon design of a combination of one or more hardware components in a host system.”. Furthermore, column 5 lines 44-64, figure 4 (and related text), column 7 lines 8-17, column 7 lines 27-45, “In some implementations, in order to reduce the amount of time needed to load the memory image, a hypervisor of the emulator may determine the memory addresses that the emulator has allocated to the virtual machine running on the emulator, and the memory loading operation can be performed for only those of the memory addresses that have been allocated to the virtual machine running on the emulator. In other words, the hypervisor may exclude memory addresses of the memory that are not utilized by the virtual machine running on the emulator when loading the memory image onto the memory. In some implementations, loading the memory image from the snapshot onto the memory of the emulator may require the emulator to emulate memory access cycles, which can be time consuming. To further speed up the memory image loading process, a separate channel that bypasses the virtual machine running on the emulator can be used to copy the contents of the memory image file onto the memory used by the virtual machine running on the emulator.”). With respect to claim 8 (currently amended), Zidenberg is silent to disclose, however in an analogous art, Engebretsen teaches wherein the hardware device is simulated based on code specifying the functionality of the hardware device, the code being based on a hardware description language (See paragraph [0011], “One step in testing and validating a hardware system may include using one or more field-programmable gate arrays (FPGAs) to simulate the circuit design of the hardware system (e.g., an IC, memory, analog circuit components, and the like). An FPGA is a special-purpose hardware system that is designed to be configured by a customer or designer after the FPGA is manufactured. Specifically, a customer or designer uses a hardware description language (HDL) to configure the FPGA to act like the desired hardware design. Accordingly, when a bug is found in the design, the customer can update the HDL and change the configuration of the FPGA. Once the customer/designer is satisfied with the performance of the FPGA, the corresponding circuit design may then be used to fabricate the hardware system that was modeled by the FPGA, for example, an application-specific integrated circuit (ASIC).”. See paragraph [0026], “At block 115, the hardware design that was tested at block 105 may be used to configure one or more FPGAs using, for example, an HDL (e.g., Verilog, VHDL, and the like). Similar tests may also be executed on the FPGAs that were run using the hardware simulator--e.g., timing diagrams--to identify any errors or possible optimizations to the design of the hardware system. In addition, the firmware tested at block 110 may be tested against the circuit design implemented on the FPGA. For example, the firmware drivers (e.g., computing systems) may be connected to the FPGA to control the hardware elements modeled by the FPGA. Specifically, the firmware drivers serve as an interface between the FPGA and the firmware. The firmware drivers may execute the firmware internally and transmit the resulting control signals to the FPGAs.”). Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Zidenberg’s teaching, which set forth techniques for obtaining the performance of an integrated circuit design, wherein a hardware device is simulated based on code specifying the functionality of the hardware device, the code being based on a hardware description language as suggested by Engebretsen, as Engebretsen would provide an enhanced mechanism for testing or validating a hardware system using a firmware tool (see abstract and paragraph [0004]). With respect to claim 10 (currently amended), Zidenberg teaches wherein the hardware device is simulated using a field-programmable gate array or using an existing hardware implementation of the hardware device (See column 4 lines 4-18, “Emulator 202 may include real hardware 210 such as programmable logic 218, one or more processor(s) 212, memory device(s) 214, and/or network device(s) 216. The components of hardware 210 are actual integrated circuit devices (e.g., real silicon) that can be used to emulate or mimic the functionalities of one or more integrated circuit designs. For example, a hardware design of an integrated circuit device can be loaded onto programmable logic 218 (e.g., field programmable gate array (FPGA), programmable logic (PLD), etc.) to configure programmable logic 218 to behave like the integrated circuit design under test. In some implementations, one or more components of hardware 210 can be omitted, and the functionality of the omitted component can also be emulated by emulator 210.”). With respect to claim 11 (currently amended), Zidenberg teaches wherein the hardware device is a memory controller for interfacing with one or more memory modules via one or more memory module slots (See column 3 lines 9-13, “Hardware resources 110 are implemented using actual integrated circuit device components such as microprocessors, memory controllers, semiconductor memories such as DRAMs, network adapter devices such as network interface cards, etc.”. See column 3 lines 41-58, “To reduce the runtime of benchmarking a pre-silicon integrated circuit design on an emulator, the benchmark test is first executed on computer system 100 implementing a version of the integrated circuit device in silicon. In most cases, computer system 100 may contain an older version or generation of the chip being designed. During execution of the benchmark test on the real silicon, snapshots of the system state are taken sequentially at different times during execution of the test. The snapshots can be taken, for example, at a periodic interval, and may include a dump of the system memory (e.g., memory image of the system's DRAM) and the state of the virtual machine (e.g., the values of processor and/or peripheral registers, counters, interrupts, etc.). Each snapshot can be stored with a snapshot identifier to indicate the relative order of the snapshots and which benchmark test the snapshot was taken from. These snapshots of the system state can be used as checkpoints to replay a portion of the benchmark test.”. See column 5 lines 5-10, “The integrated circuit design can be, for example, a pre-silicon microprocessor design, a pre-silicon memory controller design, a pre-silicon network adapter device design, or a pre-silicon design of a combination of one or more hardware components in a host system.”. Furthermore, column 5 lines 44-64, figure 4 (and related text), column 7 lines 8-17, column 7 lines 27-45, “In some implementations, in order to reduce the amount of time needed to load the memory image, a hypervisor of the emulator may determine the memory addresses that the emulator has allocated to the virtual machine running on the emulator, and the memory loading operation can be performed for only those of the memory addresses that have been allocated to the virtual machine running on the emulator. In other words, the hypervisor may exclude memory addresses of the memory that are not utilized by the virtual machine running on the emulator when loading the memory image onto the memory. In some implementations, loading the memory image from the snapshot onto the memory of the emulator may require the emulator to emulate memory access cycles, which can be time consuming. To further speed up the memory image loading process, a separate channel that bypasses the virtual machine running on the emulator can be used to copy the contents of the memory image file onto the memory used by the virtual machine running on the emulator.”). With respect to claim 12 (original), Zidenberg is silent to disclose, however in an analogous art, Engebretsen teaches wherein the one or more unit tests relate to an electronic signal training flow for initializing the one or more memory modules (See paragraph [0030], “FIG. 2 is a method 200 of testing a hardware system using an output of a firmware simulator, according to one embodiment described herein. Specifically, method 200 may make the hardware validation process shown in FIG. 1 cheaper or faster by using a firmware simulator to populate the script file for testing the firmware initialization sequence. At block 205, the firmware program code is executed on a simulated model of the hardware system. This may be the same process that was described in block 110 of FIG. 1 where the firmware code is executed using a firmware simulator. Specifically, the firmware simulator (e.g., Simics) may aid in testing and configuring the firmware for a particular type of embedded hardware system. To do this, the firmware simulator establishes a simulated hardware environment that is a model of the set-up of the hardware system.”). Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Zidenberg’s teaching, which set forth techniques for obtaining the performance of an integrated circuit design, wherein the one or more unit tests relate to an electronic signal training flow for initializing the one or more memory modules as suggested by Engebretsen, as Engebretsen would provide an enhanced mechanism for testing or validating a hardware system using a firmware tool (see abstract and paragraph [0004]).
With respect to claim 13 (original), Zidenberg teaches wherein the timeline of changes of the transaction data relates to a response of the one or more memory modules to an electronic training signal provided by the memory controller (see column 5 lines 38-43, “Each snapshot may include a state of the virtual machine (e.g., including processor register and counter values, peripheral device counters and register values, etc.), and a memory image of the host or system memory (e.g., a DRAM dump) at the time when the snapshot is taken.”. See column 5 lines 44-64, “For each of the captured snapshots, the memory image can be stored to a binary file at block 306, and the virtual machine state can be stored to an emulator file format such as QEMU copy on write (qcow) format at block 308. Each of these files may include a snapshot identifier (e.g., a sequence number, time stamp, etc.) that can be used to indicate the time and/or order in which the snapshots were taken, as well as a test identifier indicating the corresponding performance test. It should be noted that in some implementations, the memory image of the host or system memory is not stored together with the virtual machine state in the emulator file format, but is instead stored separately as a binary file. Loading a memory image from an emulator file onto an emulator can be time consuming. Storing the memory image separately allows the memory image to be loaded onto an emulator using a separate channel to speed up the process. In some implementations, each virtual machine state can be stored in the emulator image file format as an overlay that represents state changes from a base image or state changes from a preceding overlay of a preceding virtual machine state.”. Furthermore, see column 7 lines 17-45). With respect to claim 14 (original), Zidenberg teaches wherein the timeline of changes of the transaction data comprises a timeline of state or content changes of one or more registers of the simulated hardware device, with values of the one or more registers indicating a result of a training of the one or more memory (see column 5 lines 38-43, “Each snapshot may include a state of the virtual machine (e.g., including processor register and counter values, peripheral device counters and register values, etc.), and a memory image of the host or system memory (e.g., a DRAM dump) at the time when the snapshot is taken.”. See column 5 lines 44-64, “For each of the captured snapshots, the memory image can be stored to a binary file at block 306, and the virtual machine state can be stored to an emulator file format such as QEMU copy on write (qcow) format at block 308. Each of these files may include a snapshot identifier (e.g., a sequence number, time stamp, etc.) that can be used to indicate the time and/or order in which the snapshots were taken, as well as a test identifier indicating the corresponding performance test. It should be noted that in some implementations, the memory image of the host or system memory is not stored together with the virtual machine state in the emulator file format, but is instead stored separately as a binary file. Loading a memory image from an emulator file onto an emulator can be time consuming. Storing the memory image separately allows the memory image to be loaded onto an emulator using a separate channel to speed up the process. In some implementations, each virtual machine state can be stored in the emulator image file format as an overlay that represents state changes from a base image or state changes from a preceding overlay of a preceding virtual machine state.”. Furthermore, see column 7 lines 17-45). With respect to claim 17 (currently amended), Zidenberg teaches wherein the processing circuitry is to obtain the timeline of changes of the transaction data by simulating the hardware device based on the one or more simulation parameters, storing transaction data encountered at one or more interfaces of the hardware device during simulation of the hardware device, and determining the timeline of changes of the transaction data based on the stored transaction data (See figures 2, 3A-3B, 4 (and related text) and column 1 line 59 – column 2 line 15, “To reduce the amount of time needed to run a benchmark test for a pre-silicon integrated circuit design, the techniques disclosed herein leverage the ability of virtual machines to easily restore execution from a saved state. A full benchmark test can be run on a virtual machine of a hardware system that contains the actual integrated circuit device, and snapshots of the system state are captured at different times during execution of the benchmark test. When a pre-silicon design of the integrated circuit device becomes available, the performance of the design can be assessed by emulating the design in an emulator, and running snippets of the benchmark test on a virtual machine of the emulator. Each snapshot of the benchmark test taken with the real hardware can be loaded onto the emulator, and performance checkpoints are collected by running each snapshot for a short duration of runtime. The performance of the pre-silicon design can then be extrapolated from the performance checkpoints. Thus, instead of running the full benchmark test on the emulator, only snippets (e.g., portions) of the benchmark test are executed, and significant time savings can be achieved when evaluating the performance of a pre-silicon integrated circuit design.”. See column 3 lines 40-58, “To reduce the runtime of benchmarking a pre-silicon integrated circuit design on an emulator, the benchmark test is first executed on computer system 100 implementing a version of the integrated circuit device in silicon. In most cases, computer system 100 may contain an older version or generation of the chip being designed. During execution of the benchmark test on the real silicon, snapshots of the system state are taken sequentially at different times during execution of the test. The snapshots can be taken, for example, at a periodic interval, and may include a dump of the system memory (e.g., memory image of the system's DRAM) and the state of the virtual machine (e.g., the values of processor and/or peripheral registers, counters, interrupts, etc.). Each snapshot can be stored with a snapshot identifier to indicate the relative order of the snapshots and which benchmark test the snapshot was taken from. These snapshots of the system state can be used as checkpoints to replay a portion of the benchmark test.”. See column 4 lines 48-61, “Running a full benchmark test on emulator 202 to evaluate the performance of a hardware design may take a significantly long time and can be impractical in many situations. Instead of running the full duration of a benchmark test, snapshots of the system state taken at different times during execution of the benchmark test on real silicon can be loaded onto emulator 202, and a snippet of the benchmark test can be run on emulator 202. The performance of the hardware design can be measured for each of the short runs, and be extrapolated to derive a performance score representing the performance of running the full benchmark test on the hardware design. Additional details of the benchmarking techniques are described further with reference to FIGS. 3-4.”). With respect to claim 18 (original), Zidenberg teaches wherein the act of storing the transaction data comprises storing a state or content of one or more registers of the simulated hardware device encountered during simulation, wherein the act of determining the timeline of changes of the transaction data comprises determining a timeline of state or content changes of the one or more registers of the simulated hardware device, and wherein the one or more unit tests are performed using the timeline of state or content changes of the one or more registers of the simulated hardware device (see the rejection of at least claims 1 and 14, which addresses similar claim scope). With respect to claim 19 (currently amended), Zidenberg teaches an apparatus for preparing data suitable for performing unit tests [[on firmware code]], [[the firmware code being suitable for interacting with a hardware device]], the apparatus comprising interface circuitry and processing circuitry (see figure 6) to: simulate the hardware device based on one or more simulation parameters, the one or more simulation parameters being defined by one or more unit tests [[to be performed on the firmware code]]; determine a timeline of changes of the transaction data encountered at the one or more interfaces of the hardware device (See figures 2, 3A-3B, 4 (and related text) and column 1 line 59 – column 2 line 15, “To reduce the amount of time needed to run a benchmark test for a pre-silicon integrated circuit design, the techniques disclosed herein leverage the ability of virtual machines to easily restore execution from a saved state. A full benchmark test can be run on a virtual machine of a hardware system that contains the actual integrated circuit device, and snapshots of the system state are captured at different times during execution of the benchmark test. When a pre-silicon design of the integrated circuit device becomes available, the performance of the design can be assessed by emulating the design in an emulator, and running snippets of the benchmark test on a virtual machine of the emulator. Each snapshot of the benchmark test taken with the real hardware can be loaded onto the emulator, and performance checkpoints are collected by running each snapshot for a short duration of runtime. The performance of the pre-silicon design can then be extrapolated from the performance checkpoints. Thus, instead of running the full benchmark test on the emulator, only snippets (e.g., portions) of the benchmark test are executed, and significant time savings can be achieved when evaluating the performance of a pre-silicon integrated circuit design.”. See column 3 lines 40-58, “To reduce the runtime of benchmarking a pre-silicon integrated circuit design on an emulator, the benchmark test is first executed on computer system 100 implementing a version of the integrated circuit device in silicon. In most cases, computer system 100 may contain an older version or generation of the chip being designed. During execution of the benchmark test on the real silicon, snapshots of the system state are taken sequentially at different times during execution of the test. The snapshots can be taken, for example, at a periodic interval, and may include a dump of the system memory (e.g., memory image of the system's DRAM) and the state of the virtual machine (e.g., the values of processor and/or peripheral registers, counters, interrupts, etc.). Each snapshot can be stored with a snapshot identifier to indicate the relative order of the snapshots and which benchmark test the snapshot was taken from. These snapshots of the system state can be used as checkpoints to replay a portion of the benchmark test.”. See column 4 lines 48-61, “Running a full benchmark test on emulator 202 to evaluate the performance of a hardware design may take a significantly long time and can be impractical in many situations. Instead of running the full duration of a benchmark test, snapshots of the system state taken at different times during execution of the benchmark test on real silicon can be loaded onto emulator 202, and a snippet of the benchmark test can be run on emulator 202. The performance of the hardware design can be measured for each of the short runs, and be extrapolated to derive a performance score representing the performance of running the full benchmark test on the hardware design. Additional details of the benchmarking techniques are described further with reference to FIGS. 3-4.”). store transaction data encountered at one or more interfaces of the hardware device during simulation of the hardware device (see figures 3A-3B, storing and loading snapshot).
Zidenberg performs test in a host/simulation device in relationship with hardware/silicon. Zidenberg is silent to disclose, however in an analogous art, Engebretsen teaches firmware code, the firmware code being suitable for interacting with a hardware device (See figures 1-2 and 4 (and related text) and abstract, “When testing or validating a hardware system, a script file representing a portion of the firmware may be used to test the system instead of using the firmware code itself. For example, the script file may be plurality of register commands that perform the same initialization sequence as the firmware. Before connecting the hardware system to firmware drivers, the script file may be used to debug the initialization sequence. Instead of generating this script file manually, a firmware testing tool may be used. While executing the firmware, the tool may record the different register access commands performed during the initialization process.”. See paragraph [0004], “The system and computer program product include executing firmware in a firmware testing tool. The firmware testing tool includes a simulated model of the hardware system, where, while executing the firmware, the firmware testing tool writes data to registers in the simulated model that force the simulated model into a desired state. The method, system, computer program product include recording values written to the registers while executing the firmware. The system and computer program product include converting the recorded values into a plurality of register commands to generate a script, where the script is configured to force a physical representation of the hardware system into the desired state.”. Furthermore, see paragraphs [0029], [0044]-[0045]). Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Zidenberg’s teaching, which set forth techniques for obtaining the performance of an integrated circuit design, wherein the firmware code being suitable for interacting with a hardware device as suggested by Engebretsen, as Engebretsen would provide an enhanced mechanism for testing or validating a hardware system using a firmware tool (see abstract and paragraph [0004]). With respect to claim 20 (original), Zidenberg teaches wherein the act of storing the transaction data comprises storing a state or content of one or more registers of the simulated hardware device encountered during simulation, wherein the act of determining the timeline of changes of the transaction data comprises determining a timeline of state or content changes of the one or more registers of the simulated hardware device, and wherein the one or more unit tests are performed using the timeline of state or content changes of the one or more registers of the simulated hardware device (see the rejection of at least claims 1 and 14, which addresses similar claim scope). With respect to claim 23, the claim is directed to a method that corresponds to the apparatus recited in claim 1, respectively (see the rejection of claim 1 above).
With respect to claim 25, Zidenberg teaches a machine-readable storage medium including program code, when executed, to cause a machine to perform the method of claim 23 (see figure 6 and column 10 lines 12-14).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Zidenberg et al. (US Pat. No. 11,307,882) in view of Engebretsen et al. (US Pub. No. 2014/0173348) and further in view of Kuyel (US Pat. No. 10,970,442).
With respect to claim 9 (currently amended), Zidenberg in view Engebretsen is silent to disclose, however in an analogous art, Kuyel teaches wherein the hardware device is simulated based on a register transfer layer representation or a netlist representation of the hardware device (See column 9 lines 40-48, “In a simulator implemented based on some embodiments of the disclosed technology, transactions are fed to a reference model and the components (coverage collector, transaction loggers, etc.) constituting the reference model simulation environment. When the HDL simulator, the UVM environment, and a register-transfer level (RTL) model are used in the disclosed system, the complete internal state of the hardware component under debug is reproduced in the simulation environment.”). Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify the combination of Zidenberg and Engebretsen, wherein the hardware device is simulated based on a register transfer layer representation or a netlist representation of the hardware device as suggested by Kuyel, as Kuyel would provide an enhanced mechanism hardware and firmware debugging (see abstract and column 1 lines 25-52).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Froelich et al. (US Pub. No. 2022/0163587) discloses a system for data creation, storage, analysis, and training while margin testing includes a margin test generator coupled through an interface to a Device Under Test (DUT). The margin test generator is structured to modify test signals for testing the DUT during one or more testing states of a test session to create testing results. The testing results are stored in a data repository along with a DUT identifier of the DUT tested during the test session. A comparator determine whether any results of the DUT test results match a predictive outcome that is based from an analysis of previous DUT tests. If so, a message generator produces an indication that the tested DUT matched the predictive outcome. (see abstract).
Ramiro Rebollo et al. (US Pub. No. 2020/0327042) discloses a method for automatic generation of integration tests from unit tests includes automatically running, by a processor, a unit test for each model of a plurality of models making-up a system. The method also includes automatically determining, by the processor, integration of unit test cases of interfacing models for one or more groups of interfacing models of the plurality of models. The method additionally includes automatically running, by the processor, an integration test for each group of interfacing models using the unit tests for each model of the interfacing models. The method further includes automatically detecting, by the processor, at least one of inconsistent, incomplete or incorrect data transmitted between the interfacing models for each group of interfacing models. (see abstract).
Ovadia (US Pat. No. 9,858,371) discloses a method for generating a minimized combined scenario for use in simulation, from a post-silicon validation test that includes a combined scenario, may include obtaining a failed scenario loop of a scenario of the combined scenario that includes combined action scenarios that were executed in loops during a post-silicon validation test of a system on chip; and adding any loops of other scenarios of the combined scenario that were executed at least partially concurrently with the failed scenario loop, while discarding any loops of other scenarios of the combined scenario that were completed during the post-silicon validation test before the failed scenario loop or did not commence before the failed scenario loop was completed. (see abstract).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANIBAL RIVERACRUZ whose telephone number is (571)270-1200. The examiner can normally be reached Monday-Friday 9:30 AM-6:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hyung S Sough can be reached at 5712726799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANIBAL RIVERACRUZ/Primary Examiner, Art Unit 2192