Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1-20 are currently pending.
Response to Amendment
Applicant’s amendment filed on October 9, 2025 has been entered.
Response to Arguments
Applicant’s arguments with respect to claim) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Ramadoss et al. (US Patent No. 10,043,232 B1).
As to Claims 1 and 8. (Currently Amended) A method comprising:
in response to receiving a plurality of instructions to perform one or more data computation operations; See: (136) In one embodiment the compute cluster 614 includes a stall notification module 606 that maintains an activity scoreboard associated with the compute units 604 within the compute cluster 614. The activity scoreboard maintains an active or blocked status for each of the compute units 604. When execution on a compute unit becomes blocked due to a blocking event, the stall notification module 606 updates the activity scoreboard for the compute units 604. In one embodiment, once all compute units 604 within the compute cluster 614 become blocked, the stall notification module 606 can signal a rebalance module that a workload rebalance can be performed on the compute units 604. In one embodiment the stall notification module 606 can signal the rebalance module 602 that a workload rebalance can be performed on the compute units 604 when one or more compute units are stalled or when the number of stalled compute units exceeds a threshold.
sending, from a compute unit associated with a first cache (L1), a parameter associated with the one or more data computation operations to a co-compute unit within a second cache (L3); See: (135) FIG. 6 illustrates a preemptable GPGPU compute system 600, according to an embodiment. The GPGPU compute system 600 includes a compute cluster 614 having a set of compute units 604. The compute units 604 can process workloads dispatched to the compute units. During operation, instructions and data can be fetched and loaded from cache memory, such as an L1 cache 608. In one embodiment each of the compute units 604 are structured in a similar manner as the graphics multiprocessor 400 of FIG. 4. However, the compute units 604 can be any of the instruction level execution units described herein. The compute unit 604 can include a fetch unit to fetch instruction to execute and a decode unit to decode the fetched instructions. In one embodiment the compute units 604 includes execution resources having one or more single precision floating point units, double precision floating point units, integer arithmetic logic units, load/store unit, and/or special function units.
(136) In one embodiment the compute cluster 614 includes a stall notification module 606 that maintains an activity scoreboard associated with the compute units 604 within the compute cluster 614. The activity scoreboard maintains an active or blocked status for each of the compute units 604. When execution on a compute unit becomes blocked due to a blocking event, the stall notification module 606 updates the activity scoreboard for the compute units 604. In one embodiment, once all compute units 604 within the compute cluster 614 become blocked, the stall notification module 606 can signal a rebalance module that a workload rebalance can be performed on the compute units 604. In one embodiment the stall notification module 606 can signal the rebalance module 602 that a workload rebalance can be performed on the compute units 604 when one or more compute units are stalled or when the number of stalled compute units exceeds a threshold.
(137) In one embodiment, upon receipt of the notice from the stall notification module 606, the rebalance module 602 can query scheduling logic to determine if any workloads are pending execution. If any new or existing tasks or workloads are available to be scheduled to the compute units 604, the rebalance module can signal the compute units 604 to perform a migration of the blocked workloads. To prepare to migrate the blocked workloads, the compute units 604 can evaluate each of the pending pipeline activities associated with the workload. The existing pipeline activity can either be allowed to drain through the pipeline or activity can be dropped and flagged for replay once the workload is resumed on the compute units 604. Once the pipeline has drained, the pipeline state associated with the workload can be saved to temporary memory. In one embodiment the compute cluster 614 includes a scratchpad memory 612 that can be used as temporary memory for the compute cluster. The compute cluster 614 can store the pipeline state and other context information associated with the workload within the scratchpad memory 612. Alternatively, the compute cluster 614 can store the pipeline state in a cache memory, such as a level-three (L3) cache that is shared by multiple compute clusters; and
launching, at the co-compute unit (compute cluster is deemed to include co-compute units (subset of compute units) as they include other compute units that work together), a wave (dispatcher/scheduler) to perform a data computational operation of the one or more data computation operations based on the parameter and using the second cache; See: (133) A compute unit within a GPGPU is the core processing element of the GPGPU. Compute units are majority of the hardware assets within a GPGPU and are used by 3D workloads, general purpose compute workloads, and media workloads. In previous implementations, when a workload is dispatched for execution to the compute units the workload either runs to completion or the entire workload is pre-empted to make way for a new workload. Embodiments described herein enable rebalancing of compute assets using a preempt mechanism on a subset of compute units within a compute cluster. The rebalance could be based on stalling events like a page fault or barriers and synchronization semaphores. These events typically stall the compute unit until the event is resolved, which reduces compute asset utilization;
(137) In one embodiment, upon receipt of the notice from the stall notification module 606, the rebalance module 602 can query scheduling logic to determine if any workloads are pending execution. If any new or existing tasks or workloads are available to be scheduled to the compute units 604, the rebalance module can signal the compute units 604 to perform a migration of the blocked workloads. To prepare to migrate the blocked workloads, the compute units 604 can evaluate each of the pending pipeline activities associated with the workload. The existing pipeline activity can either be allowed to drain through the pipeline or activity can be dropped and flagged for replay once the workload is resumed on the compute units 604. Once the pipeline has drained, the pipeline state associated with the workload can be saved to temporary memory. In one embodiment the compute cluster 614 includes a scratchpad memory 612 that can be used as temporary memory for the compute cluster. The compute cluster 614 can store the pipeline state and other context information associated with the workload within the scratchpad memory 612. Alternatively, the compute cluster 614 can store the pipeline state in a cache memory, such as a level-three (L3) cache that is shared by multiple compute clusters.
(138) In one embodiment the compute cluster 614 additionally includes a power module 605. The power module 605 can be used to power gate idle compute units 604. Once a stalled workload on the compute unit 604 is migrated off the compute units and the pipeline state associated with that workload is saved, the power module 605 can power gate the compute units 604 if no additional workloads are pending. The power module 605 can then re-enable the compute units 604 once a stalled workload is unblocked or new workloads are available to be scheduled.
As to Claims 2 and 9. (Original) The method of claim 1, wherein the second cache (L3) comprises a different-level cache than the first cache (L1); See: (134-138).
As to Claims 3 and 10. (Currently Amended) The method of claim 1, further comprising: sending an instruction from the compute unit to the co-compute unit to launch the wave (rebalance/schedule/dispatch) to perform the data computation operation of the one or more data computation operations: See: (133) A compute unit within a GPGPU is the core processing element of the GPGPU. Compute units are majority of the hardware assets within a GPGPU and are used by 3D workloads, general purpose compute workloads, and media workloads. In previous implementations, when a workload is dispatched for execution to the compute units the workload either runs to completion or the entire workload is pre-empted to make way for a new workload. Embodiments described herein enable rebalancing of compute assets using a preempt mechanism on a subset of compute units within a compute cluster. The rebalance could be based on stalling events like a page fault or barriers and synchronization semaphores. These events typically stall the compute unit until the event is resolved, which reduces compute asset utilization.
(134) Embodiments described herein provide techniques for enhanced preemption of compute clusters of a GPGPU. One embodiment provides a hardware mechanism to track blocking events as the events occur. The hardware then generates a notification to a re-balance module. The re-balance module can then switch out the stalled workload and reschedule any pending compute tasks ono the previously stalled compute assets. One embodiment enables a hardware mechanism to migrate and restore state information (e.g., general purpose registers, instruction pointers, etc.) for a stalled task. The state information is stored in temporary storage (e.g., scratchpad memory) when a workload is migrated from a compute unit. The state can be quickly restored from the temporary memory when the workload is ready to be resumed.
As to Claims 4 and 11. (Previously Presented) The method of claim 1, further comprising: in response to receiving the parameter, establishing, by the co-compute unit, a register in the second cache; See: (133-134).
As to Claims 5 and 12. (Currently Amended) The method of claim 4, further comprising: determining, based on the data computation operation of the one or more data computation operations, a determined size of the register, wherein the register is established based on the determined size; See: (157) Upon initialization of a GPGPU device, preemption logic can program a threshold value and the comparator logic will begin monitoring the usage values transmitted for the register file. As the register file size varies, the comparator logic can generate an interrupt to the preemption logic when the register file size is lower than the programmed threshold value. When the preemption logic receives the interrupt, the preemption logic can initiate the preemption sequence to migrate or remove the currently running task.
As to Claims 6 and 13. (Original) The method of claim 4, wherein the register includes a uniform register; See: (188) In some embodiments, the processor 1602 includes cache memory 1604. Depending on the architecture, the processor 1602 can have a single internal cache or multiple levels of internal cache. In some embodiments, the cache memory is shared among various components of the processor 1602. In some embodiments, the processor 1602 also uses an external cache (e.g., a Level-3 (L3) cache or Last Level Cache (LLC)) (not shown), which may be shared among processor cores 1607 using known cache coherency techniques. A register file 1606 is additionally included in processor 1602 which may include different types of registers for storing different types of data (e.g., integer registers, floating point registers, status registers, and an instruction pointer register). Some registers may be general-purpose registers, while other registers may be specific to the design of the processor 1602.
As to Claims 7. (Currently Amended) The method of claim 1, further comprising:
sending, from the co-compute unit to the compute unit, data resulting from a performance
of the data computation operation of the one or more data computation operations; See: (144) FIG. 8 illustrates task migration for a stalled compute cluster, according to an embodiment. In one embodiment a multi-context GPGPU (806A) can execute workloads from a first application 802 and second application 804. In the event a workload for the second application 804 encounters a stall, for example, due to a page fault, a barrier, or other synchronization events, the workload for the second application 804 can be removed from the stalled clusters. Once the stalled workloads are removed, the multi-context GPGGPU (806B) can execute a workload for the first application 802. This will mechanism will increase the compute unit utilization within the compute cluster, improving the efficiency and performance of the GPGPU.
As to Claim 14. (Currently Amended) The processor of claim 8, wherein each co-compute unit is configured to send data resulting from a performance of the data computation operation of the one or more data computation operations to a respective compute unit: See: (144).
As to Claim 15. (Currently Amended) A method comprising: in response to receiving instructions to perform one or more data computation operations,
sending, from a compute unit associated with a first cache, a parameter associated with the one or more data computation operations to a scheduler coupled to a plurality of co-compute units in a second cache; See: (134-138); (173) In one embodiment, the logic to implement fine-granularity reset is implemented within the scheduler micro-controller 1310, which includes firmware logic to continuously monitor the execution state of the compute units within each reset block 1320A-1320B. In one embodiment, each reset block 1320A-1320N includes logic to support the scheduler micro-controller 1310. For example and in one embodiment each reset block 1320A-1320N includes an interrupt module 1325A-1325N to trigger an interrupt to the scheduler micro-controller 1310 in the event that the reset block detects a fault within a compute unit. The reset blocks 1320A-1320N can include fault detection logic that determine whether a compute unit within the reset block has encountered a fault. The interrupt module 1325A-1325N of the reset block 1320A-1320N can then interrupt the scheduler micro-controller 1310. The scheduler micro-controller can handle the interrupt by re-distributing the threads within the faulting reset block and resetting the faulting compute units. Once the compute units 1304A-1304N are reset, the compute units can begin accepting dispatch of new threads.
(174) FIG. 14 is a flow diagram of a fine granularity reset logic 1400, according to an embodiment. The fine granularity reset logic 1400 can reside within a scheduler micro-controller, such as the scheduler micro-controller 1310 of FIG. 13. The fine granularity reset logic 1400 enables hardware reset of one or more compute units within a reset block in response to a hardware fault, such as a soft error or another form of single event upset. Once fault detection logic takes notice of a faulting compute unit, the reset logic 1400 can receive a notice that a compute unit requires a reset, as shown at block 1402. In one embodiment the notice is received as an interrupt from a reset block that defines a reset boundary for the compute unit.
scheduling, by the scheduler, a performance of a data computational operation of the one or more data computation operations by a co-compute unit of the plurality of co- compute units; See: (134-138); (173-174); and
performing, by the co-compute unit, the data computation operation of the one or more data computation operations based on the parameter and using the second cache; See: (134-138); (173-174).
As to Claim 16. (Original) The method of claim 15, wherein the second cache comprises a different- level cache than the first cache; See: (134-138).
As to Claim 17. (Previously Presented) The method of claim 15, further comprising: identifying, by the co-compute unit, the compute unit based on instructions received from the scheduler; See: (134-138); (173-174).
As to Claim 18. (Currently Amended) The method of claim 17, further comprising: storing data resulting from performing performance of the data computation operation of the one or more data computation operations in a data buffer associated with the compute unit; See (144); (42) When the host interface 206 receives a command buffer via the I/O unit 204, the host interface 206 can direct work operations to perform those commands to a front end 208. In one embodiment the front end 208 couples with a scheduler 210, which is configured to distribute commands or other work items to a processing cluster array 212. In one embodiment the scheduler 210 ensures that the processing cluster array 212 is properly configured and in a valid state before tasks are distributed to the processing clusters of the processing cluster array 212;
(47) In one embodiment, when the parallel processing unit 202 is used to perform graphics processing, the scheduler 210 can be configured to divide the processing workload into approximately equal sized tasks, to better enable distribution of the graphics processing operations to multiple clusters 214A-214N of the processing cluster array 212. In some embodiments, portions of the processing cluster array 212 can be configured to perform different types of processing. For example a first portion may be configured to perform vertex shading and topology generation, a second portion may be configured to perform tessellation and geometry shading, and a third portion may be configured to perform pixel shading or other screen space operations, to produce a rendered image for display. Intermediate data produced by one or more of the clusters 214A-214N may be stored in buffers to allow the intermediate data to be transmitted between clusters 214A-214N for further processing.
(48) During operation, the processing cluster array 212 can receive processing tasks to be executed via the scheduler 210, which receives commands defining processing tasks from front end 208. For graphics processing operations, processing tasks can include indices of data to be processed, e.g., surface (patch) data, primitive data, vertex data, and/or pixel data, as well as state parameters and commands defining how the data is to be processed (e.g., what program is to be executed). The scheduler 210 may be configured to fetch the indices corresponding to the tasks or may receive the indices from the front end 208. The front end 208 can be configured to ensure the processing cluster array 212 is configured to a valid state before the workload specified by incoming command buffers (e.g., batch-buffers, push buffers, etc.) is initiated.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHARLES RONES whose telephone number is (571)272-4085. The examiner can normally be reached M-F 9-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cordelia Zecher can be reached at 571-272-7771. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
CHARLES RONES
Supervisory Patent Examiner
Art Unit 2136
/CHARLES RONES/Supervisory Patent Examiner, Art Unit 2168