Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The previous action is withdrawn/superseded.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because it discloses a computer-readable medium that may include signal embodiments. This rejection can be withdrawn by amending to specify a non-transitory computer-readable medium.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Du (Pub No. US 20230267567 A1) in view of Du (Pub No. US 11055808 B2) hereinafter named Du’808.
As per claim 1, Du teaches the claimed:
1. An apparatus for graphics processing, comprising: at least one memory;
and at least one processor coupled to the at least one memory and, based at least in part on information stored in the at least one memory, the at least one processor is configured to: initiate an execution of a set of instructions including a first instruction and a second instruction; (Du [0041]: “A streaming processor may execute a multitude of threads. Such threads may, in some instances, be grouped together according to particular code (e.g., instructions) and the grouped threads can be executed in parallel. For instance, a group of threads (e.g., a warp, a wavefront, a wave, etc., hereinafter referred to as a wave) may include a particular number of threads, with each thread repeatedly executing a same portion of code, such as a shader (e.g., instructions associated with rendering an image in specific, unique ways) or a kernel (e.g., a compute kernel). The streaming processor may execute, in parallel, (1) a multitude of threads and/or (2) a multitude of groups of threads, and the techniques described herein may be applied in any case. As an example, the streaming processor may execute a multitude of groups of threads, with each group of threads corresponding to execution of a respective portion of code. For instance, the streaming processor may interleave between the groups of threads, such that each group may be executed. As another example, the streaming processor may execute a multitude of threads, with each thread corresponding to execution of a respective portion of code. For convenience, a thread may be referred to herein as being a particular thread or a group of threads, both associated with execution of a particular portion of code.”. This includes a memory. Du [0022]: “The term application may refer to software. As described herein, one or more techniques may refer to an application (e.g., software) being configured to perform one or more functions. In such examples, the application may be stored in a memory (e.g., on-chip memory of a processor, system memory, or any other memory). Hardware described herein, such as a processor may be configured to execute the application. For example, the application may be described as including code that, when executed by the hardware, causes the hardware to perform one or more techniques described herein. As an example, the hardware may access the code from a memory and execute the code accessed from the memory to perform one or more techniques described herein. In some examples, components are identified in this disclosure. In such examples, the components may be hardware, software, or a combination thereof. The components may be separate components or sub-components of a single component.”).
Du alone does not explicitly teach the remaining claim limitations.
However, Du in combination with Du ‘808 teaches the claimed:
determine whether data is pending for one of the first instruction or the second instruction; (Du [0025]: “A compiler may select a wave size (e.g., 1× or 2×) based on the shader type and performance statistics, and may communicate with the driver to set the streaming processor (SP) wave size context register. However, because the shader execution is highly dynamic, it may be difficult to always select the optimal wave size (e.g., the wave size associated with a shorter overall execution time). In one or more aspects, the compiler's role in selecting the wave size may be eliminated. A uniform 1× wave granularity in GPR allocation may be used. The wave size may be changed dynamically per instruction during execution time. Accordingly, the instruction fetch may be more efficient as a single instruction may be executed for two or more different waves. Memory and cache utilization may be more efficient as texture processing and load/store operations may be executed on paired or grouped waves in a single region in close proximity.” Instructions are paired and executed in waves. Du ‘808 teaches determining whether the wave data is pending. Du ‘808 col. 18 lines 40-65: “As illustrated in FIGS. 5A and 5B, aspects of the present disclosure can include context queue 570 to store the context information of the wave slots 550-555. For example, context queue 570 can include 16 context queue slots 571-586. However the number of context queue slots can be scalable to include other amounts, e.g., 32 context queue slots. In some aspects, if wave slots 550-555 have pending wave data being processed, they can issue a request to acquire an execution slot with a wave identification (ID), which each context register can include. Further, upon granting the execution slot, a scheduler can copy active or frequently changed context registers from the context queue 570 to the granted execution slot in a few cycles, fetch the wave instruction, and then start executing the instruction. Additionally, if a wave or execution slot is in a waiting state, it can copy the active context register information to the context queue 570 in a few cycles and surrender its execution slot. In some aspects, if wave input data and/or output results are considered to be in a waiting state, they may not occupy wave or execution slots. Moreover, the context queue 570 can include a wide bandwidth for reduced latency issues. For example, latency may occur during wave instruction switching between the execution slots and the context queue.” This determination of pending data can be used for either of the two instructions, as they are contained in the wave slot.).
and suspend the execution of the first instruction and the second instruction based on the data being pending for one of the first instruction or the second instruction.
(Du [0061]: The execution slots 635 and the wave slots 650 may form a hierarchical structure. GPU workload waves may be first loaded into the wave slots 650. Waves in the wave slots 650 may be selected for execution, e.g., based on a priority. The waves selected for execution may be copied from the corresponding wave slots to one or more of the execution slots 635. In particular, the associated context registers including context states or values may be copied from the corresponding wave slots to the one or more of the execution slots 635. Based on the instruction, an appropriate EU may be selected to execute the operations on the waves in the execution slots 635. Once the execution is completed or suspended, the waves may be copied back from the corresponding execution slots 635 to one or more of the wave slots 650. In particular, the associated context registers including context states or values may be copied back from the corresponding execution slots to the one or more of the wave slots 650.” As taught by Du ‘808 col. 18 lines 40-65, execution can be based on whether instructions are pending. Thus, the execution can be completed or suspended based on the pending status.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the indication of data being pending as taught by Du’808 with the system of Du in order to control when to execute the pair of instructions based on the data manipulated by them.
As per claims 19 and 20, these claims are similar in scope to limitations recited in claim 1, and thus are rejected under the same rationale.
As per claim 2, Du alone does not explicitly teach the claimed limitations.
However, Du in combination with Du ‘808 teaches the claimed:
2. The apparatus of claim 1, wherein the at least one processor is further configured to: identify the first instruction and the second instruction as a low priority wave pair based on the data being pending for one of the first instruction or the second instruction. (Du teaches pairing instructions in wave form. Du [0025]: “A compiler may select a wave size (e.g., 1× or 2×) based on the shader type and performance statistics, and may communicate with the driver to set the streaming processor (SP) wave size context register. However, because the shader execution is highly dynamic, it may be difficult to always select the optimal wave size (e.g., the wave size associated with a shorter overall execution time). In one or more aspects, the compiler's role in selecting the wave size may be eliminated. A uniform 1× wave granularity in GPR allocation may be used. The wave size may be changed dynamically per instruction during execution time. Accordingly, the instruction fetch may be more efficient as a single instruction may be executed for two or more different waves. Memory and cache utilization may be more efficient as texture processing and load/store operations may be executed on paired or grouped waves in a single region in close proximity.” Du [0061]: “The execution slots 635 and the wave slots 650 may form a hierarchical structure. GPU workload waves may be first loaded into the wave slots 650. Waves in the wave slots 650 may be selected for execution, e.g., based on a priority. The waves selected for execution may be copied from the corresponding wave slots to one or more of the execution slots 635. In particular, the associated context registers including context states or values may be copied from the corresponding wave slots to the one or more of the execution slots 635. Based on the instruction, an appropriate EU may be selected to execute the operations on the waves in the execution slots 635. Once the execution is completed or suspended, the waves may be copied back from the corresponding execution slots 635 to one or more of the wave slots 650. In particular, the associated context registers including context states or values may be copied back from the corresponding execution slots to the one or more of the wave slots 650.” Du ‘808 teaches a hierarchy of levels for wave slots, which can be combined with Du’s levels of priority. Du ‘808 col. 14 line 59-col. 15 line 3: “As indicated herein, aspects of the present disclosure can convert wave slots into a wave slot hierarchy. Further, aspects of the present disclosure can partition the wave slots into a hierarchy of different levels. For example, a first level of the hierarchy can be the wave slots, each of which can be accessed in parallel or at the same time as other wave slots. The second level of the hierarchy can be the context queue, which can track the wave instructions that are being processed, e.g., by execution units. As the data or wave instructions are processed and cycle back sequentially, the context information stored in the context queue can be copied and sent to the wave or execution slots.” The hierarchy of levels of Du ‘808 can be combined with the priority of instructions as taught by Du. Hierarchy of levels of priority implies low priority. Du ‘808 col. 18 lines 40-65: “As illustrated in FIGS. 5A and 5B, aspects of the present disclosure can include context queue 570 to store the context information of the wave slots 550-555. For example, context queue 570 can include 16 context queue slots 571-586. However the number of context queue slots can be scalable to include other amounts, e.g., 32 context queue slots. In some aspects, if wave slots 550-555 have pending wave data being processed, they can issue a request to acquire an execution slot with a wave identification (ID), which each context register can include. Further, upon granting the execution slot, a scheduler can copy active or frequently changed context registers from the context queue 570 to the granted execution slot in a few cycles, fetch the wave instruction, and then start executing the instruction. Additionally, if a wave or execution slot is in a waiting state, it can copy the active context register information to the context queue 570 in a few cycles and surrender its execution slot. In some aspects, if wave input data and/or output results are considered to be in a waiting state, they may not occupy wave or execution slots. Moreover, the context queue 570 can include a wide bandwidth for reduced latency issues. For example, latency may occur during wave instruction switching between the execution slots and the context queue.” The movement of the pending wave data to the context queue can be combined with a different level of hierarchy as lower-level waves in the hierarchy going to the context queue. Du ‘808 col. 14 line 59-col. 15 line 3: “As indicated herein, aspects of the present disclosure can convert wave slots into a wave slot hierarchy. Further, aspects of the present disclosure can partition the wave slots into a hierarchy of different levels. For example, a first level of the hierarchy can be the wave slots, each of which can be accessed in parallel or at the same time as other wave slots. The second level of the hierarchy can be the context queue, which can track the wave instructions that are being processed, e.g., by execution units. As the data or wave instructions are processed and cycle back sequentially, the context information stored in the context queue can be copied and sent to the wave or execution slots.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the hierarchy of levels for different wave slots containing instructions as taught by Du ‘808 with the system of Du in order to give different instructions pairs different levels of priority.
As per claim 3, Du alone does not explicitly teach the claimed limitations.
However, Du in combination with Du ‘808 teaches the claimed:
3. The apparatus of claim 2, wherein the at least one processor is further configured to: set one or more bits in a context register based on identification of the first instruction and the second instruction as the low priority wave pair. (Du ‘808 col. 7 lines 30-46: “GPUs can process multiple types of data or data packets in a GPU pipeline. For instance, in some aspects, a GPU can process two types of data or data packets, e.g., context register packets and draw call data. A context register packet can be a set of global state information, e.g., information regarding a global register, shading program, or constant data, which can regulate how a graphics context will be processed. For example, context register packets can include information regarding a color format. In some aspects of context register packets, there can be a bit that indicates which workload belongs to a context register. Also, there can be multiple functions or programming running at the same time and/or in parallel. For example, functions or programming can describe a certain operation, e.g., the color mode or color format. Accordingly, a context register can define multiple states of a GPU.” Du ‘808 col. 14 line 59-col. 15 line 3: “As indicated herein, aspects of the present disclosure can convert wave slots into a wave slot hierarchy. Further, aspects of the present disclosure can partition the wave slots into a hierarchy of different levels. For example, a first level of the hierarchy can be the wave slots, each of which can be accessed in parallel or at the same time as other wave slots. The second level of the hierarchy can be the context queue, which can track the wave instructions that are being processed, e.g., by execution units. As the data or wave instructions are processed and cycle back sequentially, the context information stored in the context queue can be copied and sent to the wave or execution slots.” The bit indicates levels. This necessarily includes a low level because there is a hierarchy. The second level is the lower level.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the bits in a context register to indicate priority for an instruction as taught by Du ‘808 with the system of Du in order to control the flow of instructions and contextualize the pair of instructions with all the others in the sequence, and show that organization in a computer-readable way.
As per claim 4, Du alone does not explicitly teach the claimed limitations.
However, Du in combination with Du ‘808 teaches the claimed:
4. The apparatus of claim 3, wherein the at least one processor is further configured to: remove an assignment of the first instruction and the second instruction from an execution unit based on the identification of the first instruction and the second instruction as the low priority wave pair; (Du ‘808 col. 1 line 44-col. 2 line 8: “In an aspect of the disclosure, a method, a computer-readable medium, and an apparatus are provided. The apparatus may be a graphics processing unit (GPU). In some aspects, the apparatus can determine one or more context states of at least one context register in each of multiple wave slots. The apparatus can also send information corresponding to the one or more context states in one of the multiple wave slots to a context queue. Additionally, the apparatus can convert the information corresponding to the one or more context states to context information compatible with the context queue. The apparatus can also store the context information compatible with the context queue in the context queue. In some aspects, the apparatus can remove the one or more context states of at least one context register from the one of the wave slots when the information corresponding to the one or more context states is sent to the context queue. In further aspects, the apparatus can send the context information compatible with the context queue to one of the multiple wave slots. Moreover, the apparatus can convert the context information compatible with the context queue to the information corresponding to the one or more context states. The apparatus can also copy the information corresponding to the one or more context states when the information corresponding to the one or more context states is sent to the context queue. In some aspects, the apparatus can also convert the multiple wave slots to multiple execution slots. Further, the apparatus can send wave data corresponding to the one of the wave slots to one of multiple execution units. The apparatus can also receive wave data corresponding to the one of the multiple wave slots from one of multiple execution units.” The reception of wave slots from the multiple execution units means that the wave slots were removed from the execution unit to the context queue. The examiner is treating “the apparatus” as moving instructions into the context queue, which contains the lower level priority instructions. Du ‘808 col. 14 line 59-col. 15 line 3: “As indicated herein, aspects of the present disclosure can convert wave slots into a wave slot hierarchy. Further, aspects of the present disclosure can partition the wave slots into a hierarchy of different levels. For example, a first level of the hierarchy can be the wave slots, each of which can be accessed in parallel or at the same time as other wave slots. The second level of the hierarchy can be the context queue, which can track the wave instructions that are being processed, e.g., by execution units. As the data or wave instructions are processed and cycle back sequentially, the context information stored in the context queue can be copied and sent to the wave or execution slots.” The second level is the lower level which causes instructions to be sent.).
and add the first instruction and the second instruction to a wave queue based on the removal of the assignment of the first instruction and the second instruction from the execution unit. (The pair of instructions taught by Du can be contained in the wave slot for execution and can be the slot sent to the queue taught above. Sending it to the queue is removing it from the execution unit.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the removing of instructions from execution unit to queue based on hierarchy as taught by Du ‘808 with the system of Du in order to allow a queue for certain instruction pairs that have different priority and allow them to be controlled or removed from execution in different ways.
As per claim 5, Du alone does not explicitly teach the claimed limitations.
However, Du in combination with Du ‘808 teaches the claimed:
5. The apparatus of claim 3, wherein to set the one or more bits in the context register, the at least one processor is configured to: set the one or more bits in the context register at a streaming processor (SP) in a graphics processing unit (GPU) or a scheduler of the SP. (Du ‘808 col. 8 lines 34-46: “FIG. 3 illustrates an example GPU 300 in accordance with one or more techniques of this disclosure. More specifically, FIG. 3 illustrates a streaming processor (SP) system in GPU 300. As shown in FIG. 3, GPU 300 includes high level sequencer (HLSQ) 302, VPC 304, thread processor (TP) 306, UCHE 308, RB 310, and VPC 312. GPU 300 also includes SP 320, master engine 322, sequencer 324, local memory 326, wave scheduler and context register 328, texture unit (TEX) or load controller 330, instruction cache 332, execution units (EUs) 334, general purpose register (GPR) 336, distributor 338, constant RAM 340, and distributor 342. The wave scheduler and context register 328 can also include one or more wave slots.” The streaming processor includes the context register which has the bits.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the location of the context register in a stream processor as taught by Du ‘808 with the system of Du in order to allow the stream of instructions to be run in parallel and for indication of which instructions of different priority.
As per claim 6, Du teaches the claimed:
6. The apparatus of claim 2, wherein the low priority wave pair includes a lower priority compared to a normal priority wave pair in a wave pair hierarchy. (Du [0061]-[0062]: “[0061] The execution slots 635 and the wave slots 650 may form a hierarchical structure. GPU workload waves may be first loaded into the wave slots 650. Waves in the wave slots 650 may be selected for execution, e.g., based on a priority. The waves selected for execution may be copied from the corresponding wave slots to one or more of the execution slots 635. In particular, the associated context registers including context states or values may be copied from the corresponding wave slots to the one or more of the execution slots 635. Based on the instruction, an appropriate EU may be selected to execute the operations on the waves in the execution slots 635. Once the execution is completed or suspended, the waves may be copied back from the corresponding execution slots 635 to one or more of the wave slots 650. In particular, the associated context registers including context states or values may be copied back from the corresponding execution slots to the one or more of the wave slots 650.
[0062] As indicated above, FIG. 6 shows an SP scheduling flow. Inside the wave scheduler, the waiting wave queue may be stored in the wave slots 650, which may make up the bottom layer of the wave slot hierarchy. Input and memory data accumulation may be performed at the wave slots 650. In one or more examples, in configurations where both the 1× and the 2× wave sizes are utilized, each wave slot may physically accommodate 128 fibers, and at runtime may hold 64 fibers (i.e., a 1× wave) or 128 fibers (i.e., a 2× wave) based on the wave size setting provided by the compiler. In one configuration, each cluster may include 8 wave slots. Execution slots may be located above the wave slots in the hierarchy. In one configuration, each cluster may include 4 execution slots, each of which may also physically accommodate 128 fibers. The execution slots may be configured to fetch instructions from the level 1 (L1) instruction cache and issue instructions to EUs. Within a same cluster, a switch may be located in between and connected to the wave slots and the execution slots.” This wave slot hierarchy necessarily includes lower and higher levels. The wave slots can be combined with the wave pair taught above.).
As per claim 7, Du alone does not explicitly teach the claimed limitations.
However, Du in combination with Du ‘808 teaches the claimed:
7. The apparatus of claim 6, wherein to identify the first instruction and the second instruction as the low priority wave pair, the at least one processor is configured to:
reduce a priority of the first instruction and the second instruction in the wave pair hierarchy,
and wherein the low priority wave pair for the first instruction and the second instruction corresponds to a partially ready wave pair. (Du ‘808 col. 15 lines 19-34: “ As indicated herein, the aforementioned wave hierarchy can send context information from the wave slots to a context queue, store context information in the context queue, and then send the context information back to the wave slots when it is ready to be executed. Accordingly, in some aspects, this can be a first-in-first-out (FIFO) hierarchy of wave data. By forming this wave hierarchy, aspects of the present disclosure can address the aforementioned congestion problems. For instance, aspects of the present disclosure can increase the capacity of each wave slot, so that each wave slot can execute more wave instructions and better account for any latency issues. Further, aspects of the present disclosure can allow for the reduction of the wave instruction ratio between the wave slots and the execution units, while still maintaining the same amount of functionality or execution ability.” The instructions in the context queue are partially ready and the queue is lower on the hierarchy, as taught above in the rejection to claim 6. Then, when it is ready it is moved to wave slots for execution and becomes normal priority. The hierarchy can be combined with priority for the instructions as taught by Du. Du [0061]: “The execution slots 635 and the wave slots 650 may form a hierarchical structure. GPU workload waves may be first loaded into the wave slots 650. Waves in the wave slots 650 may be selected for execution, e.g., based on a priority. The waves selected for execution may be copied from the corresponding wave slots to one or more of the execution slots 635. In particular, the associated context registers including context states or values may be copied from the corresponding wave slots to the one or more of the execution slots 635. Based on the instruction, an appropriate EU may be selected to execute the operations on the waves in the execution slots 635. Once the execution is completed or suspended, the waves may be copied back from the corresponding execution slots 635 to one or more of the wave slots 650. In particular, the associated context registers including context states or values may be copied back from the corresponding execution slots to the one or more of the wave slots 650.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the hierarchy levels and storage or processing of instructions in a lower level based on readiness, as taught by Du ‘808 with the system of Du in order to organize the priority levels taught by Du and process the instructions more efficiently.
As per claim 8, Du alone does not explicitly teach the claimed limitations.
However, Du in combination with Du ‘808 teaches the claimed:
8. The apparatus of claim 1, wherein the at least one processor is further configured to: monitor for whether the data remains pending for one of the first instruction or the second instruction. (Du ‘808 col. 18 lines 40-65: “As illustrated in FIGS. 5A and 5B, aspects of the present disclosure can include context queue 570 to store the context information of the wave slots 550-555. For example, context queue 570 can include 16 context queue slots 571-586. However the number of context queue slots can be scalable to include other amounts, e.g., 32 context queue slots. In some aspects, if wave slots 550-555 have pending wave data being processed, they can issue a request to acquire an execution slot with a wave identification (ID), which each context register can include. Further, upon granting the execution slot, a scheduler can copy active or frequently changed context registers from the context queue 570 to the granted execution slot in a few cycles, fetch the wave instruction, and then start executing the instruction. Additionally, if a wave or execution slot is in a waiting state, it can copy the active context register information to the context queue 570 in a few cycles and surrender its execution slot. In some aspects, if wave input data and/or output results are considered to be in a waiting state, they may not occupy wave or execution slots. Moreover, the context queue 570 can include a wide bandwidth for reduced latency issues. For example, latency may occur during wave instruction switching between the execution slots and the context queue.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the monitoring of pending wave data as taught by Du ‘808 with the system of Du in order to determine which wave pairs should be assigned to be executing next.
As per claim 9, Du alone does not explicitly teach the claimed limitations.
However, Du in combination with Du ‘808 teaches the claimed:
9. The apparatus of claim 8, wherein the at least one processor is further configured to: determine that the data does not remain pending for one of the first instruction or the second instruction; and identify the first instruction and the second instruction as a normal priority wave pair based on the data not being pending for one of the first instruction or the second instruction. (Du ‘808 col. 14 line 59-col. 15 line 18: “As indicated herein, aspects of the present disclosure can convert wave slots into a wave slot hierarchy. Further, aspects of the present disclosure can partition the wave slots into a hierarchy of different levels. For example, a first level of the hierarchy can be the wave slots, each of which can be accessed in parallel or at the same time as other wave slots. The second level of the hierarchy can be the context queue, which can track the wave instructions that are being processed, e.g., by execution units. As the data or wave instructions are processed and cycle back sequentially, the context information stored in the context queue can be copied and sent to the wave or execution slots.
As indicated above, the wave hierarchy can optimize the execution capability of the wave slots, such that idle or standby time at each wave slot is minimized. Aspects of the present disclosure could allow for the reduction of the ratio of wave slots to execution units, e.g., from a ratio of ten-to-five to a ratio of six-to-five. Aspects of the present disclosure can also introduce a data cross bar for the transfer or conversion of context information from the wave slots to the context queue. In some aspects, the size of this data cross bar may not be very large compared to other data cross bars, as the conversion of context information from the wave slots to the context queue does not have a high throughput. Additionally, this wave hierarchy can reduce the size of other data cross bars, as the number of wave slots can be reduced.” Du ‘808 col. 18 lines 40-65: “As illustrated in FIGS. 5A and 5B, aspects of the present disclosure can include context queue 570 to store the context information of the wave slots 550-555. For example, context queue 570 can include 16 context queue slots 571-586. However the number of context queue slots can be scalable to include other amounts, e.g., 32 context queue slots. In some aspects, if wave slots 550-555 have pending wave data being processed, they can issue a request to acquire an execution slot with a wave identification (ID), which each context register can include. Further, upon granting the execution slot, a scheduler can copy active or frequently changed context registers from the context queue 570 to the granted execution slot in a few cycles, fetch the wave instruction, and then start executing the instruction. Additionally, if a wave or execution slot is in a waiting state, it can copy the active context register information to the context queue 570 in a few cycles and surrender its execution slot. In some aspects, if wave input data and/or output results are considered to be in a waiting state, they may not occupy wave or execution slots. Moreover, the context queue 570 can include a wide bandwidth for reduced latency issues. For example, latency may occur during wave instruction switching between the execution slots and the context queue.” After data is no longer pending, the execution is granted and it is moved from the context queue, which is for lower priority instructions as described above. Thus, the instruction will now be normal priority.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the different priority based on data no longer being pending as taught by Du’808 with the system of Du in order to control the overall flow of data and determine which instruction pairs are currently pending and which can be processed.
As per claim 10, Du alone does not explicitly teach the claimed limitations.
However, Du in combination with Du ‘808 teaches the claimed:
10. The apparatus of claim 9, wherein to identify the first instruction and the second instruction as the normal priority wave pair, the at least one processor is configured to: increase a wave priority in a wave pair hierarchy for the first instruction and the second instruction, wherein the normal priority wave pair includes a higher priority compared to a low priority wave pair in the wave pair hierarchy. (Du ‘808 col. 14 line 59-col. 15 line 3: “As indicated herein, aspects of the present disclosure can convert wave slots into a wave slot hierarchy. Further, aspects of the present disclosure can partition the wave slots into a hierarchy of different levels. For example, a first level of the hierarchy can be the wave slots, each of which can be accessed in parallel or at the same time as other wave slots. The second level of the hierarchy can be the context queue, which can track the wave instructions that are being processed, e.g., by execution units. As the data or wave instructions are processed and cycle back sequentially, the context information stored in the context queue can be copied and sent to the wave or execution slots.” The hierarchy can be combined with the priority levels of Du. Du [0061]: “The execution slots 635 and the wave slots 650 may form a hierarchical structure. GPU workload waves may be first loaded into the wave slots 650. Waves in the wave slots 650 may be selected for execution, e.g., based on a priority. The waves selected for execution may be copied from the corresponding wave slots to one or more of the execution slots 635. In particular, the associated context registers including context states or values may be copied from the corresponding wave slots to the one or more of the execution slots 635. Based on the instruction, an appropriate EU may be selected to execute the operations on the waves in the execution slots 635. Once the execution is completed or suspended, the waves may be copied back from the corresponding execution slots 635 to one or more of the wave slots 650. In particular, the associated context registers including context states or values may be copied back from the corresponding execution slots to the one or more of the wave slots 650.”
The context queue is the lower priority level in the hierarchy and can be increased to normal priority when sent for execution.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the hierarchy levels for execution vs context for instructions as taught by Du ‘808 with the system of Du in order to have a hierarchy for the priority levels taught by Du for execution instructions.
As per claim 11, Du teaches the claimed:
11. The apparatus of claim 9, wherein the at least one processor is further configured to: complete the execution of the first instruction and the second instruction based on identification of the first instruction and the second instruction as the normal priority wave pair (Du [0061]: “The execution slots 635 and the wave slots 650 may form a hierarchical structure. GPU workload waves may be first loaded into the wave slots 650. Waves in the wave slots 650 may be selected for execution, e.g., based on a priority. The waves selected for execution may be copied from the corresponding wave slots to one or more of the execution slots 635. In particular, the associated context registers including context states or values may be copied from the corresponding wave slots to the one or more of the execution slots 635. Based on the instruction, an appropriate EU may be selected to execute the operations on the waves in the execution slots 635. Once the execution is completed or suspended, the waves may be copied back from the corresponding execution slots 635 to one or more of the wave slots 650. In particular, the associated context registers including context states or values may be copied back from the corresponding execution slots to the one or more of the wave slots 650.” They are selected based on priority. This can include normal priority in the hierarchy. The instructions’ execution will be based on that priority.).
As per claim 12, Du teaches the claimed:
12. The apparatus of claim 11, wherein to complete the execution of the first instruction and the second instruction, the at least one processor is configured to: complete the execution of the first instruction and the second instruction in a paired wave mode. (Du [0065]: “FIG. 8 is a diagram 800 illustrating an example GPR layout in accordance with one or more techniques of this disclosure. In one or more examples, the sequencer may add tags to incoming waves to indicate two consecutive (back-to-back) incoming waves as a wave pair. The waves in a wave pair may be stored in consecutive or non-consecutive wave slots. As described above, the wave pair may adopt a first (#0) GPR layout and a second (#1) GPR layout, which may help to avoid GPR port conflict, and may allow the execution of half precision ALU operations at the 2× rate. The SP (in particular, the wave scheduler) may, based on the instruction, dynamically select whether to copy a single wave or a wave pair to each execution slot for execution. A decision on whether to copy a single wave or a wave pair may correspond to a decision on workload execution granularity. Copying a single wave to each execution slot for execution may correspond to a first (smaller) granularity. Copying a wave pair to each execution slot for execution (which may be referred to as a super wave mode execution) may correspond to a second (larger) granularity. The SP (in particular, the wave scheduler) may select the first granularity when the waves are associated with a texture (sample) block or a branching block, and may select the second granularity when the waves are not associated with a texture (sample) block or a branching block. Therefore, in one configuration, the SP (in particular, the wave scheduler) may begin to execute the waves with the second granularity (i.e., in the wave pair/super wave mode). The SP (in particular, the wave scheduler) may switch to executing the waves with the first granularity (i.e., in the single wave mode) when a branching block or a texture (sample) block is encountered.”).
As per claim 13, Du alone does not explicitly teach the claimed limitations.
However, Du in combination with Du ‘808 teaches the claimed:
13. The apparatus of claim 8, wherein the at least one processor is further configured to: complete the execution of the first instruction and the second instruction based on an execution unit not being idle for one of the first instruction or the second instruction. (Du ‘808 col. 14 lines 37-59: “As indicated above, aspects of the present disclosure can utilize a hierarchy-based data access procedure to copy and/or save context information into the context queue, e.g., with RAM-based memory storage. GPUs herein can also optimize the execution capability of wave slots that would otherwise be in an idle or standby state. Moreover, aspects of the present disclosure can utilize a reduced number of wave or execution slots while still optimizing or maintaining the same execution capability of these wave or execution slots. In some aspects, aspects of the present disclosure can utilize a reduced number of wave slots, e.g., six execution slots compared to ten wave slots, and/or utilize more efficient wave or execution slots while maintaining the same level of efficiency. Accordingly, the wave slots can execute more wave data and no longer be in an idle state waiting on wave data to be processed. For instance, if a wave slot is not being utilized to execute data, then aspects of the present disclosure can store the context register information of the wave slot in a context queue. In some instances, the context information for each wave instruction that is waiting to be executed may be stored in the context queue.”.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the determination of whether and execution unit is idle as taught by Du ‘808 with the system of Du in order to avoid using an execution unit to execute the wave pair before it is ready.
As per claim 14, Du teaches the claimed:
14. The apparatus of claim 1, wherein to suspend the execution of the first instruction and the second instruction, the at least one processor is configured to: stop the execution of the first instruction and the second instruction further based on a synchronization point or a branching instruction for the first instruction and the second instruction. (Du [0065]: “…The texture block or the branching block may refer to a program segment or a group of instructions, where a texture block may refer to a group of sample instructions, and a branching block may refer to an “if-then-else” program block. In particular, with the first granularity, the SP may execute single waves sequentially in one execution slot, or may execute waves in separate (e.g., two) execution slots in parallel, where a single wave is executed in each execution slot. After the branching block or the texture (sample) block is completed, the SP (in particular, the wave scheduler) may switch back to executing the remaining waves with the second granularity (i.e., in the wave pair/super wave mode).” The remaining waves are different instructions from the first and second. Du [0061]: “The execution slots 635 and the wave slots 650 may form a hierarchical structure. GPU workload waves may be first loaded into the wave slots 650. Waves in the wave slots 650 may be selected for execution, e.g., based on a priority. The waves selected for execution may be copied from the corresponding wave slots to one or more of the execution slots 635. In particular, the associated context registers including context states or values may be copied from the corresponding wave slots to the one or more of the execution slots 635. Based on the instruction, an appropriate EU may be selected to execute the operations on the waves in the execution slots 635. Once the execution is completed or suspended, the waves may be copied back from the corresponding execution slots 635 to one or more of the wave slots 650. In particular, the associated context registers including context states or values may be copied back from the corresponding execution slots to the one or more of the wave slots 650.” It would be obvious to suspend the execution during a branching point for one portion of the branch.).
As per claim 15, Du teaches the claimed:
15. The apparatus of claim 1, wherein to suspend the execution of the first instruction and the second instruction, the at least one processor is configured to: stop the execution of the first instruction and starting the execution of the second instruction based on the second instruction being ready for the execution. (Du teaches that the two instructions in the pair can be executed sequentially. Du [0065]: “FIG. 8 is a diagram 800 illustrating an example GPR layout in accordance with one or more techniques of this disclosure. In one or more examples, the sequencer may add tags to incoming waves to indicate two consecutive (back-to-back) incoming waves as a wave pair. The waves in a wave pair may be stored in consecutive or non-consecutive wave slots. As described above, the wave pair may adopt a first (#0) GPR layout and a second (#1) GPR layout, which may help to avoid GPR port conflict, and may allow the execution of half precision ALU operations at the 2× rate. The SP (in particular, the wave scheduler) may, based on the instruction, dynamically select whether to copy a single wave or a wave pair to each execution slot for execution. A decision on whether to copy a single wave or a wave pair may correspond to a decision on workload execution granularity. Copying a single wave to each execution slot for execution may correspond to a first (smaller) granularity. Copying a wave pair to each execution slot for execution (which may be referred to as a super wave mode execution) may correspond to a second (larger) granularity. The SP (in particular, the wave scheduler) may select the first granularity when the waves are associated with a texture (sample) block or a branching block, and may select the second granularity when the waves are not associated with a texture (sample) block or a branching block. Therefore, in one configuration, the SP (in particular, the wave scheduler) may begin to execute the waves with the second granularity (i.e., in the wave pair/super wave mode). The SP (in particular, the wave scheduler) may switch to executing the waves with the first granularity (i.e., in the single wave mode) when a branching block or a texture (sample) block is encountered. The texture block or the branching block may refer to a program segment or a group of instructions, where a texture block may refer to a group of sample instructions, and a branching block may refer to an “if-then-else” program block. In particular, with the first granularity, the SP may execute single waves sequentially in one execution slot, or may execute waves in separate (e.g., two) execution slots in parallel, where a single wave is executed in each execution slot. After the branching block or the texture (sample) block is completed, the SP (in particular, the wave scheduler) may switch back to executing the remaining waves with the second granularity (i.e., in the wave pair/super wave mode).”).
As per claim 16, Du teaches the claimed:
16. The apparatus of claim 1, wherein the at least one processor is further configured to: obtain an indication of the set of instructions including the first instruction and the second instruction, wherein the set of instructions is associated with the graphics processing; (Du [0021]: “By way of example, an element, or any portion of an element, or any combination of elements may be implemented as a “processing system” that includes one or more processors (which may also be referred to as processing units). Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), general purpose GPUs (GPGPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, systems-on-chip (SOCs), baseband processors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functionality described throughout this disclosure. One or more processors in the processing system may execute software. Software can be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software components, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.”).
and assign the first instruction and the second instruction to an execution unit in a set of execution units at a graphics processing unit (GPU), and wherein to initiate the execution of the first instruction and the second instruction, the at least one processor is configured to: (Du [0037]: “Context states can be utilized to determine how an individual processing unit functions, e.g., a vertex fetcher (VFD), a vertex shader (VS), a shader processor, or a geometry processor, and/or in what mode the processing unit functions. In order to do so, GPUs can use context registers and programming data. In some aspects, a GPU can generate a workload, e.g., a vertex or pixel workload, in the pipeline based on the context register definition of a mode or state. Certain processing units, e.g., a VFD, can use these states to determine certain functions, e.g., how a vertex is assembled. As these modes or states can change, GPUs may need to change the corresponding context. Additionally, the workload that corresponds to the mode or state may follow the changing mode or state.” These GPUs would be used for the instructions.).
initiate the execution of the first instruction and the second instruction at the execution unit. (Du [0049]: “In some aspects, the wave scheduler 328 may process a pixel workload and/or issue instructions to various execution units, e.g., EUs 334. The wave scheduler 328 may also help to ensure data dependency between instructions, e.g., data dependency between ALU operands due to the pipeline latency and/or texture sample return data dependency based on a synchronization mechanism. Additionally, the wave scheduler 328 may have a load reference counter (LRC) to count outstanding texture or memory requests that are issued to the TP 306 or UCHE 308, as well as a corresponding data return request. In some aspects, if the LRC value is greater than zero, this may indicate there is outstanding data. As such, instructions that are dependent on the return of the outstanding data may not be able to execute until the LRC value decreases to zero. After the shader processing is completed, the streaming processor 320 may send the processing results to a downstream block (e.g., the RB 310).” The issue of instructions to the execution units is the initiation.).
As per claim 17, Du teaches the claimed:
17. The apparatus of claim 1, wherein the first instruction is associated with a first wave and the second instruction is associated with a second wave at a graphics processing unit (GPU), wherein the first wave and the second wave correspond to a wave pair, (Du [0025]: “A compiler may select a wave size (e.g., 1× or 2×) based on the shader type and performance statistics, and may communicate with the driver to set the streaming processor (SP) wave size context register. However, because the shader execution is highly dynamic, it may be difficult to always select the optimal wave size (e.g., the wave size associated with a shorter overall execution time). In one or more aspects, the compiler's role in selecting the wave size may be eliminated. A uniform 1× wave granularity in GPR allocation may be used. The wave size may be changed dynamically per instruction during execution time. Accordingly, the instruction fetch may be more efficient as a single instruction may be executed for two or more different waves. Memory and cache utilization may be more efficient as texture processing and load/store operations may be executed on paired or grouped waves in a single region in close proximity.”
Du [0065]: “FIG. 8 is a diagram 800 illustrating an example GPR layout in accordance with one or more techniques of this disclosure. In one or more examples, the sequencer may add tags to incoming waves to indicate two consecutive (back-to-back) incoming waves as a wave pair. The waves in a wave pair may be stored in consecutive or non-consecutive wave slots. As described above, the wave pair may adopt a first (#0) GPR layout and a second (#1) GPR layout, which may help to avoid GPR port conflict, and may allow the execution of half precision ALU operations at the 2× rate. The SP (in particular, the wave scheduler) may, based on the instruction, dynamically select whether to copy a single wave or a wave pair to each execution slot for execution. A decision on whether to copy a single wave or a wave pair may correspond to a decision on workload execution granularity. Copying a single wave to each execution slot for execution may correspond to a first (smaller) granularity. Copying a wave pair to each execution slot for execution (which may be referred to as a super wave mode execution) may correspond to a second (larger) granularity.”).
wherein the set of instructions is a set of shader instructions associated with a shader code for the graphics processing, (Du [0025]: “A compiler may select a wave size (e.g., 1× or 2×) based on the shader type and performance statistics, and may communicate with the driver to set the streaming processor (SP) wave size context register. However, because the shader execution is highly dynamic, it may be difficult to always select the optimal wave size (e.g., the wave size associated with a shorter overall execution time). In one or more aspects, the compiler's role in selecting the wave size may be eliminated. A uniform 1× wave granularity in GPR allocation may be used. The wave size may be changed dynamically per instruction during execution time. Accordingly, the instruction fetch may be more efficient as a single instruction may be executed for two or more different waves. Memory and cache utilization may be more efficient as texture processing and load/store operations may be executed on paired or grouped waves in a single region in close proximity.”).
wherein the first instruction is a first texture load instruction or a first memory load instruction, and wherein the second instruction is a second texture load instruction or a second memory load instruction. (Du [0043]: “FIG. 3 illustrates an example GPU 300 in accordance with one or more techniques of this disclosure. More specifically, FIG. 3 illustrates a streaming processor system in GPU 300. As shown in FIG. 3, GPU 300 includes high level sequencer (HLSQ) 302, texture processor (TP) 306, UCHE 308, and RB 310. GPU 300 also includes streaming processor 320, sequencer 324, wave scheduler and context register 328, texture unit (TEX) or load controller 330, instruction cache 332, EUs 334, general purpose register (GPR) 336, and distributor 338. The wave scheduler and context register 328 may also include one or more wave slots.”).
As per claim 18, Du teaches the claimed:
18. The apparatus of claim 1, further comprising: outputting an indication of suspension of the execution of the first instruction and the second instruction. (Du [0061]: “The execution slots 635 and the wave slots 650 may form a hierarchical structure. GPU workload waves may be first loaded into the wave slots 650. Waves in the wave slots 650 may be selected for execution, e.g., based on a priority. The waves selected for execution may be copied from the corresponding wave slots to one or more of the execution slots 635. In particular, the associated context registers including context states or values may be copied from the corresponding wave slots to the one or more of the execution slots 635. Based on the instruction, an appropriate EU may be selected to execute the operations on the waves in the execution slots 635. Once the execution is completed or suspended, the waves may be copied back from the corresponding execution slots 635 to one or more of the wave slots 650. In particular, the associated context registers including context states or values may be copied back from the corresponding execution slots to the one or more of the wave slots 650.” The change of the registers is the indication. Du [0036]: “GPUs can process multiple types of data or data packets in a GPU pipeline. For instance, in some aspects, a GPU can process two types of data or data packets, e.g., context register packets and draw call data. A context register packet can be a set of global state information, e.g., information regarding a global register, shading program, or constant data, which can regulate how a graphics context will be processed. For example, context register packets can include information regarding a color format. In some aspects of context register packets, there can be a bit that indicates which workload belongs to a context register. Also, there can be multiple functions or programming running at the same time and/or in parallel. For example, functions or programming can describe a certain operation, e.g., the color mode or color format. Accordingly, a context register can define multiple states of a GPU.” This indication in the register that can show suspension of a certain workload.).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS JOHN FOSTER whose telephone number is (571)272-5053. The examiner can normally be reached Mon, Fri 8:30-6. Tues-Thurs 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS JOHN FOSTER/Examiner, Art Unit 2616
/DANIEL F HAJNIK/Supervisory Patent Examiner, Art Unit 2616
Prosecution Insights