Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Response to Amendment
This is in response to applicant’s amendment/response filed on 02/19/2026, which has been entered and made of record. Claims 1, 19 and 20 have been amended. Claim 3 has been cancelled. Claims 1-2, 4-21 are pending in the application.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/19/2026 has been entered.
Response to Arguments
Applicant’s arguments on 02/19/2026 have been fully considered but are moot because the arguments do not apply to any of the references being used in the current rejection.
Applicant submits the prior arts did not disclose “One or more of the shader stages are executed in both the geometry processing phase and the rasterisation phase. In particular, at least the final shader stage is executed in both the geometry processing phase and the rasterisation phase. The same transformed geometry position data is determined in both the geometry processing phase and the rasterisation phase by executing the final shader stage. - The shader stage output data that is written to a memory as an output of the geometry processing phase is fetched from the memory in the rasterisation phase. In the rasterisation phase, the one or more shader stages including the final shader stage are executed on the fetched shader stage output data.”
The newly provided Ellis et al. (US Pub 2017/0206698 A1) can teach these features because Ellis in ¶0046 discloses “output data for a thread that is to be subsequently used as input data for plural execution threads (e.g. output data from any “common” calculations in a shader program being executed) can be (and in an embodiment is) stored locally to the graphics processing unit in the further local memory, thereby avoiding having to write that data out to external (main) memory and then read it in from external memory to each thread's register (which can be expensive in terms of bandwidth and power).”.
Ellis issues a pilot thread that executes (the instructions for) graphics processing (shader) program expressions (operations) that will produce a common (the same) result for all the threads (work items) of a group of threads (work items) that the graphics processing (shader) program is to be executed for, in an embodiment so as to generate and store the output values for those expressions (¶0151). Ellis further teaches the results (data value(s)) that are generated by execution of (the instructions for) the common expressions in the graphics processing (shader) program should be, and are in an embodiment, made available for use to other threads that are executing the graphics processing (shader) program (and in particular to the “main threads” or the (other) threads that start the graphics processing (shader) program at a later point and thus do not themselves execute the instructions for the common expressions in question). These common expression results (data values) can be provided for use for other execution threads in any suitable and desired manner (¶0161), and the common expression results are stored in the further local memory, e.g. so as to be accessible to other execution threads that are executing the graphics processing (shader) program (¶0162). Since Ellis’s shaders include executing geometry phase and rasteriser phase out of order, it is obvious to have “one or more of the shader stages are executed in both the geometry processing phase and the rasterisation phase.” and the common expression results will be “The shader stage output data that is written to a memory as an output of the geometry processing phase is fetched from the memory in the rasterisation phase.”.
Applicant submits “It is noted again that occlusion culling does not "determine transformed geometry position data." In particular, occlusion culling does not determine the position of anything; instead, occlusion culling determines whether a fragment is occluded at a particular position.”
The examiner disagrees with Applicant’s premises and conclusion. Applicant’s specification (¶0073 of publication) states “The shader stage output data may include the outputs generated by one or more shader stages (for example, excluding at least the final shader stage—e.g. excluding clipping/culling and/or geometry shading stages) during the geometry processing of each patch”. Therefore, the clipping/culling is the final shader stage.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 4, 6-12, 15-21 are rejected under 35 U.S.C. 103 as being unpatentable over Howson et al. (US Pub 2015/0317818 A1) in view of Mei et al. (US Pub 2015/0235341 A1) and Paltashev (US Pub 2017/0372509), further in view of Ellis et al. (US Pub 2017/0206698 A1).
As to claim 1, Howson discloses a graphics processing system comprising:
geometry processing logic configured to: in a geometry processing phase execute a plurality of shader stages, including a final shader stage of the plurality of shader stages, on input graphics data items to determine transformed geometry position data (Howson, ¶0051, “Hull shader 29 can calculate tessellation factors for edges of a given patch and may also further modify the transformed control points.” ¶0074, “The implementation can run threads of computation that receive inputs and/or produce outputs from another thread in order to implement a sequence of geometry modifications required. In other examples, intermediately processed geometry data may be stored in memory and read/written as required by one or more threads of computation implementing the geometry processing.”), and
write to a memory, shader stage output data for one or more shader stages of the plurality of shader stages (Howson, ¶0051, “Shader 37 can perform further programmable shading operations on geometry and produce vertices to be stored in vertex cache 41.” ¶0074, “The implementation of this geometry processing may be performed differently in different implementations. For example, where a programmable cluster of computation units is available to perform geometry computation, shading computation (e.g., shading of ray intersections, vertex shading, pixel shading, and so on), ray traversal, or other kinds of tasks, that programmable cluster can be configured to implement the geometry processing required. The implementation can run threads of computation that receive inputs and/or produce outputs from another thread in order to implement a sequence of geometry modifications required. In other examples, intermediately processed geometry data may be stored in memory and read/written as required by one or more threads of computation implementing the geometry processing.” ¶0075, “some geometry modifications may be incremental. So, even if it is can be indicated that a certain portion of an incremental series of geometry elements are located within a particular tile, it may not be possible to generate only those instances, because later instances may depend on iterating through earlier primitives in a programmatic sequence. In one implementation, unnecessarily repetition computation is reduced by storing incremental geometry data in a local memory, such as a cache. This has the advantage that, where the instances are spread over more than one tile, predecessor geometry elements may not need to be recreated for each tile. In such an implementation, it can be defined which portions of an incremental series of geometry are within a particular tile or 3-D acceleration structure element, and then those elements can be first looked for within a cache, before beginning a geometry process to reproduce those elements, which may require iterating through predecessor elements that are not within that tile or acceleration structure element.” Fig. 13, ¶0101, “These processes can communicate with memory arbitration/control 470, which is responsible for managing cache hierarchy 476 (and for example, main memory 478); this element can use the caching hints described above. Processor scheduling 474 is responsible for coarse-grained scheduling or allocation of different tasks to different computation resources.”;
and rasterisation logic configured to: in a rasterization phase, fetch the shader stage output data from the memory and execute one or more shader stages including the final shader stage on the fetched shader stage output data to determine the transformed geometry position data and use the transformed geometry position data to generate a rendering output (Howson, Fig. 1, Fig. 2B-Fig. 2C, ¶0056, “Rasterization 54 can use an output of a screen space geometry buffer 51, which expresses final geometry mapped to screen space.” ¶0081, “FIG. 2C depicts a rasterization flow where screen space tiling 123 can be performed by beginning to process a tile (125) including accessing 2-D screen space object extent data (127) (e.g., an object list) and checking whether identified final geometry is cached (133) and to the extent not cached, providing identified source geometry and geometry process identification (129) to a geometry processing unit. On retrieved or recreated final geometry, HSR (visible surface identified) can be performed. In some implementations, even though certain final geometry is not visible within a tile, some of that geometry may be cached, because it may be used for ray traversal operations. In some situations, a cache requested flag may be set for certain final geometry, which would indicate demand on the ray traversal side for certain portions of final geometry (or vice versa).”).
Howson does not discloses write to a memory, as an output of the geometry processing phase, shader stage output data for one or more shader stages of the plurality of shader stages not including a final shader stage of the plurality of shader stages used to determine the transformed geometry position data.
Mei teaches write to a memory, as an output of the geometry processing phase, shader stage output data for one or more shader stages of the plurality of shader stages not including a final shader stage of the plurality of shader stages used to determine the transformed geometry position data (Mei, abstract, “allocate a shared data channel in on-chip graphics memory of the GPU that is shared by at least two stages of a graphics processing pipeline. Shader units in the GPU may execute the at least two stages of the graphics processing pipeline. The GPU may store, in the shared data channel in on-chip graphics memory, data produced by each of the at least two stages of the graphics processing pipeline executing on the shader units.” Fig. 4, ¶0055, “shared data channel 50A may be shared by stages of graphics processing pipeline 24 to store data produced by the stages.” ¶0056, “Specifically, shared data channel 50A may store data 52 produced by hull shader stage 30 of graphics processing pipeline 24 and may further store data 54 produced by geometry shader stage 36 of graphics processing pipeline 24. Data 52 may be consumed by domain shader stage 34 of graphics processing pipeline 24 and data 54 may be consumed by pixel shader stage of graphics processing pipeline 24.” ¶0058, “the same data produced by some stages of graphics processing pipeline 24 may be consumed multiple times by other stages of graphics processing pipeline 24.” ¶0065, “geometry shader 36, respectively, so that cache mode shared channel 56 and shared data channel 50A do not only store data produced by vertex shader 28 and hull shader 30, respectively. GPC 42 may determine the amount of space of cache mode shared channel 56 in both components shared primitive queue 50B and shared vertex cache 50C, and the amount of space of shared data channel 50A to reserve by, for example, determining the amount of space necessary to store output from domain shader 34 and geometry shader 36 for a given number of waves in shader cluster 46.”
PNG
media_image1.png
595
956
media_image1.png
Greyscale
).
Wei also teaches rasterisation logic configured to: in a rasterization phase, fetch the shader stage output data from the memory and execute one or more shader stages including the final shader stage on the fetched shader stage output data to determine the transformed geometry position data and use the transformed geometry position data to generate a rendering output (Wei, ¶0036, “Rasterizer stage 37 is typically a fixed function stage that is responsible for clipping primitives and preparing primitives for pixel shader stage 38. For example, rasterizer stage 37 may perform clipping (including custom clip boundaries), perspective divide, viewport/scissor selection and implementation, render target selection and primitive setup. In this way, rasterizer stage 37 may generate a number of fragments for shading by pixel shader stage 38.”
PNG
media_image2.png
738
305
media_image2.png
Greyscale
).
Howson and Mei are considered to be analogous art because all pertain to image rendering. It would have been obvious before the effective filing date of the claimed invention to have modified Howson with the features of “write to a memory, as an output of the geometry processing phase, shader stage output data for one or more shader stages of the plurality of shader stages not including the final shader stage of the plurality of shader stages used to determine the transformed geometry position data” and “rasterisation logic configured to: in a rasterization phase, fetch the shader stage output data from the memory and execute one or more shader stages including the final shader stage on the fetched shader stage output data to determine the transformed geometry position data and use the transformed geometry position data to generate a rendering output” as taught by Mei. The suggestion/motivation would have been because the on-chip memory in the GPU may store more data that are ready to be consumed by the shader units executing stages of the shader pipeline, thereby increasing utilization of the shader units and increasing the performance of the GPU (Mei, ¶0017).
Paltashev teaches write to a memory, as an output of the geometry processing phase, shader stage output data for one or more shader stages of the plurality of shader stages not including the final shader stage of the plurality of shader stages used to determine the transformed geometry position data (Paltashev, Fig. 1, ¶0017, “The graphics pipeline 102 in the example graphics processing system 100 includes a geometry front-end 115 that processes patches received from the input assembler 105. The geometry front-end operates in object space. Some implementations of the geometry front-end 115 include one or both of a vertex shader 120 for performing shading operations on vertices received from the input assembler 105 and a hull shader 125 that operates on patches (or the control points of patches) received from the input assembler 105. In some variations, the hull shader 125 can also generate and output tessellation factors that are used for tessellating the patches, as well as other patch parameters or constants. In some implementations, the output patches, control points, tessellation factors and the like can be provided to one or more buffers 127, such as primitive buffers, index buffers, vertex buffers, and the like. In some implementations, the vertex shader 120 or the hull shader 125 are implemented as software running on one or more hardware components such as processors, processor cores, compute units, and the like. Some implementations of the geometry front-end 115 can include other shaders or different combinations of shaders that are used to perform similar functionality to the vertex shader 120 and the hull shader 125.” ¶0019, “Some implementations of the geometry back-end 135 include an additional vertex shader 145 that is used to shade the vertices of the primitives (or subdivided primitives) produced by the geometry shader 140.” “the geometry back-end 135 provides output to the buffers 127 and access information stored in the buffers 127. “ ¶0023, “values in the hierarchical buffer 160 are updated using feedback from the graphics pipeline. For example, the hierarchical buffer 160 can be updated using z-values of rendered objects in response to completion of geometry workloads such as geometry workloads that include a high number of primitives. For another example, the hierarchical buffer 160 can be updated in response to flushing one or more lines in the cache 112.” ¶0024, “a shader in a geometry portion of the graphics pipeline, such as the hull shader 125 in the geometry front-end 115, can access information representing an object such as a patch that represents a portion of a model of a scene in object space. The patch information is accessed from the cache 112, the buffers 127, or other storage locations. The shader also accesses the hierarchical buffer 160 or cache 112 to retrieve one or more far-z values that indicate a furthest distance of a previously rendered portion of one or more tiles that overlaps the patch in screen space.”
PNG
media_image3.png
630
905
media_image3.png
Greyscale
).
Paltashev also teaches “rasterisation logic configured to: in a rasterization phase, fetch the shader stage output data from the memory and execute one or more shader stages including the final shader stage on the fetched shader stage output data to determine the transformed geometry position data and use the transformed geometry position data to generate a rendering output” (Paltashev, ¶0012, “Occlusion culling is used to remove primitives or rasterized fragments that are obscured by opaque objects that intervene between the primitive or rasterized fragment and the virtual camera. For example, the rasterized fragments that represent the image of the 3-D model can be projected into screen space and gathered into groups of tiles that each represent a portion of the screen space, such as an 8×8 group of tiles. The 3-D position of each rasterized fragment is represented by coordinates (x, y, z) of a pixel or ranges of coordinates when the fragment includes more than one pixel.” ¶0020, “A pixel shader 150 in the graphics pipeline 102 shades pixels or rasterized fragments (e.g., a piece of a primitive mapped into screen space) based on the primitives or fragments from the geometry back-end 135.”).
Howson and Paltashev are considered to be analogous art because all pertain to image rendering. It would have been obvious before the effective filing date of the claimed invention to have modified Howson with the features of “write to a memory, as an output of the geometry processing phase, shader stage output data for one or more shader stages of the plurality of shader stages not including the final shader stage of the plurality of shader stages used to determine the transformed geometry position data” and “rasterisation logic configured to: in a rasterization phase, fetch the shader stage output data from the memory and execute one or more shader stages including the final shader stage on the shader stage output data to determine the transformed geometry position data and use the transformed geometry position data to generate a rendering output” as taught by Paltashev. The suggestion/motivation would have been that primitives and other higher-order objects such as patches are culled in the geometry front-end or back-end of the graphics pipeline so that occluded objects are not emitted to the geometry back-end or pixel portion of the graphics pipeline (Paltashev, ¶0014.).
Further, Ellis teaches in a geometry processing phase, execute a plurality of shader stages, including a final shader stage of the plurality of shader stages, on input graphics data items to determine transformed geometry position data (Fig. 5, ¶0234, “the graphics processing pipeline 30 includes a number of stages, including vertex shader 31, a hull shader 32, a tesselator 33, a domain shader 34, a geometry shader 35, a rasterisation stage 36, an early Z (depth) and stencil test stage 37, a renderer in the form of a fragment shading stage 38, a late Z (depth) and stencil test stage 39, a blending stage 40, a tile buffer 41 and a downsampling and writeout (multisample resolve) stage 42.” Since Ellis’s shader code is executed out of order, any stage with end result can be mapped to the final stage.), and
write to a memory, as an output of the geometry processing phase, shader stage output data for one or more shader stages of the plurality of shader stages not including the final shader stage of the plurality of shader stages used to determine the transformed geometry position data (¶0140, “the graphics processing (shader) program is in an embodiment arranged such that (the instructions for) any graphics processing (shader) program expressions (operations) that will produce a common (the same) result for all the threads (work items) of a group of threads (work items) that the graphics processing (shader) program is to be executed for appear earlier in (and in an embodiment at the beginning of) the execution order of the graphics processing (shader) program, any (instructions for) graphics processing (shader) program expressions (operations) that are (definitely) required to be executed for each thread (work item) individually (i.e. that potentially will produce a different result for each individual thread (work item)) appear towards the end of the graphics processing (shader) program execution (and at least after some, and in an embodiment after all, of the (instructions for) graphics processing (shader) program expressions (operations) that will produce common results for plural threads (work items)).” ¶0164, “The results (data value(s)) that are generated by execution of (the instructions for) the common expressions should be (and in an embodiment are) made available (stored in the further local memory) for use by other threads that are executing the graphics processing (shader) program (and in particular to the “main threads” or the (other) threads that start the graphics processing (shader) program at a later point and thus do not themselves execute the instructions for the common expressions in question) before the other threads require those results.” ¶0169, “an execution thread can be configured to be able to start execution of the shader program at a later stage in the shader program in any suitable and desired manner. For example, threads could be allocated different initial program counter-values to set their different “entry points” to the shader program (and in one embodiment this is done). In this case, the different start points within the shader program could be indicated by means of different absolute program counter-values, or there could be one program counter-value that, e.g., indicates a first start point in the shader program, with an offset or offsets to that program counter-value then being used to indicate and trigger other start points within the shader program.”); and
rasterisation logic configured to: in a rasterisation phase, fetch the shader stage output data from the memory and execute one or more shader stages including the final shader stage on the fetched shader stage output data to determine the transformed geometry position data and use the transformed geometry position data to generate a rendering output (Ellis, ¶0046, “output data for a thread that is to be subsequently used as input data for plural execution threads (e.g. output data from any “common” calculations in a shader program being executed) can be (and in an embodiment is) stored locally to the graphics processing unit in the further local memory, thereby avoiding having to write that data out to external (main) memory and then read it in from external memory to each thread's register (which can be expensive in terms of bandwidth and power).” ¶0176, “instructions or groups of plural instructions in the graphics processing (shader) program are indicated as being dependent upon the storing of the common expressions results (data values) by a thread, and threads that start the graphics processing (shader) program after (the instructions for) the common expressions in the graphics processing (shader) program (but before the instruction(s) or set(s) of plural instructions that are dependent upon the common expressions) are only allowed to begin executing the instruction(s) or set(s) of plural instructions in question once the common expressions results (data values) have been stored.” ¶0194, “The programmable execution unit(s) of the graphics processing unit that executes the graphics processing (shader) program(s) may, and in an embodiment does, function (operate) as (the graphics processing pipeline may include) any suitable and desired graphics processing shader stage (shaders), such as a vertex shader, a geometry shader, fragment shader, compute shader, etc. In an embodiment it can operate as each of these shaders” ¶0196. ¶0236-0237. Fig. 6).
Howson, Mei, Paltashev and Ellis are considered to be analogous art because all pertain to image rendering pipeline. It would have been obvious before the effective filing date of the claimed invention to have modified Howson with the features of “write to a memory, as an output of the geometry processing phase, shader stage output data for one or more shader stages of the plurality of shader stages not including the final shader stage of the plurality of shader stages used to determine the transformed geometry position data; and rasterisation logic configured to: in a rasterisation phase, fetch the shader stage output data from the memory and execute one or more shader stages including the final shader stage on the fetched shader stage output data to determine the transformed geometry position data and use the transformed geometry position data to generate a rendering output.” as taught by Ellis. The suggestion/motivation would have been in order to avoiding having to write that data out to external (main) memory and then read it in from external memory to each thread's register (which can be expensive in terms of bandwidth and power) (Ellis, ¶0046) and the claim would have been obvious because “a person of ordinary skill has good reason to pursue the known options within his or her technical grasp. If this leads to the anticipated success, it is likely the product not of innovation but of ordinary skill and common sense.”.
As to claim 2, claim 1 is incorporated and the combination of Howson, Mei, Paltashev and Ellis discloses writing shader stage output data to a memory for one or more shader stages of the plurality of shader stages comprises writing shader stage output data to a memory for each instance of the one or more shader stages of the plurality of shader stages (Howson, ¶0005, “Both rasterization and ray tracing use geometry data as an input, to define objects that are located in the 3-D scene.” ¶0056, “Rasterization 54 can use an output of a screen space geometry buffer 51, which expresses final geometry mapped to screen space. For example, rasterization 54 can operate on tiles of geometry.” Also see Mei, Fig.4, Paltashev, Fig. 1.)
As to claim 4, claim 1 is incorporated and the combination of Howson, Mei, Paltashev and Ellis discloses said one or more shader stages comprise a pre-determined shader stage (Howson, Fig. 1, also see Mei, Fig.4, Paltashev, Fig. 1.).
As to claim 6, claim 1 is incorporated and the combination of Howson, Mei, Paltashev and Ellis discloses some of the graphics data items are control points describing a patch to be tessellated to generate a plurality of tessellated primitives, and wherein the shader stages comprise one or more of:(i) vertex shading, (ii) hull shading, (iii) domain shading, (iv) tessellation, (v) geometry shading, and (vi) and clipping (Howson, Fig . 1, ¶0048, “geometry data 25 can be submitted (e.g. streamed) for processing. In an example, patches of geometry can be supplied to a vertex shader 27, which can in turn provide shaded vertex data to a hull shader 29. Vertex shader 27 can, for example, programmatically move control points of patches (e.g., a combination of one or more of translation, rotation, or scaling), or perform initial vertex lighting, or both, or other types of programmatic operations on these control points. These control points can be provided to units that perform tessellation-related functions.” also see Mei, Fig.4, Paltashev, Fig. 1.).
As to claim 7, claim 1 is incorporated and the combination of Howson, Mei, Paltashev and Ellis discloses the rasterisation logic is configured to generate the rendering output in a rendering space which is divided into a plurality of tiles, wherein sub-primitives are derived from the input graphics data items, a sub-primitive being used for rendering a tile if the sub-primitive is at least partially in the tile (Howson, ¶0019, “The method comprises producing geometric primitives located in a 3-D scene from source geometry data and defining a set of tile object lists. Each tile object tile contains data indicating which source geometry from the set of source geometry results in geometric primitives that are within a boundary of a respective tile of pixels of a 2-D image. The method also comprises producing an acceleration structure comprising a graph of elements, each defining a respective volume in the 3-D scene and rendering the 2-D image from the scene by using the tile object lists to identify a visible surface at each pixel of the 2-D image.” ¶0020, “The method also comprises defining a set of tile object lists. Each tile object list identifies source geometry and a respective set of modification processes to be performed on the identified source geometry to produce final geometry within a tile of pixels within a 2-D image to be rendered from the 3-D scene, and rendering the 2-D image by identifying visible surfaces for pixels within the 2-D image on a tile by tile basis. The rendering comprises identifying source geometry from the tile object list for each tile, and performing the set of modification processes on the source geometry to produce final geometry. A visible surface for each pixel in that tile is identified based on the produced final geometry, and to complete the rendering for a group of the pixels, rays are emitted from the visible surface for pixels and traversed in the 3-D scene. The rays are traversed in the 3-D scene.”).
As to claim 8, claim 1 is incorporated and the combination of Howson, Mei, Paltashev and Ellis discloses the input graphics data items describe geometry within a 3D scene to be rendered, and wherein the rendering output is a rendered image of the scene (Howson, ¶0019, “The method comprises producing geometric primitives located in a 3-D scene from source geometry data and defining a set of tile object lists. Each tile object tile contains data indicating which source geometry from the set of source geometry results in geometric primitives that are within a boundary of a respective tile of pixels of a 2-D image. The method also comprises producing an acceleration structure comprising a graph of elements, each defining a respective volume in the 3-D scene and rendering the 2-D image from the scene by using the tile object lists to identify a visible surface at each pixel of the 2-D image, and tracing rays in the 3-D scene, using the acceleration structure, from the identified visible surfaces to identify a primitive intersected by each of the rays, if any, and producing information contributing to a final shade for the visible surface at each pixel.”).
As to claim 9, claim 1 is incorporated and the combination of Howson, Mei, Paltashev and Ellis discloses the shader stage output data written to memory comprises vertex data output from a first shader stage and additional data from a further shader stage that is subsequent to the first shader stage (Howson, Fig .1, ¶0015, “the source geometry data comprising vertex data and two or more sets of vertex connectivity data.” ¶0039, “vertexes of primitives can be used as input to tessellation and to geometry shaders or displacement engines that can amplify a number of primitives by dicing a larger primitive into smaller ones, or otherwise deform geometry so that an extent of geometry after these further geometry operations is different from an original extent.” ¶0051, “Shader 37 can perform further programmable shading operations on geometry and produce vertices to be stored in vertex cache 41.”).
As to claim 10, claim 1 is incorporated and the combination of Howson, Mei, Paltashev and Ellis discloses the rasterisation logic is configured to generate the rendering output in a rendering space which is divided into a plurality of tiles, the rasterisation logic further comprising: a cache configured to store a hierarchy of shader stage output data with different levels of the hierarchy representing output data from different re-executed stages of the plurality of shader stages for use in generating the rendering outputs for the tiles (Howson, Fig .14, ¶0024, “Such systems may comprise one or more memories for storing source geometry, and cache(s) for storing finalized geometry. These memories may be part of a memory hierarchy.”. ¶0051, “Shader 37 can perform further programmable shading operations on geometry and produce vertices to be stored in vertex cache 41. Vertices stored in vertex cache 41 can serve as a basis for producing data to be used during rasterization (e.g., tile-based rasterization) and during ray tracing operations.” ¶0052-0053, Fig. 13, ¶0101-¶0102, “These vectorized execution units 504-506 communicate with a cache hierarchy 510. Cache hierarchy 510 may provide a set of private memories (e.g., an L1 cache) for each execution unit, as well as other levels of cache hierarchy.” ¶0106, “More specifically relating to geometry processing, a vertex shader or geometry shader (outputting polygons/geometric primitives) may have its output sent towards or enqueued to a hierarchy building process.” Also see Paltashev, ¶0022, “A hierarchical-Z (HiZ) buffer 160 is used to store values that indicate distances associated with portions of the screen space. Some implementations of the hierarchical buffer 160 store values of the z coordinate range for hierarchically ordered groups of tiles as a pair of values: (near-Z, far-Z). For example, different levels in the hierarchical buffer 160 can include 2×2 groups of tiles, 4×4 groups of tiles that encompass the 2×2 groups of tiles, and an 8×8 group of tiles that encompasses the 4×4 groups of tiles.” ¶0036, “the tiles can be selected from a level of a hierarchical set of tiles in which the tiles in a higher level encompass a predetermined number of tiles in the next lower level. The geometry back-end 505 accesses the hierarchical buffer 415 that stores near-z and far-z values for the tiles in the different levels of the hierarchy.”).
As to claim 11, claim 10 is incorporated and the combination of Howson, Mei, Paltashev and Ellis discloses the cache comprises a plurality of memory pools, wherein different ones of the memory pools are configured to store shader stage output data from different levels of the hierarchy (Howson, ¶0068, “One principal difference is that efficiency in ray tracing complex scenes typically is enhanced by creating an acceleration structure that has different levels of abstraction of geometry in the 3-D scene. For example, an acceleration structure may be a hierarchical arrangement of elements, where each of the elements bounds a respective volume in 3-D space.” ¶0086, “FIG. 7 depicts an approach to tuning a level of coarseness in an acceleration structure that will be used to subdivide a 3-D scene into different portions, for the purpose of on-demand creation of final geometry of those different portions. A coarse acceleration structure may be defined based on a criteria of how much memory can be allocated to storing the coarse acceleration structure.” ¶0091, “FIG. 9 depicts an example memory map 382 of how vertex data and vertex connectivity data can be arranged in memory. In FIG. 9, vertex definition data for low-resolution geometry 388 can be stored separately from supplemental vertex definition data 390, and vertex connectivity data for low resolution geometry 392 can be stored separately from vertex connectivity data for high-resolution geometry 394. In order to create geometry at a particular resolution, the base line low resolution geometry may always be accessed, and then selections from supplemental vertex definition data can be accessed, along with appropriate portions of vertex connectivity data.” Fig. 14, ¶0102, “These vectorized execution units 504-506 communicate with a cache hierarchy 510. Cache hierarchy 510 may provide a set of private memories (e.g., an L1 cache) for each execution unit, as well as other levels of cache hierarchy.” Also see Paltashev, ¶0022, “A hierarchical-Z (HiZ) buffer 160 is used to store values that indicate distances associated with portions of the screen space. Some implementations of the hierarchical buffer 160 store values of the z coordinate range for hierarchically ordered groups of tiles as a pair of values: (near-Z, far-Z). For example, different levels in the hierarchical buffer 160 can include 2×2 groups of tiles, 4×4 groups of tiles that encompass the 2×2 groups of tiles, and an 8×8 group of tiles that encompasses the 4×4 groups of tiles.” ¶0036, “the tiles can be selected from a level of a hierarchical set of tiles in which the tiles in a higher level encompass a predetermined number of tiles in the next lower level. The geometry back-end 505 accesses the hierarchical buffer 415 that stores near-z and far-z values for the tiles in the different levels of the hierarchy.”).
As to claim 12, claim 10 is incorporated and the combination of Howson, Mei, Paltashev and Ellis discloses the cache is configured to store shader stage output data for one or more levels of the hierarchy below a level of output data of a pre-determined shader stage, but wherein the cache is configured to not store shader stage output data for any level of the hierarchy above the level of the pre-determined shader stage output data (Howson, ¶0086, “FIG. 7 depicts an approach to tuning a level of coarseness in an acceleration structure that will be used to subdivide a 3-D scene into different portions, for the purpose of on-demand creation of final geometry of those different portions. A coarse acceleration structure may be defined based on a criteria of how much memory can be allocated to storing the coarse acceleration structure.” Fig .8, ¶0087, “such process also can involve producing portions of an acceleration structure” Fig.9, ¶0091, “FIG. 9 depicts an example memory map 382 of how vertex data and vertex connectivity data can be arranged in memory.” ¶0093, “For example, geometry production that is intended for consumption by a rasterization subsystem may clip geometry to a view frustrum of an image, or of a tile within the image, and/or cull backfacing primitives.” Fig. 10, 420, 424, ¶0097, “At 420, caching/cacheability characteristics for geometry and/or acceleration data produced are determined. Such determination can use indicia 424 of demand for such geometry and/or acceleration structure elements. Such determination also can use metadata collected at 418.” “These caching/cacheability characteristics may be effected by directing a cache coherency manager or cache replacement manager to expect a particular number of reads for a given set of data, and then to allow the data to be evicted.” Only portions of an acceleration structure are stored in cache. Also see Paltashev, ¶0022-0024.).
As to claim 15, claim 10 is incorporated and the combination of Howson, Mei, Paltashev and Ellis discloses the cache is configured to store sub-primitives derived from input graphics data items and wherein the rasterisation logic further comprises: a cache controller configured to: receive control stream data for a tile, retrieve sub-primitives which are stored in the cache and which are indicated by the control stream data for the tile, and provide the retrieved sub-primitives to one or more processing units to be rendered (Howson , Fig.1, Fig.2C, “FIG. 2C depicts a rasterization flow where screen space tiling 123 can be performed by beginning to process a tile (125) including accessing 2-D screen space object extent data (127) (e.g., an object list) and checking whether identified final geometry is cached (133) and to the extent not cached, providing identified source geometry and geometry process identification (129) to a geometry processing unit. On retrieved or recreated final geometry, HSR (visible surface identified) can be performed. In some implementations, even though certain final geometry is not visible within a tile, some of that geometry may be cached, because it may be used for ray traversal operations. In some situations, a cache requested flag may be set for certain final geometry, which would indicate demand on the ray traversal side for certain portions of final geometry (or vice versa).”).
As to claim 16, claim 15 is incorporated and the combination of Howson, Mei, Paltashev and Ellis discloses the one or more processing units comprise: a hidden surface removal unit configured to remove primitive fragments which are hidden; and a texturing/shading unit configured to apply one or both of texturing and shading to primitive fragments (Howson, Fig .2C, ¶0079, “the primitives may be passed to hidden surface removal unit 70, which removes any surfaces which are not visible in the tile, and the resulting pixel data may be passed to a texture and shading unit 80 which applies pixel or texture shading before the final pixel values for display are written to memory.”).
As to claim 17, claim 1 is incorporated and the combination of Howson, Mei, Paltashev and Ellis discloses the shader stage output data comprises vertex data output from a first shader stage and additional data from a further shader stage that is subsequent to the first shader stage (Howson, Fig .1, ¶0015, “the source geometry data comprising vertex data and two or more sets of vertex connectivity data.” ¶0039, “vertexes of primitives can be used as input to tessellation and to geometry shaders or displacement engines that can amplify a number of primitives by dicing a larger primitive into smaller ones, or otherwise deform geometry so that an extent of geometry after these further geometry operations is different from an original extent.” ¶0051, “Shader 37 can perform further programmable shading operations on geometry and produce vertices to be stored in vertex cache 41.”).
As to claim 18, claim 1 is incorporated and the combination of Howson, Mei, Paltashev and Ellis discloses the transformed geometry position data represents transformed positions of the input graphics data items in a rendering space (Howson, Fig1, ¶0056, “This unit 52 also may implement ray shaders 53. Ray shaders 63 may be triggered based on outputs of ray tracing unit(s) 59. Ray tracing units may identify closest intersections between final geometry and rays.” ¶0074, “The implementation of this geometry processing may be performed differently in different implementations. For example, where a programmable cluster of computation units is available to perform geometry computation, shading computation (e.g., shading of ray intersections, vertex shading, pixel shading, and so on), ray traversal, or other kinds of tasks, that programmable cluster can be configured to implement the geometry processing required. The implementation can run threads of computation that receive inputs and/or produce outputs from another thread in order to implement a sequence of geometry modifications required. In other examples, intermediately processed geometry data may be stored in memory and read/written as required by one or more threads of computation implementing the geometry processing.” ¶0075, “some geometry modifications may be incremental. So, even if it is can be indicated that a certain portion of an incremental series of geometry elements are located within a particular tile, it may not be possible to generate only those instances, because later instances may depend on iterating through earlier primitives in a programmatic sequence. In one implementation, unnecessarily repetition computation is reduced by storing incremental geometry data in a local memory, such as a cache. This has the advantage that, where the instances are spread over more than one tile, predecessor geometry elements may not need to be recreated for each tile. In such an implementation, it can be defined which portions of an incremental series of geometry are within a particular tile or 3-D acceleration structure element, and then those elements can be first looked for within a cache, before beginning a geometry process to reproduce those elements, which may require iterating through predecessor elements that are not within that tile or acceleration structure element.” ¶0064, “modified primitives may be generated incrementally, such that one modified primitive is created from the input primitive, and then a second modified primitive is created from that modified primitive and so on until a sequence of modified primitives has been derived. As such, in addition to storing an identifier identifying the primitive from which the sequence of modified primitives is derived, data indicating the first and/or last position in the sequence of those modified primitives located within the tile may be stored. This data may be stored in the object list for that tile or in another region of memory.”).
As to claim 19, the combination of Howson, Mei, Paltashev and Ellis discloses a graphics processing method comprising: a geometry processing phase which comprises: executing a plurality of shader stages including a final stage of the plurality of shader stages, on input graphics data items to determine transformed geometry position data, and writing to a memory, as an output of the geometry processing phase, shader stage output data for one or more shader stages of the plurality of shader stages not including the final shader stage of the plurality of shader stages used to determine the transformed geometry position data; and a rasterisation phase which comprises: fetching the shader stage output data from the memory; executing one or more shader stages including the final shader stage on the fetched shader stage output data to determine the transformed geometry position data and use the transformed geometry position data to generate a rendering output (See claim 1 for detailed analysis.).
As to claim 20, the combination of Howson, Mei, Paltashev and Ellis discloses a non-transitory computer readable storage medium having stored thereon a computer readable dataset description of an integrated circuit that, when processed in an integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture a graphics processing system which comprises: geometry processing logic configured to: in a geometry processing phase, execute a plurality of shader stages including a final stage of the plurality of shader stages, on input graphics data items to determine transformed geometry position data, and write to a memory, as an output of the geometry processing phase, shader stage output data for one or more shader stages of the plurality of shader stages not including a final shader stage of the plurality of shader stages used to determine the transformed geometry position data; and rasterisation logic configured to: in a rasterisation phase, fetch the shader stage output data from the memory; execute one or more shader stages including the final shader stage on the fetched shader stage output data determine transformed geometry position data and use the transformed geometry position data to generate a rendering output. (See claim 1 for detailed analysis.).
As to claim 21, claim 1 is incorporated and the combination of Howson, Mei, Paltashev and Ellis discloses the plurality of shader stages comprises a first shader stage and the rasterisation logic is configured to, in the rasterisation phase, execute one or more shader stages including the final shader stage and excluding the first shader stage on the shader stage output data to determine the transformed geometry position data and use the transformed geometry position data to generate a rendering output (Howson, Fig. 1, rasterization excludes the vertex shader and hull shader.
PNG
media_image4.png
332
299
media_image4.png
Greyscale
.
Mei, Fig. 4, rasterization excludes the vertex shader.
PNG
media_image5.png
544
747
media_image5.png
Greyscale
Paltashev, Fig. 1, rasterization excludes the vertex shader and hull shadder.
PNG
media_image6.png
559
666
media_image6.png
Greyscale
)
Claims 5 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Howson et al. (US Pub 2015/0317818 A1) in view of Mei et al. (US Pub 2015/0235341 A1), Paltashev (US Pub 2017/0372509), Ellis et al. (US Pub 2017/0206698 A1) and Fishwick et al. (US Pub 2014/0292782 A1).
As to claim 5, claim 1 is incorporated and Howson discloses the rasterisation logic is configured to generate the rendering output in a rendering space which is divided into a plurality of tiles (Howson, ¶0019-20).
Howson does not explicitly disclose the geometry processing logic is further configured to generate priorities for items of shader stage output data of a hierarchy of shader stage output data based on the number of tiles that the items of shader stage output data overlap.
Fishwick teaches generate priorities for items of shader stage output data of a hierarchy of shader stage output data based on the number of tiles that the items of shader stage output data overlap (Fishwick, ¶0013, ¶0051, ¶0056, “for each of the parameter blocks, a count is determined of the number of tiles that overlap with at least one parameter of the parameter block. These counts therefore provide an indication of the number of tiles with which the parameters of a parameter block are associated. In other words, the count for a parameter block indicates how many of the tiles will be processed using at least one parameter from the parameter block.” ¶0075, “Eviction policies described herein consider how many tiles a parameter block overlaps with. Parameters from a parameter block containing primitives which overlap with only one or a small number of tiles are less likely to be required again to process another tile than parameters from a parameter block which contains primitives that overlap with many tiles. This realisation leads to the eviction policies which weight the eviction of parameters of parameter blocks from the parameter cache 118 based on tile coverage.”).
Howson and Fishwick are considered to be analogous art because all pertain to image rendering. It would have been obvious before the effective filing date of the claimed invention to have modified Howson with the features of “generate priorities for items of shader stage output data of a hierarchy of shader stage output data based on the number of tiles that the items of shader stage output data overlap.” as taught by Fishwick. The suggestion/motivation would have been in order to have cache eviction policies consider how many tiles a parameter block overlaps with (Fiskwick, ¶0075)
Paltashev also teaches generate priorities for items of shader stage output data of a hierarchy of shader stage output data based on the number of tiles that the items of shader stage output data overlap (Paltashev, ¶0012, “the z coordinate range of the fragment is compared to the value of the z coordinate range stored in the hierarchical buffer for a group of tiles that overlaps the fragment.” ¶0022, “The z coordinate range of an object can be compared to the value of the z coordinate range stored in the hierarchical buffer 160 for a group of tiles that overlaps the object. The comparison indicates whether the object is in front of, behind, or in the same depth range as the previously rendered pixels or fragments.” ¶0029, “the level 302 may be selected because the object 320 overlaps with no more than four of the tiles 310, regardless of the relative positions of the object 320 and the tiles 310 in the plane of the screen. In contrast, the object 320 would overlap more than four of the tiles 315 in the level 301, which indicates that the granularity of the level 301 is finer than optimal for the object 320, and the object 320 would be completely encompassed by the tile 305 in the level 303, which indicates that the granularity of the level 303 is coarser than optimal for the object 320.”).
Howson and Paltashev are considered to be analogous art because all pertain to image rendering. It would have been obvious before the effective filing date of the claimed invention to have modified Howson with the features of “generate priorities for items of shader stage output data of a hierarchy of shader stage output data based on the number of tiles that the items of shader stage output data overlap” as taught by Paltashev. The suggestion/motivation would have been that primitives and other higher-order objects such as patches are culled in the geometry front-end or back-end of the graphics pipeline so that occluded objects are not emitted to the geometry back-end or pixel portion of the graphics pipeline (Paltashev, ¶0014.).
As to claim 14, claim 10 is incorporated and the combination of Howson, Mei, Paltashev and Fishwick discloses the rasterisation logic is configured to have a cache controller arranged to evict items of shader stage output data from the cache based on generated priorities based on the number of tiles that the items of shader stage output data overlap (Howson, ¶0097, “These caching/cacheability characteristics may be effected by directing a cache coherency manager or cache replacement manager to expect a particular number of reads for a given set of data, and then to allow the data to be evicted.” ¶0102, “A cache coherency controller 515 may consume caching hints and control which data is evicted from cache hierarchy 510 and which data is pre-fetched.” Fishwick, ¶0013, ¶0051, ¶0056, “for each of the parameter blocks, a count is determined of the number of tiles that overlap with at least one parameter of the parameter block. These counts therefore provide an indication of the number of tiles with which the parameters of a parameter block are associated. In other words, the count for a parameter block indicates how many of the tiles will be processed using at least one parameter from the parameter block.” ¶0075, “Eviction policies described herein consider how many tiles a parameter block overlaps with. Parameters from a parameter block containing primitives which overlap with only one or a small number of tiles are less likely to be required again to process another tile than parameters from a parameter block which contains primitives that overlap with many tiles. This realisation leads to the eviction policies which weight the eviction of parameters of parameter blocks from the parameter cache 118 based on tile coverage.”).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Howson et al. (US Pub 2015/0317818 A1) in view of Mei et al. (US Pub 2015/0235341 A1), Paltashev (US Pub 2017/0372509), Ellis et al. (US Pub 2017/0206698 A1) and McCombe (US Pub 2013/0113800)
As to claim 13, claim 10 is incorporated and the combination of Howson, Mei and Paltashev discloses the rasterisation logic is configured to retrieve shader stage output data from the hierarchy stored in the cache (Howson, ¶0024, “both rasterization and ray tracing subsystems” “memories for storing source geometry, and cache(s) for storing finalized geometry. These memories may be part of a memory hierarchy.” ¶0051, “Vertices stored in vertex cache 41 can serve as a basis for producing data to be used during rasterization (e.g., tile-based rasterization) and during ray tracing operations “ ¶0106, “a vertex shader or geometry shader (outputting polygons/geometric primitives) may have its output sent towards or enqueued to a hierarchy building process.” ¶0107, “Producers can enqueue outputs in a queue to be read by one or more consumers; collections can be formed based on enqueued outputs.”).
Howson does not explicitly disclose a bottom-up manner.
However, a bottom-up retrieval in a tree structure are obvious to one of ordinary skill in the art.
McCombe teaches a bottom-up acceleration structure (McCombe, ¶0051, ¶0095, “FIG. 17 provides an example of bottom-up photon acceleration structure building.” ¶0099, “FIG. 17 depicted that photons were processed in a bottom-up fashion, where larger and larger volumes that abstract larger regions of 3-D space are created in multiple passes.”)
Howson and McCombe are considered to be analogous art because all pertain to image rendering. It would have been obvious before the effective filing date of the claimed invention to have modified Howson with the features of “a bottom-up manner.” as taught by McCombe. The suggestion/motivation would have been in order to arrive at a balanced system, with consideration of acceleration structure build time and ray traversal time (McCombe, ¶0050).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YU CHEN whose telephone number is (571)270-7951. The examiner can normally be reached on M-F 8-5 PST Mid-day flex.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached on 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YU CHEN/Primary Examiner, Art Unit 2613
Prosecution Insights