DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner Notes
Examiner cites particular columns and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
The examiner encourages Applicant to submit an authorization to communicate with the examiner via the Internet by making the following statement (from MPEP 502.03):
“Recognizing that Internet communications are not secure, I hereby authorize the USPTO to communicate with the undersigned and practitioners in accordance with 37 CFR 1.33 and 37 CFR 1.34 concerning any subject matter of this application by video conferencing, instant messaging, or electronic mail. I understand that a copy of these communications will be made of record in the application file.”
Please note that the above statement can only be submitted via Central Fax, Regular postal mail, or EFS Web (PTO/SB/439).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. § 101 because the claimed invention is directed to non-statutory subject matter.
Independent claim 20 recites a “computer-readable medium,” which the specification states may be “any available media.” (Applicant’s Specification, [0025], Computer-readable media includes computer storage media. Storage media may be any available media that may be accessed by a computer. By way of example, and not limitation, such computer-readable media may comprise a random access memory (RAM), a read-only memory (ROM), an electrically erasable programmable ROM (EEPROM), optical disk storage, magnetic disk storage, other magnetic storage devices, combinations of the aforementioned types of computer-readable media, or any other medium that may be used to store computer executable code in the form of instructions or data structures that may be accessed by a computer). The broadest reasonable interpretation of a claim drawn to a computer-readable medium covers forms of transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media. Transitory propagating signals are non-statutory subject matter. In re Nuijten, 500 F.3d 1346, 1356-57, 84 U.S.P.Q.2d 1495, 1502 (Fed. Cir. 2007) (transitory embodiments are not directed to statutory subject matter). See also Subject Matter Eligibility of Computer Readable Media, 1351 Off. Gaz. Pat. Office 212 (Feb. 23, 2010). Examiner suggests adding the word “non-transitory.”
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 7-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kondguli et al. (US 2021/0366188 A1) in view of Baize et al. (US 11676328).
As per claim 1, Kondguli teaches the invention substantially as claimed including an apparatus for graphics processing, comprising:
at least one memory ([0043], system 700 may include ... a memory 703); and
at least one processor coupled to the at least one memory ([0043], system 700 may include the GPU 701, a central processing unit (CPU) 702 and a memory 703) and, based at least in part on information stored in the at least one memory, the at least one processor, individually or in any combination, is configured to:
obtain an indication of a plurality of workloads for the graphics processing ([0003], a binning architecture may use a batch-based binning process in which the binner distributes primitives across batches and bins the primitives of each batch separately instead of binning an entire set of primitives at one time), wherein the plurality of workloads corresponds to a workload order for a workload submission sequence ([0024], each batch of primitives touches a subset of the tiles forming the screen space. The tiles may be rendered by a backend graphics hardware pipeline in a specific order (linear order, Morton order, hierarchical Morton order, etc.) and in a monotonic direction (e.g., all batches start from the top-left tile and proceed to the bottom right-tile));
perform a binning process for a first workload of the plurality of workloads, wherein the first workload is first in the workload order ([0003], the binner distributes primitives across batches and bins the primitives of each batch separately instead of binning an entire set of primitives at one time; and [0030],batch-based primitive-rendering pipeline may perform primitive-to-tile mapping (binning) in a batch-by-batch manner);
divide the first workload into a plurality of first sub-workloads and a second workload of the plurality of workloads into a plurality of second sub-workloads, wherein the second workload is second in the workload order ([0003], a binning architecture may use a batch-based binning process in which the binner distributes primitives across batches and bins the primitives of each batch separately instead of binning an entire set of primitives at one time. The size of each batch may be determined at runtime based on various resource constraints and/or Application Programming Interface (API) constraints);
perform, upon completion of the binning process for the first workload, a rendering process for at least one first sub-workload of the plurality of first sub-workloads and a binning process for at least one second sub-workload of the plurality of second sub-workloads ([0035], the Previous Order is updated as tiles of the new batch are pushed to backend processing to represent a new MR to LR order. At 407, the process returns to 403 for the next new batch. Process 400 continues until rendering of the screen space is complete).
Kondguli fails to specifically teach, perform a rendering process for the at least one second sub-workload prior to a completion of the rendering process for the at least one first sub-workload.
However, Baize teaches, perform a rendering process for the at least one second sub-workload prior to a completion of the rendering process for the at least one first sub-workload (Column 2, Lines 24-25, performing an out-of-order rendering and streaming of image data based on a risk map).
Kondguli and Baize are analogous because they are each related to graphics processing. Kondguli teaches a method for out-of-order batched-based graphics processing. ([0025], The subject matter disclosed herein changes the order, or sequence, in which a batch processes a screen space, thereby reducing DRAM traffic; and [0028], The DRAM traffic reduction also improves performance of the GPU during time intervals that would otherwise be bottlenecked by memory bandwidth, which improves the overall performance of the GPU. The DRAM traffic reduction also helps reduce power consumption by, for example, an SOC embodying a GPU and, in turn, provides an improved a power budget for the GPU). Baize teaches a method of out-of-order graphics rendering. (Abstract, system then performs an out-of-order rendering of primitives that fall within the second view frustum based on the risk value for each first image pixel that is replaced in a second image with a rendered primitive from the second view frustum). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention that based on the combination, the GPU of Kondguli would be modified with the out-of-order rendering mechanism taught by Baize resulting in a system that processes and renders graphics workloads in an out-of-order fashion. Therefore, it would have been obvious to combine the teachings of Kondguli and Baize.
As per claim 2, Kondguli teaches, wherein the plurality of first sub-workloads corresponds to a plurality of first primitive batches and the plurality of second sub-workloads corresponds to a plurality of second primitive batches ([0030], A batch-based primitive-rendering pipeline may perform primitive-to-tile mapping (binning) in a batch-by-batch manner. Batches may include different sets of primitives).
As per claim 7, Kondguli teaches, wherein the at least one processor, individually or in any combination, is further configured to:
execute a memory read process for the at least one first sub-workload prior to the performance of the rendering process for the at least one first sub-workload ([0027], the subject matter disclosed herein may reverse tile-processing order across consecutive batches so that a screen-space tile that is present in a cache is processed first and additional DRAM traffic is not incurred).
As per claim 8, Kondguli teaches, wherein the at least one processor, individually or in any combination, is further configured to:
execute the memory read process for the at least one second sub-workload prior to the completion of the rendering process for the at least one first sub-workload and prior to the performance of the rendering process for the at least one second sub-workload ([0027], the subject matter disclosed herein may reverse tile-processing order across consecutive batches so that a screen-space tile that is present in a cache is processed first and additional DRAM traffic is not incurred).
As per claim 9, Kondguli teaches, wherein to execute the memory read process for the at least one first sub-workload, the at least one processor, individually or in any combination, is configured to retrieve data for the at least one first sub-workload from a memory ([0024], the on-chip and/or off-chip dynamic random access memory (DRAM) corresponding to a screen space may be re-referenced by primitives across batches; and [0027], reverse tile-processing order across consecutive batches so that a screen-space tile that is present in a cache is processed first and additional DRAM traffic is not incurred); and
wherein to execute the memory read process for the at least one second sub-workload, the at least one processor, individually or in any combination, is configured to retrieve data for the at least one second sub-workload from the memory ([0024], the on-chip and/or off-chip dynamic random access memory (DRAM) corresponding to a screen space may be re-referenced by primitives across batches; and [0027], reverse tile-processing order across consecutive batches so that a screen-space tile that is present in a cache is processed first and additional DRAM traffic is not incurred).
As per claim 10, Kondguli teaches, wherein the rendering process for the at least one second sub-workload is started prior to a completion of the memory read process for the at least one second sub-workload ([0027], the subject matter disclosed herein may reverse tile-processing order across consecutive batches so that a screen-space tile that is present in a cache is processed first and additional DRAM traffic is not incurred).
As per claim 11, Kondguli teaches, wherein the at least one processor, individually or in any combination, is further configured to:
allocate the plurality of first sub-workloads and the plurality of second sub-workloads to a preconfigured storage after the division of the first workload and the second workload ([0024], Consecutive batches frequently access and process the same screen space because the primitives occupying the same screen space may often be split across multiple batches. Thus, the on-chip and/or off-chip dynamic random access memory (DRAM) corresponding to a screen space may be re-referenced by primitives across batches... By processing the same screen-space tiles simultaneously or in close temporal proximity, the data necessary for processing the primitives of a batch would be resident in on-chip or off-chip caches; and [0025], very new batch might process the screen-space tiles first that were most recently processed by the preceding batch to take advantage of any on-chip and/or off-chip caches that may have been tuned to efficiently service traffic patterns having good temporal locality).
As per claim 12, Kondguli teaches, wherein the preconfigured storage is an on-chip storage at a graphics processing unit (GPU) ([0043], system 700 that includes a GPU 701 that that changes a binning processing order to reduce DRAM traffic according to the subject matter disclosed herein).
As per claim 13, Kondguli teaches, wherein the performance of the rendering process for the at least one first sub-workload and the performance of the binning process for the at least one second sub-workload are simultaneous ([0025], the subject matter disclosed herein uses temporal data locality information that may be available in connection with a binning operation to change the order in which screen-tile spaces are processed for consecutive batches; and [0029], provides a re-reference aware tile-processing order that may modify a tile-processing order and/or a tile-processing direction across batches so that the same screen-space tiles (across batches) may be processed simultaneously as much as possible or in close temporal proximity with a minimal change in GPU architecture).
As per claim 14, Kondguli teaches, wherein the performance of the rendering process for the at least one first sub-workload and the performance of the binning process for the at least one second sub-workload are in parallel ([0029], provides a re-reference aware tile-processing order that may modify a tile-processing order and/or a tile-processing direction across batches so that the same screen-space tiles (across batches) may be processed simultaneously as much as possible or in close temporal proximity with a minimal change in GPU architecture. A first few and a last few of the tiles of simultaneous batches may experience locality, and in its simplest form, the subject matter disclosed herein may change tile-processing direction across simultaneous batches, and color and depth cache locality may be captured for temporally local tiles).
As per claim 15, Kondguli teaches, wherein the completion of the rendering process for the at least one first sub-workload is prior to a completion of the rendering process for the at least one second sub-workload ([0025], the subject matter disclosed herein uses temporal data locality information that may be available in connection with a binning operation to change the order in which screen-tile spaces are processed for consecutive batches; and [0029], provides a re-reference aware tile-processing order that may modify a tile-processing order and/or a tile-processing direction across batches so that the same screen-space tiles (across batches) may be processed simultaneously as much as possible or in close temporal proximity with a minimal change in GPU architecture).
As per claim 16, Baize teaches, wherein the at least one processor, individually or in any combination, is further configured to:
output an indication of the performance of the rendering process for the at least one second sub-workload prior to the completion of the rendering process for the at least one first sub-workload (Column 7, Lines 19-25, Performing (at 318) the out-of-order prioritized rendering and streaming includes updating the risk map by computing the risk that is associated with pixels of last generated scene, selecting data points within the updated view frustum based on the updated risk map, and rendering and streaming the data points in the order that they are selected based on the updated risk map).
As per claim 17, Baize teaches, wherein to output the indication of the performance of the rendering process for the at least one second sub-workload, the at least one processor, individually or in any combination, is configured to:
transmit the indication of the performance of the rendering process for the at least one second sub-workload; or
store the indication of the performance of the rendering process for the at least one second sub-workload (Column 7, Lines 19-25, Performing (at 318) the out-of-order prioritized rendering and streaming includes updating the risk map by computing the risk that is associated with pixels of last generated scene, selecting data points within the updated view frustum based on the updated risk map, and rendering and streaming the data points in the order that they are selected based on the updated risk map).
As per claim 18, Kondguli teaches, wherein the binning process for the first workload is a sorting process for the first workload and the binning process for the second workload is the sorting process for the second workload ([0025], uses temporal data locality information that may be available in connection with a binning operation to change the order in which screen-tile spaces are processed for consecutive batches); and
wherein the division of the first workload into the plurality of first sub-workloads and the division of the second workload into the plurality of second sub-workloads is performed at a graphics processing unit (GPU) ([0002], a graphics processing unit (GPU) that changes a binning processing order to reduce DRAM traffic; and [0003], a binning architecture may use a batch-based binning process in which the binner distributes primitives across batches and bins the primitives of each batch separately instead of binning an entire set of primitives at one time. The size of each batch may be determined at runtime based on various resource constraints and/or Application Programming Interface (API) constraints).
As per claim 19, this is the “method claim” corresponding to claim 1 and is rejected for the same reasons. The same motivation used in the rejection of claim 1 is applicable to the instant claim.
As per claim 20, this is the “computer-readable medium claim” corresponding to claim 1 and is rejected for the same reasons. The same motivation used in the rejection of claim 1 is applicable to the instant claim.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Kondguli-Baize as applied to claim 1 and in further view of Achrenius et al. (US 20210398349 A1).
As per claim 3, the combination of Kondguli-Baize fails to specifically teach, wherein the at least one processor, individually or in any combination, is further configured to: divide each of the plurality of first primitive batches into a plurality of first primitive sub-batches and each of the plurality of second primitive batches into a plurality of second primitive sub-batches.
However, Achrenius teaches, wherein the at least one processor, individually or in any combination, is further configured to:
divide each of the plurality of first primitive batches into a plurality of first primitive sub-batches and each of the plurality of second primitive batches into a plurality of second primitive sub-batches ([0015], For at least one batch of the plurality of batches the rendering includes, for each of the plurality of bins, rendering primitives of a first sub-batch rasterizing to that bin, and for each of the plurality of bins, rendering primitives of a second sub-batch rasterizing to that bin).
The combination of Kondguli-Baize and Achrenius are analogous because they are each related to graphics processing. Kondguli teaches a method for out-of-order batched-based graphics processing. Baize teaches a method of out-of-order graphics rendering. Achrenius teaches a method of out-of-order rendering that subdivides batches for processing. (Abstract, rendering a plurality of primitives of a frame, the plurality of primitives being divided into a plurality of batches of primitives and the frame being divided into a plurality of bins. For at least one batch of the plurality of batches the rendering includes, for each of the plurality of bins, rendering primitives of a first sub-batch rasterizing to that bin, and for each of the plurality of bins, rendering primitives of a second sub-batch rasterizing to that bin). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention that based on the combination, the GPU rendering process taught by the combination of Kondguli-Baize would be modified with the batch dividing mechanism taught by Achrenius resulting in a system that processes and renders graphics workloads in an out-of-order fashion. Therefore, it would have been obvious to combine the teachings of the combination of Kondguli-Baize and Achrenius.
Claims 4-6 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Kondguli-Baize-Achrenius as applied to dependent claim 3 and in further view of Street (US 20090141033 A1).
As per claim 4, the combination of Kondguli-Baize-Achrenius fails to specifically teach, wherein the at least one processor, individually or in any combination, is further configured to: combine each of the plurality of first primitive sub-batches into a group of first vertices and each of the plurality of second primitive sub-batches into a group of second vertices, wherein the group of first vertices is associated with a first granularity and the group of second vertices is associated with a second granularity.
However, Street teaches, wherein the at least one processor, individually or in any combination, is further configured to:
combine each of the plurality of first primitive sub-batches into a group of first vertices and each of the plurality of second primitive sub-batches into a group of second vertices ([0079], granularity variable indicates a number of vertices associated with command 900 that comprise a vertex group, or subset of the vertices associated with command 900), wherein the group of first vertices is associated with a first granularity and the group of second vertices is associated with a second granularity ([0079], In a case that command 900 processes vertices, the granularity instructs the SPIM 208A to provide feedback upon completion of a number of the vertices, e.g., a vertex group, which is a subset of the vertices associated with command 900. Using the granularity value, the SPIM 208A can provide feedback as each vertex group is processed, which feedback can be in addition to the feedback provided by SPIM 208A when the secondary processor completes processing all of the vertices associated with command 900, thus providing additional, and/or a finer level of, feedback).
.
The combination of Kondguli-Baize-Achrenius and Street are analogous because they are each related to graphics processing. Kondguli teaches a method for out-of-order batched-based graphics processing. Baize teaches a method of out-of-order graphics rendering. Achrenius teaches a method of out-of-order rendering that subdivides batches for processing. Street teaches a method for graphics rendering using groups of vertices that are each associated with a granularity. (Abstract, a command stream, which can comprise one or more commands, is provided to the secondary processing unit, so that the unit can locate and operate on buffered graphics pipeline data. Secondary processing is managed and monitored so as to synchronize data access by the secondary processing unit with the graphics pipeline processing modules; [0034], GPU driver 114 provides GPU 104 with a processing configuration, which GPU 104 uses to render the batch of graphics primitives. GPU 104 renders the batch of graphics primitives, and outputs a raster image of the graphics primitives; and [0079], granularity variable indicates a number of vertices associated with command 900 that comprise a vertex group, or subset of the vertices associated with command 900. In accordance with one or more embodiments, the secondary processor outputs information, as feedback, to indicate that a number of vertices that comprise a vertex group have been processed. In a case that command 900 processes vertices, the granularity instructs the SPIM 208A to provide feedback upon completion of a number of the vertices, e.g., a vertex group, which is a subset of the vertices associated with command 900. Using the granularity value, the SPIM 208A can provide feedback as each vertex group is processed). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention that based on the combination, the GPU rendering process taught by the combination of Kondguli-Baize-Achrenius would be modified with the vertex processing mechanism taught by Street resulting in a GPU capable of out-of-order graphics processing using groups of vertices. Therefore, it would have been obvious to combine the teachings of the combination of Kondguli-Baize-Achrenius and Street.
As per claim 5, Street teaches, wherein the at least one processor, individually or in any combination, is further configured to:
execute, at a shader processor (SP) of a graphics processing unit (GPU), the group of first vertices at the first granularity and the group of second vertices at the second granularity ([0004], vertex shader is applied to the image geometry for an image and generates vertex coordinates and attributes of vertices within the image geometry; and [0079], the granularity instructs the SPIM 208A to provide feedback upon completion of a number of the vertices, e.g., a vertex group, which is a subset of the vertices associated with command 900. Using the granularity value, the SPIM 208A can provide feedback as each vertex group is processed, which feedback can be in addition to the feedback provided by SPIM 208A when the secondary processor completes processing all of the vertices associated with command 900, thus providing additional, and/or a finer level of, feedback).
As per claim 6, the combination of Kondguli-Baize-Achrenius fails to specifically teach wherein the plurality of first primitive sub-batches corresponds to at least one first vertex attribute and the plurality of second primitive sub-batches corresponds to at least one second vertex attribute.
However, Street teaches, wherein the plurality of first primitive sub-batches corresponds to at least one first vertex attribute and the plurality of second primitive sub-batches corresponds to at least one second vertex attribute ([0079], granularity variable indicates a number of vertices associated with command 900 that comprise a vertex group, or subset of the vertices associated with command 900).
The same motivation used in the rejection of claim 4 is applicable to the instant claim.
Conclusion
Burke et al. (US 2018/0300933 A1) : Teaches out-of-order graphics rendering:
[0169], utilizing more than two engines to perform the out-of-order rendering. For example, a GPU may include multiple engines or two or more GPUs may be linked together. The out-of-order mode may break apart the requirement to perform synchronization between the two or more engines. In the out-of-order mode, the driver and/or GPU may be free to issue work without synchronization (except when resolving out of order work) and may co-issue work distributed across multiple parallel processing devices which otherwise would require synchronization. This may allow for multiple GPUs to work on the same frame without internally synchronizing application resources. For example, the work re-orderer may distribute the work across the GPUs/engines without having to maintain special registers, synchronization information, or other order dependency detection (e.g. because the application guarantees that it can deal with the out-of-order processing
Bozier (US 2016/0148082 A1)-teaches out-of-order rendering:
[0186], tiles may be rendered out-of-order as defined by the document (e.g., 1001). For example, some output systems where the delivery of out-of-order tiles does not affect the performance of the output system may include: [0187] (i) a system where the rendered tiles are accumulated in a full-page frame buffer, and where tile content is copied to an appropriate place in the full-page frame buffer; [0188] (ii) a system where the rendered tiles are stored in a tiled image format that supports out of order storage of tiled data, such as the TIFF image format; and [0189] (iii) a system where rendered tiles are sent over a network connection to a remote system, where the expense of re-ordering the tiles is less than the latency in transferring the data;
Nikam et al. (US 2022/0327654 A1)- Teaches batch-based rendering:
[0097], the apparatus may divide, based on the determined size of each of the plurality of primitive batches, the plurality of primitives into the plurality of primitive batches, as described in connection with the examples in FIGS. 1-11. For example, GPU pipeline 1102 may divide, based on the determined size of each of the plurality of primitive batches, the plurality of primitives into the plurality of primitive batches. Further, processing unit 120 may perform 1208; and
Sikachev et al. (US 20200143589 A1)
Abstract, graphics processing method, which comprises providing an original set of vertices of a terrain mesh; producing a new set of vertices from the original set of vertices; and, for a given vertex in the original set of vertices: (i) obtaining texture coordinates for vertices in a subset of vertices in the new set of vertices that corresponds to the given vertex in the original set of vertices; and (ii) combining the obtained texture coordinates to obtain a texture coordinate for the given vertex in the original set of vertices; and
[0045], revised position calculation in parallel for sub-groups (or “batches”) of vertices in the original set of vertices 41.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MELISSA A HEADLY whose telephone number is (571)272-1972. The examiner can normally be reached Monday- Friday 9-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bradley Teets can be reached at 571-272-3338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MELISSA A HEADLY/Examiner, Art Unit 2197
Prosecution Insights