DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment / Arguments
Claim Objections. Applicant’s amendment overcomes the objection to the claims.
Double Patenting. Applicant’s Terminal Disclaimer overcomes the nonstatutory double patenting rejection.
103 Rejections. Applicant’s arguments (no amendment) have been considered and are unpersuasive. The 103 rejections are maintained.
Applicant’s arguments that Molar does not teach the “performing a pre-pass phase of rendering on the plurality of pieces of geometry, based on the assigning….to generate information regarding overlap of each of the plurality of pieces of geometry regarding overlap…to a plurality of screen regions” is unpersuasive. Firstly, Applicant’s arguments, especially to the Molar reference, do not specifically address any of the specific mappings in the Office Action, and do not point out any specific errors in the examiner’s rationale or mapping.
Secondly, to the substance of Applicant’s arguments. Applicant’s arguments against the Molar reference can be found at page 9 of the Remarks, and are reproduced below for convenience. The examiner’s response follows.
PNG
media_image1.png
286
744
media_image1.png
Greyscale
Applicant’s Arguments Against Molar Reference
As shown above, first Applicant makes a broad alleged statement (without any citation to Molnar), that “Molnar’s system takes shapes and runs a mathematical bounding-box calculation on the CPU”. The examiner disagrees, and cited to portions of Molar that do teach Applicant’s claimed features (which Applicant did not address).
Then, Applicant cites to one partial sentence of paragraph [0042] of Applicant’s specification as filed to, presumably, support or describe Applicant’s claim 1. Unfortunately, Applicant’s quote from [0042] actually does not relate to what is recited in claim 1 and therefore is irrelevant. This portion of [0042], from Applicant, when read carefully, refers to a parallel rendering and geometry analysis (this isn’t in claim 1 or the independent claims), and also refers to a “Z pre-pass” which also isn’t in claim 1, and “geometry rendering” (also not in claim 1). Therefore, Applicant’s citation to the partial sentence in paragraph [0042] of Applicant’s specification is not related to what Applicant is claiming and, thus, irrelevant to support any understanding of Applicant’s claim 1.
Then, Applicant ends her arguments by admitting that “Molnar merely performs a sorting calculation on raw primitives before any rendering takes place”. This admission by Applicant is also not persuasive and, respectfully, tends to work in favor of the rejection, as the “sorting calculation” of Molar could actually also teach information that is generated regarding overlap, per Applicant’s claim 1.
Finally, to the performing pre-pass phase of rendering and the prior art, the examiner points to page 26 of Molar, and even the portions specifically put in the last office action and again, in this one, and also to please look carefully at what is recited in Applicant’s claim 1. In the pre-pass phase of rendering, information regarding overlap of each of the pieces of geometry to the screen regions is generated. This is in claim 1. Molar teaches generating information that includes, but is not limited to, an overlap factor which is the average number of regions a raw primitive overlaps. This was specifically in the Non-Final and now Final Rejection. Molar’s overlap factor is one example of “information regarding overlap” that is generated in a pre-pass phase of rendering, that happens before rendering occurs.
The 103 Rejections are maintained. Please see remainder of this Office Action for details.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 4, 6-9, 11, 13-16, 18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Molnar, S., Cox, M., Ellsworth, D., & Fuchs, H. (1994). A sorting classification of parallel rendering. IEEE computer graphics and applications, 14(4), 23-32 (“Molnar”) in view of Tolo, Lars Olav. "Multi-GPU Rendering with Vulkan API." Master's thesis, The University of Bergen, 2018 (“Tolo”) (both refs cited in parent).
Regarding claim 1:
Molnar teaches: a method ((Molnar, page 23, parallel rendering) (Tolo, Abstract, method of multi-GPU rendering), comprising: assigning a plurality of pieces of geometry of an image frame (Molnar, page 24, “Sort-first” section, which taches initially assigning primitive to renderers in some arbitrary fashion) to a plurality of GPUs (Tolo, Abstract, teaches multi-GPU rendering. Modifying the applied references, such that the “renderers” of Molar are GPUs, as per Tolo, both references related to multi-processor rendering, would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A));
performing a pre-pass phase of rendering on the plurality of pieces of geometry, based on the assigning of the plurality of pieces of geometry, to generate information regarding overlap of each of the plurality of pieces of geometry to a plurality of screen regions (Molnar teaches determining “Processing and communication costs” corresponding to Applicant’s claimed “information regarding overlap of [primitives/geometry pieces] to screen regions”. See page 26, “Processing and communication costs”, which reads, in part:
The first steps in sort-first are to "pre-transform" the raw primitives so that their screen extents are known and to classify them with respect to screen-region buckets. Each bucket belongs to a region, and a primitive falls in several buckets when it overlaps several regions.
We define an overlap factor 0,.which is the average number of regions a raw primitive overlaps. If the cost to precompute a primitives screen coordinates is •<pre-xform» and the cost to put the primitive in each of its buckets is «bucket,», then the overhead for these stages is n, «prexform» and n,U, «bucket,». Next, primitives on the wrong renderer must be distributed to the correct renderer(s).
See also Molnar, Fig. 6, reproduced below for convenience, and also page 29: “Estimating primitive overlap”. The costs or “SF [sort first] overhead” of “pre-transform” and “bucketization”, at a minimum, teach Applicant’s claimed information);
PNG
media_image2.png
200
400
media_image2.png
Greyscale
assigning the plurality of screen regions to the plurality of GPUs based on the information (Molnar, pages 26-27 (up to “Sort-middle analysis”), the redistribution after screen regions are determined teaches assigning the screen regions to renderers/GPUs; at page 27, second column, where it is also thought make smaller regions, and “make each processor responsible for more than one region”, to help with balancing loads); and
during a geometry pass phase of rendering, rendering the plurality of pieces of geometry by the plurality of GPUs based on the assigning of the plurality of screen regions (Molnar, Fig. 6, reproduced above, geometry pass of pre and post tessellation is performed after the redistribution).
Likewise, Tolo also teaches multiple approaches to parallel rendering and distributing work amongst multi-GPU configurations, one of which can be a “sort-first” approach, like the one of Molnar (e.g. p. 19 and Ch. 2, particularly 2.2-2.3; and page 61, “Sort-first Screen Partitioning”).
It would have been obvious for one of ordinary skill in the art to have combined and modified the applied reference(-s), in view of same, to have obtained the above, and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
The prior art included each element recited in claim 1, although not necessarily in a single embodiment, with the only difference being between the claimed element and the prior art being the lack of actual combination of certain elements in a single prior art embodiment, as described above.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 2:
Molnar teaches: the method of claim 1, further comprising: determining a plurality of computing costs for performing the geometry pass phase of rendering on the plurality of pieces of geometry (Fig. 6 and pages 26-27, “Processing and communication costs”, costs for performing geometry phase of rendering is computed/determined. See also page 29, primitive overlap costs and Fig. 9),
wherein the assigning of the plurality of screen regions to the plurality of GPUs is based on the plurality of computing costs (Id. the redistribution/assigning of screen regions to renderers/GPUs is done based on costs).
It would have been obvious for one of ordinary skill in the art, as of the effective filing date of Applicant’s claims, to have further modified the applied reference(-s), in view of Molnar, to have obtained the above, motivated to perform efficient and informed load balancing.
Regarding claim 4:
It would have been obvious for one of ordinary skill in the art to have combined and modified the applied reference(-s), in view of same, to have obtained: the method of claim 2, further comprising: traversing rendering command buffer a plurality of times when performing the geometry pass phase of rendering,
wherein one or more screen regions having a higher cost are rendered during a first traversal, and remaining screen regions are rendered during a subsequent traversal, and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
Tolo teaches that command buffers can be traversed multiple times (see Fig. 2.7 and related description, which illustrates command buffer traversal during geometry pass rendering, as mapped in claim 1, or see Tolo, 2.1.The Graphics Pipeline). Re: screen regions having a higher cost being rendering during a first traversal, and remaining during a subsequent traversal, Tolo also teaches to create queues to execute commands from command buffers (see Section 2.5.2 to page 29), and also teaches that commands can be dependent on each other and, as a result, dependent commands cannot be executed in parallel (see Section 2.5.7, first paragraph).
A screen region having a higher cost can be a screen region that has a majority of a primitive that is shared with another screen region, where only a small portion of said primitive falls in the “another screen region”. For example, say there are two regions, to be rendered by two GPUs (GPU A and GPU B), as illustrated below:
[AltContent: rect][AltContent: connector][AltContent: connector][AltContent: oval]
GPU A GPU B
The command to render the oval shape corresponds to a command dependency per Tolo, one that cannot be executed in parallel. So, in this instance, rendering of GPU A region (the higher cost) would be rendered first, and in a subsequent traversal, the region of GPU B. This teaches the traversal order per Applicant’s claim 4.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 6:
Tolo teaches: the method of claim 1, further comprising: traversing a rendering command buffer by the plurality of GPUs when performing the geometry pass phase of rendering,
wherein a rendering order of the plurality of pieces of geometry does not match an order of draw calls in the rendering command buffer (see section 2.5.7, “Commands recorded in a command buffer are not necessarily executed sequentially on the GPU”, which teaches/ suggests a rendering order that does not match an order of draw calls in the rendering command buffer. See also page 15, removal of primitives in the pipeline is also taught, which would also result in the above misordering of draw calls to rendering).
It would have been obvious for one of ordinary skill in the art, as of the effective filing date of Applicant’s claims, to have further modified the applied reference(-s), in view of Molnar, to have obtained the above, motivated to have flexibility with regard to rendering.
Regarding claim 7:
It would have been obvious for one of ordinary skill in the art to have further modified the applied reference(-s), in view of same, to have obtained: the method of claim 1, wherein the assigning the plurality of pieces of geometry to the plurality of GPUs includes: defining an initial assignment of the plurality of screen regions to the plurality of GPUs for performing the pre-pass phase of rendering (see Molnar page 26, the “initial assignment of screen regions” can be that for the frame previous to the next frame being rendered),
wherein the assigning the plurality of screen regions to the plurality of GPUs based on the information defines a subsequent assignment of the plurality of screen regions (the screen region assignment can change for the next frame. See Molnar, pages 26-27),
and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
That is to say, modifying the applied references, in view of same, such to have a reassignment or subsequent assignment of screen regions, for multi-frame rendering, teaches Applicant’s claim 7, and is also taught/suggested and obvious over the prior art, as mapped above.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 8: see also claim 1.
Tolo teaches: a computer system (page 23, computer) comprising: a processor (page 23, computer has a processor, such as GPU; see also Abstract); memory coupled to the processor (page 25, memory) and having stored therein instructions that, if executed by the computer system (e.g. page 18, 24, software executable), cause the computer system to execute a method, comprising:
The method of claim 8 corresponds to that of claim 1; the same rationale for rejection applies.
Regarding claim 9: see claim 2.
These claims are similar; the same rationale for rejection applies.
Regarding claim 11: see claim 4.
These claims are similar; the same rationale for rejection applies.
Regarding claim 13: see claim 6.
These claims are similar; the same rationale for rejection applies.
Regarding claim 14: see claim 7.
These claims are similar; the same rationale for rejection applies.
Regarding claim 15: see also claim 1.
a non-transitory computer-readable medium storing a computer program for implementing a method, the computer-readable medium comprising:
The method of claim 15 corresponds to that of claim 1; the same rationale for rejection applies.
Regarding claim 16: see claim 2.
These claims are similar; the same rationale for rejection applies.
Regarding claim 20: see claim 7.
These claims are similar; the same rationale for rejection applies.
Claim(s) 3, 10 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Molnar in view of Tolo, and further in view of Bakalash (U.S. Patent App. Pub. No. 2008/0165197 A1).
Regarding claim 3:
It would have been obvious for one of ordinary skill in the art to have combined and modified the applied reference(-s), in view of same, to have obtained: the method of claim 2, wherein the plurality of screen regions is assigned to the plurality of GPUs such that corresponding costs for the plurality of screen regions is approximately equal, and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
Bakalash teaches that it is known, in multi-GPU parallel rendering, to use load balancing considerations to use “load balancing considerations, to keep the overall GPU load evenly balanced.” (see para. 63). Molnar teaches several load balancing consideration, such as costs and size of assigned regions (pages 26-27, 29). Modifying the applied references, in view of same, such to use costs for screen regions, per Molnar, as one ‘load balancing consideration’, per Bakalash, to evenly balance the load between GPUs (i.e. approximately equal), is all of taught and suggested by the prior art, and would have been obvious and predictable to one of ordinary skill.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 10: see claim 3.
These claims are similar; the same rationale for rejection applies.
Regarding claim 17: see claim 3.
These claims are similar; the same rationale for rejection applies.
Regarding claim 18: see claim 4.
These claims are similar; the same rationale for rejection applies.
Claim(s) 5, 12 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Molnar in view of Tolo, and further in view of Hakura (U.S. Patent Application Publication No. 2017/0148203 A1) (cited in parent).
Regarding claim 5:
The applied references to claim 1 do not proactively teach claim 5. Consider the following.
In analogous art, Hakura teaches: the method of claim 1, wherein the pre-pass phase of rendering is a Z pre-pass phase of rendering (see e.g. para. 4, which also teaches the benefit of performing a Z pre-pass phase to determine the existence of occluded pixels that can be ignored during subsequent shading).
It would have been obvious for one of ordinary skill in the art to have further modified the applied references, in view of Hakura, to have included the above, motivated to determine accurate rendering costs, per Molnar, which would be impacted by visibility of pixels.
Regarding claim 12: see claim 5.
These claims are similar; the same rationale for rejection applies.
Regarding claim 19: see claim 5.
These claims are similar; the same rationale for rejection applies.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
* * * * *
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Sarah Lhymn whose telephone number is (571)270-0632. The examiner can normally be reached M-F, 9:00 AM to 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Sarah Lhymn
Primary Examiner
Art Unit 2613
/Sarah Lhymn/Primary Examiner, Art Unit 2613
Prosecution Insights