DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment / Arguments
Double Patenting. The Terminal Disclaimer overcomes the nonstatutory double patenting rejection.
112(b). The amendment overcomes the 112(b) rejections to the claims.
103. Applicant’s amendment does not overcome the prior art of record. The geometry stage includes culling and clipping, which are taken care of by primitive assembly. See Tolo, page 15. The culling and clipping are examples of information of geometry and its relation to screen regions. See Applicants specification, para. 138. Applicant’s arguments are incorrect and somewhat irrelevant to what is mapped. For example, pages 12-13 of the Remarks, the examiner did not cite to the “sort-last” approach, so any suggestion of what Tolo teaches regarding that is irrelevant. Applicant’s arguments are also very piecemeal and do not address the combination of references, and conclusory. For example, Applicant makes no direct argument to why the examiner’s mappings are incorrect, and some of Applicant’s arguments are difficult, with respect to follow.
Applicant respectfully misunderstands Tolo and incorrectly describes the “sort-first” as the code on page 43-44, which is, at best, the tiling, and then rendering step and compositing results (which is the last step of the pipeline of Fig. 2-2), instead of addressing why “Geometry” isn’t geometry testing, for example, as mapped in the prior office action. In addition to not addressing this, Applicant also presented no arguments as to why the graphics pipeline, that Tolo describes and the examiner reprinted in the last office action, would somehow *not* apply in a sort-first scenario (i.e., how a sort-first approach, if one is to believe Applicant’s arguments, goes from tiling all the way to rendering, while skipping several pipeline stages in between. How does the pipeline even know what to render or display?).
Also, the examiner cited to Section 2.3.1 for “Sort-first”, which Applicant did not address specifically why this is in error.
The rejections are respectfully maintained. Please see remainder of this office action for details.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4-6, 8, 11-13, 15 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Tolo, L. O. (2018). Multi-GPU Rendering with Vulkan API (Master's thesis, The University of Bergen) (“Tolo”) (cited in parent) in view of Ashkar (U.S. Patent App. Pub. No. 2018/0082398 A1).
Regarding claim 1:
Tolo teaches: a method for graphics processing (Ch. 1, Introduction and section 2.1.1), comprising.
rendering graphics for an application using a plurality of graphics processing units (GPUs) (section 2.2, “Multi-GPU Configurations” for rendering);
dividing responsibility for the rendering of geometry of the graphics between the plurality of GPUs based on a plurality of screen regions, each GPU having a corresponding division of the responsibility which is known to the plurality of GPUs (section 2.3.1: Sort-first: “Sort-first is defined as an approach where you redistribute primitives early in the graphics pipeline, during geometry processing... This can be done by dividing the screen such that each GPU renders their own region (see Figure 2.5). When all GPUs are done rendering the frame, the final image would be composited from the contributing regions.”);
assigning a plurality of pieces of geometry of an image frame to the plurality of GPUs for geometry testing (section 2.3.1, mapped directly above, the screen is divided into regions assigned to different GPUs for geometry processing);
setting a first GPU state configuring the plurality of GPUs to perform the geometry testing (mapped above, the first “GPU state” is for geometry testing. This would be geometry stage in combination with primitive assembly, below, in Fig. 2.2 of Tolo);
PNG
media_image1.png
200
400
media_image1.png
Greyscale
performing geometry testing at the plurality of GPUs on the plurality of pieces of geometry to generate information regarding each piece of geometry including a relation of each piece of geometry to each of the plurality of screen regions (above mapped, this is geometry testing by the GPUs. See page 15, which describes that clipping, backface culling and viewport transform from the previous stages (i.e. Geometry) are done with primitive assembly. The geometry testing includes the clipping, culling, and viewport transform, which are relations to screen regions),
wherein a plurality of commands (pages 24-25, command execution by GPUs is known, including for shaders (page 27)) is executed by a plurality of shaders (see page 15, a geometry shader can implement the geometry testing) …;
setting a second GPU state configuring the plurality of GPUs to perform rendering (see Fig. 2.2 above, the second state for the GPU can be rendering. Fig. 2.1 is a generic illustration of rendering process for GPUs); and
performing the rendering of the plurality of pieces of geometry at the plurality of GPUs by executing the plurality of commands (Section 2.5.2: Logical Devices, Queues and Commands; rendering commands can be included in the executable GPU commands…,
wherein the plurality of pieces of geometry is rendered by the plurality of GPUs using the information generated for each of the plurality of pieces of geometry including the relation to each of the plurality of screen regions (see above mapping, Section 2.3.1, sort-first which divides work amongst GPUs, and the above referenced figures (2.1, 2.2) for GPU rendering including relation).
Re: wherein a plurality of commands is executed by the plurality of shaders using a first interpretation based on the first GPU state to perform the geometry testing and
performing the rendering of the plurality of pieces of geometry at the plurality of GPUs by executing the plurality of commands using a second interpretation based on the second GPU state, consider the following.
In analogous art, Ashkar teaches that it is known in the art that (emphasis added):
[0001]….Hardware state information is used to configure the GPU to interpret and render the input data provided in response to the draw call. Examples of hardware state information include user configuration state information, shader state information, context information, rasterizer state information, blend state information, depth stencil state information, and the like.
[0002] Software executing on a central processing unit (CPU) generates the hardware state information and provides it (in packet form) to a command processor, which uses the information in the packets to modify values of registers that are used to configure the GPU prior to issuing the draw call.
Applicant’s claimed first and second interpretations based on first and second GPU states can be the result of the state information being used to configure each GPU – this would b an interpretation (i.e. GPU configuration) based on a GPU state, which can be, as shown above, any one of shader state information, context information, rasterizer state information, etc. The examiner’s interpretation of “interpretation” (first and second) is a broad, reasonable interpretation that is consistent with Applicant’s specification as filed. See specification, paragraph 187.
It would have been obvious for one of ordinary skill in the art to have combined and modified the applied reference(-s), in view of same, to have obtained the above, and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
The prior art included each element recited in claim 1, although not necessarily in a single embodiment, with the only difference being between the claimed element and the prior art being the lack of actual combination of certain elements in a single prior art embodiment, as described above.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention. Additional motivation can be found to have better control of rendering in a system with multiple GPUs.
Regarding claim 4:
It would have been obvious for one of ordinary skill in the art to have further modified the applied reference(-s), in view of same, to have obtained: the method of claim 1, wherein the setting the first GPU state includes setting a first value in random access memory to define the first GPU state, wherein the setting the second GPU state includes setting a second value in the random access memory to define the second GPU state, and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
Ashkar teaches state information, which teaches Applicant’s first and second values that define GPU states (see mapping to claim 1, and/or Ashkar, Background). RAM memory is also known/taught by Ashkar (para. 38). The state information can be stored (para. 9). Modifying the applied references, in view of same, such that the state information/values are stored in RAM, all of which is taught by Ashkar, would have been obvious and predictable to one of ordinary skill.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention. Additional motivation can be found to have better control of rendering in a system with multiple GPUs.
Regarding claim 5:
Tolo teaches: the method of claim 1, wherein a shader of the plurality of shaders as executed by a corresponding GPU is configured to perform the geometry testing on a piece of geometry when the corresponding GPU is configured in the first GPU state (mapped in claim 1),
wherein the shader is configured to perform the rendering on the piece of geometry when the corresponding GPU is configured in the second state (see Section 2.1.1. shaders to perform rendering is known, or page 6, “shader is a program that runs for each primitive in one of the programmable rendering pipeline stages”).
It would have been obvious for one of ordinary skill in the art, as of the effective filing date of Applicant’s claims, to have further modified the applied reference(-s) in view of same to have obtained the above, motivated to make use of known programming to render/process graphic data.
Regarding claim 6:
It would have been obvious for one of ordinary skill in the art to have further modified the applied reference(-s), in view of same, to have obtained: the method of claim 1, wherein the performing the rendering includes: providing the information as a hint to a GPU (Tolo, page 25, hints to GPUs are known), wherein the information relates a piece of geometry to each of the plurality of screen regions (see Ashkar, background, this is taught as state information),
wherein the information is considered by the GPU if received before the performing the rendering for the piece of geometry (Tolo, page 25, hint before the task is known),
wherein the piece of geometry is fully rendered by the GPU when the information is received after the performing the rendering for the piece of geometry begins (rendered after rendering is known and definitional), and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
The prior art included each element recited in claim 6, although not necessarily in a single embodiment, with the only difference being between the claimed element and the prior art being the lack of actual combination of certain elements in a single prior art embodiment, as described above.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 8: see also claim 1.
Ashkar teaches: a computer system (para. 39, a system) comprising: a processor (para. 39, one or more processors); memory coupled to the processor and having stored therein instructions that, if executed by the computer system (para. 39, memory storing software instructions for execution), cause the computer system to execute a method for graphics processing (e.g. method as mapped in claim 1), comprising.
The instructions correspond to the method of claim 1; the same rationale for rejection applies.
Regarding claim 11: see claim 4.
These claims are similar; the same rationale for rejection applies.
Regarding claim 12: see claim 5.
These claims are similar; the same rationale for rejection applies.
Regarding claim 13: see claim 6.
These claims are similar; the same rationale for rejection applies.
Regarding claim 15: see also claim 1.
Ashkar teaches: a non-transitory computer-readable medium storing a computer program for performing a method for graphics processing, the non-transitory computer-readable medium comprising: program instructions for (para. 39, memory storing software instructions for execution on a system with processors).
The instructions correspond to the method of claim 1; the same rationale for rejection applies.
Regarding claim 18: see claim 4.
These claims are similar; the same rationale for rejection applies.
Regarding claim 19: see claim 5.
These claims are similar; the same rationale for rejection applies.
Regarding claim 20: see claim 6.
These claims are similar; the same rationale for rejection applies.
Claim(s) 2, 3, 9, 10, 16 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Tolo in view of Ashkar and further in view of Nguyen (U.S. Patent App. Pub. No. 2017/0200308 A1).
Regarding claim 2:
It would have been obvious for one of ordinary skill in the art to have combined and modified the applied reference(-s), in view of same, to have obtained: the method of claim 1, wherein the performing geometry testing includes: calling a subroutine for a first time to perform the geometry testing (Nguyen, para. 57, subroutines for pixel processing and/or shaders are known), wherein the subroutine includes the plurality of commands (Nguyen, paras. 57, 119, subroutines holding commands is known; also by definition, this is what a subroutine is in computer science) in a command buffer (Tolo, pages 34, 38, command buffers are known), and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
The prior art included each element recited in claim 2, although not necessarily in a single embodiment, with the only difference being between the claimed element and the prior art being the lack of actual combination of certain elements in a single prior art embodiment, as described above.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 3:
Nguyen teaches: the method of claim 2, wherein the performing the rendering includes: calling the subroutine for a second time to perform the rendering of the plurality of pieces of geometry (e.g. paras., 57, 63, 91, 265, and by definition, subroutines are designed to be accessed or invoked more than once).
It would have been obvious for one of ordinary skill in the art, as of the effective filing date of Applicant’s claims, to have further modified the applied reference(-s) in view of same to have obtained the above, motivated to make use of known programming to render/process graphic data.
Regarding claim 9: see claim 2.
These claims are similar; the same rationale for rejection applies.
Regarding claim 10: see claim 3.
These claims are similar; the same rationale for rejection applies.
Regarding claim 16: see claim 2.
These claims are similar; the same rationale for rejection applies.
Regarding claim 17: see claim 3.
These claims are similar; the same rationale for rejection applies.
Claim(s) 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Tolo in view of Ashkar and further in view of Grossman (U.S. Patent App. Pub. No. 2017/0004647 A1).
Regarding claim 7:
It would have been obvious for one of ordinary skill in the art to have combined and modified the applied reference(-s), in view of same, to have obtained: the method of claim 1, further comprising: interleaving in a command buffer a first plurality of commands with a second plurality of commands,
wherein the first plurality of commands is executed for generating first information for a first piece of geometry and its relation to the plurality of screen regions and for performing the rendering of the first piece of geometry,
wherein the second plurality of commands is executed for generating second information for a second piece of geometry and its relation to the plurality of screen regions and rendering of the second piece of geometry, and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
Tolo and Grossman both teach that command buffers are known (Tolo, page 38; Grossman, e.g. para. 8). Grossman also teaches that interleaving commands in a command buffer is also known (e.g. paras. 29, 43, 44). In terms of first and second commands, related to first and second pieces of geometry, to generate first and second information, respectively, this is mapped in claim 1, and taught by the region assignment to GPUs in Tolo. One set of commands can be for one GPU, a second for a second GPU, that are assigned to different screen regions and pieces of geometry. The information can be any one of that that is part of the pipeline in Tolo, as also mapped in claim 1.
The prior art included each element recited in claim 7, although not necessarily in a single embodiment, with the only difference being between the claimed element and the prior art being the lack of actual combination of certain elements in a single prior art embodiment, as described above.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 14: see claim 7.
These claims are similar; the same rationale for rejection applies.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
* * * * *
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Sarah Lhymn whose telephone number is (571)270-0632. The examiner can normally be reached M-F, 9:00 AM to 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Sarah Lhymn
Primary Examiner
Art Unit 2613
/Sarah Lhymn/Primary Examiner, Art Unit 2613
Prosecution Insights