Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Detailed Action
This action is in response to the claims filed 11/4/2024:
Claims 25 – 44 are pending.
Claims 25, 33, and 41 are independent.
Drawings
The drawings are objected to because FIGs 25-31 are low quality scans containing illegible elements. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claims 31, 39, and 44 objected to because of the following informalities: "a first draw operation that proceeds the second draw operation" should read" a first draw operation that precedes the second draw operation." Appropriate correction is required.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the claims at issue are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the reference application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO internet Web site contains terminal disclaimer forms which may be used. Please visit http://www.uspto.gov/forms/. The filing date of the application will determine what form should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to http://www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 25-29, 33-37, and 41-42 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1 and 8 of Application No. 15/858,014. Although the claims at issue are not identical, they are not patentably distinct from each other because of the mapping presented below.
Present Application 18/812,822
Application 15/858,014
25. An apparatus, comprising:at least one processor; anda graphics processing unit (GPU) including circuitry configured to execute instructions, the circuitry comprising:
processing circuitry configured to perform general-purpose graphics computations, the processing circuitry including a single instruction multiple thread (SIMT) architecture;
a graph processing unit (GrPU) including instruction execution circuitry configured to accelerate computations on a graph representation in response to a request from the processing circuitry, wherein the GrPU includes multiple single instruction multiple data (SIMD) hardware threads to concurrently traverse multiple graph representations and execute instructions associated with the multiple graph representations; and
a compilation unit (CU) including instruction execution circuitry configured to dynamically compile shader kernels locally on the GPU
1. An apparatus, comprising:at least one processor to perform operations to implement a neural network; anda graphics processing unit (GPU) including circuitry configured to accelerate neural network computations, the circuitry comprising:
processing circuitry configured to perform general-purpose graphics computations, the processing circuitry including a single instruction multiple thread (SIMT) architecture;
a local memory to store one or more graph representations associated with a neural network, the one or more graph representations to indicate node adjacency for the neural network;
a graph processing unit (GrPU) including instruction execution circuitry configured to accelerate computations on the one or more graph representations in response to a request from the processing circuitry, wherein the GrPU includes multiple single instruction multiple data (SIMD) hardware threads to concurrently traverse multiple graph representations and execute instructions associated with the multiple graph representations;
a compilation unit (CU) including instruction execution circuitry configured to dynamically compile shader kernels locally on the GPU; and
26. The apparatus of claim 25, wherein the GrPU is configured to perform a compute operation implemented via a dynamically compiled shader.
wherein the GrPU is configured to perform a compute operation implemented via a dynamically compiled shader,
27. The apparatus of claim 26, the dynamically compiled shader to be dynamically compiled by the CU and executed by the GrPU in response to a condition detected by the GPU.
the dynamically compiled shader is dynamically compiled by the CU and executed by the GrPU in response to a condition detected by the GPU,
28. The apparatus of claim 27, wherein the condition is associated with input data of a neural network computation.
the condition associated with input data of a neural network computation.
29. The apparatus of claim 28, the circuitry configured to:
detect the condition associated with input data of a neural network computation;
based on the condition, compile a modified shader that is to configure the GPU to perform a modified neural network computation; and
perform the modified neural network computation via the modified shader.
8. The apparatus of claim 1, wherein the circuitry of the GPU is configured to:
detect the condition associated with input data of a neural network computation;
based on the condition, compile a modified shader that is to configure the GPU to perform a modified neural network computation; and
perform the modified neural network computation via the compiled modified shader.
Claims 33-37 disclose the method that is implemented by the system of claims 25-29, with substantially the same limitations, respectively. Therefore, the above mapping applies to claims 33-37 as well. Claims 41-43 are directed towards a system that is substantially similar to the system of claims 25-29 with substantially the same limitations. Therefore, the above mapping applies to claims 41-43 as well.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 25, 26, 27, 30, 33, 34, 35, 38, 41, and 42 are rejected under U.S.C. §103 as being unpatentable over the combination of Parker (“OptiX: A General Purpose Ray Tracing Engine”, 2010) and Morein (US20050162437A1).
PNG
media_image1.png
926
1012
media_image1.png
Greyscale
FIG. 2 of Parker
Regarding claim 25, Parker teaches An apparatus, comprising: at least one processor; and a graphics processing unit (GPU) including circuitry configured to execute instructions, the circuitry comprising:([p. 5] "the host is the processor running the client application (usually a CPU). The device is the processor (usually a GPU) running the ray tracing code produced by OptiX, called the kernel.")
processing circuitry configured to perform general-purpose graphics computations, ([p. 1] "we introduce OptiX, a general purpose ray tracing engine. This engine combines a programmable ray tracing pipeline with a lightweight scene representation")
the processing circuitry including a single instruction multiple thread (SIMT) architecture; ([p. 9] "OptiX explicitly selects a single state for an entire SIMT unit to execute using a scheduling heuristic. Threads within the SIMT unit that do not require the state simply idle that iteration. The mechanism is outlined in Figure 12.")
a graph processing unit (GrPU) including instruction execution circuitry configured to accelerate computations on a graph representation ([p. 3] "A scene is represented as a graph. This representation is very lightweight and controls the traversal of rays through the scene" [p. 2] "Intersection programs implement ray-geometry intersection tests. As the acceleration structures are traversed, the system will invoke an intersection program to perform the geometric query" See also Acceleration Traversal in FIG. 2)
in response to a request from the processing circuitry, ([p. 3] "Figure 2: A call graph showing the control flow through the ray tracing pipeline. The yellow boxes represent user-specified programs and the blue boxes are algorithms internal to OptiX. Execution is initiated by the API call rtContextLaunch. A built-in function, rtTrace, can be employed by the ray generation program to cast rays into the scene. This function may also be called recursively by the closest hit program for shadow and secondary rays. The exception program is executed when the execution of a particular ray is terminated by an error such as excessive memory consumption")
wherein the GrPU includes multiple single instruction multiple data (SIMD) hardware threads ([p. 8] "This control flow is more friendly to SIMD execution because it is well-structured" [p. 9] "OptiX explicitly selects a single state for an entire SIMT unit to execute using a scheduling heuristic. Threads within the SIMT unit that do not require the state simply idle that iteration. The mechanism is outlined in Figure 12 [...] By scheduling threads with like states during execution, OptiX reduces the number of total state transitions made by a SIMT unit, which can substantially decrease execution time over the automatic schedule induced by the serialization hardware. Figure 13 shows an example of such a reduction." See also FIG. 12)
to concurrently traverse multiple graph representations and execute instructions associated with the multiple graph representations; and([p. 5] "To allow multiple ray-tracing operations to co-exist in a single execution, OptiX employs a user-defined ray type. A ray type is simply an index that selects a particular set of slots for any hit and closest hit programs to be executed when an intersection is found. This can be used, for example, to treat shadow rays separately from other rays. [...] The core algorithm for finding an intersection between a ray and the scene geometry involves the traversal of acceleration structures. Such data structures are a vital component of virtually every ray tracing system. They are usually spatial or object hierarchies and are used by the traversal algorithm to efficiently search for primitives that potentially intersect a given ray" [p. 6] "multiple acceleration structures in the node graph can be constructed in parallel" [p. 9] "Each ray tracing kernel launch is presented as a queue of data parallel tasks to the physical execution units")
a compilation unit (CU) including instruction execution circuitry configured to dynamically compile shader kernels ([p. 7] "the JIT stage combines all of the user-provided shader programs into one or more kernels").
However, Parker does not explicitly teach a compilation unit (CU) including instruction execution circuitry configured to dynamically compile shader kernels locally on the GPU.
PNG
media_image2.png
742
656
media_image2.png
Greyscale
FIG. 3 of Morein
Morein, in the same field of endeavor, teaches a compilation unit (CU) including instruction execution circuitry configured to dynamically compile shader kernels locally on the GPU.([¶0018] "FIG. 3 illustrates the graphics processor 120" [¶0019] "Also illustrated in FIG. 3, the compiler 126 is further comprised of a shader assembler 148 and a compiler 150. The shader assembler receives the abstract state vector whereupon an intermediate form for the vertex shader and an intermediate form for the pixel shader is created by combining the input intermediate forms with the abstract render state" [¶0014] " if the cache entry does not exist in the hardware state and shader vector, the compiler compiles the abstract state vector to thereupon produce the hardware state and shader vector. The state and shader cache generates a cache entry for the abstract state vector which was not contained therein and when the compiler has completed the compilation process" See also Claim 21 of Morein in which the compiler is explicitly comprised by the graphics processor (GPU) for local shader compilation).
Parker as well as Morein are directed towards . Therefore, Parker as well as Morein are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of Parker with the teachings of Morein by implementing Parker’s JIT shader compilation as dynamic compilation on the GPU. Morein teaches a known implementation architecture where shader compilation can be performed. The graphics processor itself contains a compiler and shader cache, and when the required compiled shader is absent, the graphic processor's compiler compiles the vertex/pixel shader intermediate representation and stores the result. Morein provides as motivation for the combination ([¶0006] "When the compiler 102 must recompile the abstract state vector 104, this creates an inefficient graphics pre-processing system. Therefore, there currently exists a need for an improved graphics processing unit using state and shader management of hardware state and shader vectors."). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 26, the combination of Parker and Morein teaches The apparatus of claim 25, wherein the GrPU is configured to perform a compute operation implemented via a dynamically compiled shader. (Morein [¶0018] "In graphics processing, the flow of input data is provided into the compiler to be provided to the hardware, where the input contains specific vertex and pixel information. Vertex shader information 140 is provided into the compiler and so is pixel shader 142 information. In one embodiment, this shader information is processed not only directly into the compiler, but also through an optimizer 144. Whereupon, in conjunction with abstract render state 146, the abstract state vector is a compilation of a vertex shader intermediate form, which is the output of the vertex shader optimization, and vertex shader constants which were not a part of the optimization, and pixel shader intermediate form, the output of the pixel shader optimization, and pixel shader constants. Thereupon, the abstract state vector contained within the register 122 contains a vertex shader intermediate form, vertex shader constants, pixel shader intermediate form, pixel shader constants, and abstract render state. The compiler 126 utilizes all of this information to assemble and compile the hardware state and shader vector which is thereupon provided to the hardware 134.").
Regarding claim 27, the combination of Parker and Morein teaches The apparatus of claim 26, the dynamically compiled shader to be dynamically compiled by the CU and executed by the GrPU in response to a condition detected by the GPU. (Morein [¶0014] " if the cache entry does not exist in the hardware state and shader vector, the compiler compiles the abstract state vector to thereupon produce the hardware state and shader vector. The state and shader cache generates a cache entry for the abstract state vector which was not contained therein and when the compiler has completed the compilation process").
Regarding claim 30, the combination of Parker and Morein teaches The apparatus of claim 27, wherein the condition is associated with an availability of a conditional simplification of the dynamically compiled shader.(Morein [¶0014] " if the cache entry does not exist in the hardware state and shader vector, the compiler compiles the abstract state vector to thereupon produce the hardware state and shader vector. The state and shader cache generates a cache entry for the abstract state vector which was not contained therein and when the compiler has completed the compilation process" [¶0019] " The shader assembler receives the abstract state vector whereupon an intermediate form for the vertex shader and an intermediate form for the pixel shader is created by combining the input intermediate forms with the abstract render state.").
Regarding claim 33, Parker teaches A method comprising: accelerating, on a graphics processing unit (GPU) in response to a request from general-purpose graphics processing circuitry ([p. 5] "the host is the processor running the client application (usually a CPU). The device is the processor (usually a GPU) running the ray tracing code produced by OptiX, called the kernel." [p. 3] "A scene is represented as a graph. This representation is very lightweight and controls the traversal of rays through the scene" [p. 2] "Intersection programs implement ray-geometry intersection tests. As the acceleration structures are traversed, the system will invoke an intersection program to perform the geometric query" See also Acceleration Traversal in FIG. 2)
including a single instruction multiple thread (SIMT) architecture,([p. 9] "OptiX explicitly selects a single state for an entire SIMT unit to execute using a scheduling heuristic. Threads within the SIMT unit that do not require the state simply idle that iteration. The mechanism is outlined in Figure 12.")
computations on a graph representation via a graph processing unit (GrPU) ([p. 3] "A scene is represented as a graph. This representation is very lightweight and controls the traversal of rays through the scene" [p. 2] "Intersection programs implement ray-geometry intersection tests. As the acceleration structures are traversed, the system will invoke an intersection program to perform the geometric query" See also Acceleration Traversal in FIG. 2)
including instruction execution circuitry having single instruction multiple data (SIMD) multiple hardware threads ([p. 8] "This control flow is more friendly to SIMD execution because it is well-structured" [p. 9] "OptiX explicitly selects a single state for an entire SIMT unit to execute using a scheduling heuristic. Threads within the SIMT unit that do not require the state simply idle that iteration. The mechanism is outlined in Figure 12 [...] By scheduling threads with like states during execution, OptiX reduces the number of total state transitions made by a SIMT unit, which can substantially decrease execution time over the automatic schedule induced by the serialization hardware. Figure 13 shows an example of such a reduction." See also FIG. 12)
to concurrently traverse multiple graph representations and execute instructions associated with the multiple graph representations; and([p. 5] "To allow multiple ray-tracing operations to co-exist in a single execution, OptiX employs a user-defined ray type. A ray type is simply an index that selects a particular set of slots for any hit and closest hit programs to be executed when an intersection is found. This can be used, for example, to treat shadow rays separately from other rays. [...] The core algorithm for finding an intersection between a ray and the scene geometry involves the traversal of acceleration structures. Such data structures are a vital component of virtually every ray tracing system. They are usually spatial or object hierarchies and are used by the traversal algorithm to efficiently search for primitives that potentially intersect a given ray" [p. 6] "multiple acceleration structures in the node graph can be constructed in parallel" [p. 9] "Each ray tracing kernel launch is presented as a queue of data parallel tasks to the physical execution units").
However, Parker does not explicitly teach dynamically compiling a shader kernel locally on the GPU via a compilation unit (CU) included within the GPU.
Morein, in the same field of endeavor, teaches dynamically compiling a shader kernel locally on the GPU via a compilation unit (CU) included within the GPU. ([¶0018] "FIG. 3 illustrates the graphics processor 120" [¶0019] "Also illustrated in FIG. 3, the compiler 126 is further comprised of a shader assembler 148 and a compiler 150. The shader assembler receives the abstract state vector whereupon an intermediate form for the vertex shader and an intermediate form for the pixel shader is created by combining the input intermediate forms with the abstract render state" [¶0014] " if the cache entry does not exist in the hardware state and shader vector, the compiler compiles the abstract state vector to thereupon produce the hardware state and shader vector. The state and shader cache generates a cache entry for the abstract state vector which was not contained therein and when the compiler has completed the compilation process" See also Claim 21 of Morein in which the compiler is explicitly comprised by the graphics processor (GPU) for local shader compilation).
Parker as well as Morein are directed towards . Therefore, Parker as well as Morein are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of Parker with the teachings of Morein by implementing Parker’s JIT shader compilation as dynamic compilation on the GPU. Morein teaches a known implementation architecture where shader compilation can be performed. The graphics processor itself contains a compiler and shader cache, and when the required compiled shader is absent, the graphic processor's compiler compiles the vertex/pixel shader intermediate representation and stores the result. Morein provides as motivation for the combination ([¶0006] "When the compiler 102 must recompile the abstract state vector 104, this creates an inefficient graphics pre-processing system. Therefore, there currently exists a need for an improved graphics processing unit using state and shader management of hardware state and shader vectors."). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 34, the combination of Parker and Morein teaches The method of claim 33, comprising performing a compute operation implemented via a dynamically compiled shader.(Morein [¶0018] "In graphics processing, the flow of input data is provided into the compiler to be provided to the hardware, where the input contains specific vertex and pixel information. Vertex shader information 140 is provided into the compiler and so is pixel shader 142 information. In one embodiment, this shader information is processed not only directly into the compiler, but also through an optimizer 144. Whereupon, in conjunction with abstract render state 146, the abstract state vector is a compilation of a vertex shader intermediate form, which is the output of the vertex shader optimization, and vertex shader constants which were not a part of the optimization, and pixel shader intermediate form, the output of the pixel shader optimization, and pixel shader constants. Thereupon, the abstract state vector contained within the register 122 contains a vertex shader intermediate form, vertex shader constants, pixel shader intermediate form, pixel shader constants, and abstract render state. The compiler 126 utilizes all of this information to assemble and compile the hardware state and shader vector which is thereupon provided to the hardware 134.").
Regarding claim 35, the combination of Parker and Morein teaches The method of claim 34, comprising dynamically compiling the dynamically compiled shader via the CU and executing the dynamically compiled shader in response to a condition detected by the GPU. (Morein [¶0014] " if the cache entry does not exist in the hardware state and shader vector, the compiler compiles the abstract state vector to thereupon produce the hardware state and shader vector. The state and shader cache generates a cache entry for the abstract state vector which was not contained therein and when the compiler has completed the compilation process").
Regarding claim 38, the combination of Parker and Morein teaches The method of claim 35, wherein the condition is associated with an availability of a conditional simplification of the dynamically compiled shader. (Morein [¶0014] " if the cache entry does not exist in the hardware state and shader vector, the compiler compiles the abstract state vector to thereupon produce the hardware state and shader vector. The state and shader cache generates a cache entry for the abstract state vector which was not contained therein and when the compiler has completed the compilation process" [¶0019] " The shader assembler receives the abstract state vector whereupon an intermediate form for the vertex shader and an intermediate form for the pixel shader is created by combining the input intermediate forms with the abstract render state.").
Regarding claim 41, Parker teaches A data processing system comprising: a memory device to store instructions; and([p. 7] "The final acceleration structure data is placed in device memory for consumption by the ray traversal code")
a heterogeneous processor comprising a central processing unit (CPU), a graphics processing unit (GPU), and a compute accelerator, ([p. 5] "the host is the processor running the client application (usually a CPU). The device is the processor (usually a GPU) running the ray tracing code produced by OptiX, called the kernel.")
the heterogeneous processor including circuitry comprising: processing circuitry configured to perform general-purpose graphics computations, ([p. 1] "we introduce OptiX, a general purpose ray tracing engine. This engine combines a programmable ray tracing pipeline with a lightweight scene representation")
the processing circuitry including a single instruction multiple thread (SIMT) architecture;([p. 9] "OptiX explicitly selects a single state for an entire SIMT unit to execute using a scheduling heuristic. Threads within the SIMT unit that do not require the state simply idle that iteration. The mechanism is outlined in Figure 12.")
a graph processing unit (GrPU) including instruction execution circuitry configured to accelerate computations on a graph representation in response to a request from the processing circuitry, ([p. 5] "the host is the processor running the client application (usually a CPU). The device is the processor (usually a GPU) running the ray tracing code produced by OptiX, called the kernel." [p. 3] "A scene is represented as a graph. This representation is very lightweight and controls the traversal of rays through the scene" [p. 2] "Intersection programs implement ray-geometry intersection tests. As the acceleration structures are traversed, the system will invoke an intersection program to perform the geometric query" See also Acceleration Traversal in FIG. 2)
wherein the GrPU includes multiple single instruction multiple data (SIMD) hardware threads ([p. 8] "This control flow is more friendly to SIMD execution because it is well-structured" [p. 9] "OptiX explicitly selects a single state for an entire SIMT unit to execute using a scheduling heuristic. Threads within the SIMT unit that do not require the state simply idle that iteration. The mechanism is outlined in Figure 12 [...] By scheduling threads with like states during execution, OptiX reduces the number of total state transitions made by a SIMT unit, which can substantially decrease execution time over the automatic schedule induced by the serialization hardware. Figure 13 shows an example of such a reduction." See also FIG. 12)
to concurrently traverse multiple graph representations and execute instructions associated with the multiple graph representations; and([p. 5] "To allow multiple ray-tracing operations to co-exist in a single execution, OptiX employs a user-defined ray type. A ray type is simply an index that selects a particular set of slots for any hit and closest hit programs to be executed when an intersection is found. This can be used, for example, to treat shadow rays separately from other rays. [...] The core algorithm for finding an intersection between a ray and the scene geometry involves the traversal of acceleration structures. Such data structures are a vital component of virtually every ray tracing system. They are usually spatial or object hierarchies and are used by the traversal algorithm to efficiently search for primitives that potentially intersect a given ray" [p. 6] "multiple acceleration structures in the node graph can be constructed in parallel" [p. 9] "Each ray tracing kernel launch is presented as a queue of data parallel tasks to the physical execution units").
However, Parker does not explicitly teach a compilation unit (CU) including instruction execution circuitry configured to dynamically compile shader kernels locally on the GPU.
Morein, in the same field of endeavor, teaches a compilation unit (CU) including instruction execution circuitry configured to dynamically compile shader kernels locally on the GPU.([¶0018] "FIG. 3 illustrates the graphics processor 120" [¶0019] "Also illustrated in FIG. 3, the compiler 126 is further comprised of a shader assembler 148 and a compiler 150. The shader assembler receives the abstract state vector whereupon an intermediate form for the vertex shader and an intermediate form for the pixel shader is created by combining the input intermediate forms with the abstract render state" [¶0014] " if the cache entry does not exist in the hardware state and shader vector, the compiler compiles the abstract state vector to thereupon produce the hardware state and shader vector. The state and shader cache generates a cache entry for the abstract state vector which was not contained therein and when the compiler has completed the compilation process" See also Claim 21 of Morein in which the compiler is explicitly comprised by the graphics processor (GPU) for local shader compilation).
Parker as well as Morein are directed towards . Therefore, Parker as well as Morein are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of Parker with the teachings of Morein by implementing Parker’s JIT shader compilation as dynamic compilation on the GPU. Morein teaches a known implementation architecture where shader compilation can be performed. The graphics processor itself contains a compiler and shader cache, and when the required compiled shader is absent, the graphic processor's compiler compiles the vertex/pixel shader intermediate representation and stores the result. Morein provides as motivation for the combination ([¶0006] "When the compiler 102 must recompile the abstract state vector 104, this creates an inefficient graphics pre-processing system. Therefore, there currently exists a need for an improved graphics processing unit using state and shader management of hardware state and shader vectors."). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 42, the combination of Parker and Morein teaches The data processing system of claim 41, wherein the GrPU is configured to perform a compute operation implemented via a dynamically compiled shader, (Morein [¶0018] "In graphics processing, the flow of input data is provided into the compiler to be provided to the hardware, where the input contains specific vertex and pixel information. Vertex shader information 140 is provided into the compiler and so is pixel shader 142 information. In one embodiment, this shader information is processed not only directly into the compiler, but also through an optimizer 144. Whereupon, in conjunction with abstract render state 146, the abstract state vector is a compilation of a vertex shader intermediate form, which is the output of the vertex shader optimization, and vertex shader constants which were not a part of the optimization, and pixel shader intermediate form, the output of the pixel shader optimization, and pixel shader constants. Thereupon, the abstract state vector contained within the register 122 contains a vertex shader intermediate form, vertex shader constants, pixel shader intermediate form, pixel shader constants, and abstract render state. The compiler 126 utilizes all of this information to assemble and compile the hardware state and shader vector which is thereupon provided to the hardware 134.")
the dynamically compiled shader to be dynamically compiled by the CU and executed by the GrPU in response to a condition detected by the GPU. (Morein [¶0014] " if the cache entry does not exist in the hardware state and shader vector, the compiler compiles the abstract state vector to thereupon produce the hardware state and shader vector. The state and shader cache generates a cache entry for the abstract state vector which was not contained therein and when the compiler has completed the compilation process").
Claims 28, 29, 36, 37, and 43 are rejected under U.S.C. §103 as being unpatentable over the combination of Parker and Morein and Astrom (“Neural Network on Compute Shader Running and Training a Neural Network using GPGPU”, 2011).
Regarding claim 28, the combination of Parker and Morein teaches The apparatus of claim 27.
However, the combination of Parker and Morein doesn't explicitly teach, wherein the condition is associated with input data of a neural network computation.
Astrom, in the same field of endeavor, teaches The apparatus of claim 27, wherein the condition is associated with input data of a neural network computation. ([p. 1] "In this thesis I look into how one can train and run an artificial neural network using Compute Shader and what kind of performance can be expected.").
The combination of Parker and Morein as well as Astrom are directed towards GPU based accelerators. Therefore, the combination of Parker and Morein as well as Astrom are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Parker and Morein with the teachings of Astrom by using the graph shader for neural network computation. Astrom teaches that any artificial neural network can be implemented in a shader. Astrom further teaches the benefits to implementing the neural network in a shader ([p. 20 Sec. 9] “This thesis shows that by using compute shader we can cut down on train-ing time for artificial neural networks when using big networks and many datasets.”). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 29, the combination of Parker, Morein, and Astrom teaches The apparatus of claim 28, the circuitry configured to: detect the condition associated with input data of a neural network computation;(Astrom [p. 10] "each time the compiler finds an identifier with the same name as the macro name then it replaces that with the macro definition" [p. 11] "In this implementation we have a layer buffer that holds a number of layers defined in a shader macro and each layer consist of the layer size, the first neuron index and the first connection index for the first neuron." compiler finding macro identifier interpreted as a condition for shader compilation associated with input data of a neural network computation)
based on the condition, compile a modified shader that is to configure the GPU to perform a modified neural network computation; and perform the modified neural network computation via the modified shader.(Astrom [p. 10] "each time the compiler finds an identifier with the same name as the macro name then it replaces that with the macro definition" [p. 11] "In this implementation we have a layer buffer that holds a number of layers defined in a shader macro and each layer consist of the layer size, the first neuron index and the first connection index for the first neuron." compiler finding macro identifier interpreted as a condition for shader compilation associated with input data of a performed modified neural network computation).
Regarding claim 36, the combination of Parker and Morein teaches The method of claim 35.
However, the combination of Parker and Morein doesn't explicitly teach, wherein the condition is associated with input data of a neural network computation.
Astrom, in the same field of endeavor, teaches the condition is associated with input data of a neural network computation. ([p. 1] "In this thesis I look into how one can train and run an artificial neural network using Compute Shader and what kind of performance can be expected.").
The combination of Parker and Morein as well as Astrom are directed towards GPU based accelerators. Therefore, the combination of Parker and Morein as well as Astrom are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Parker and Morein with the teachings of Astrom by using the graph shader for neural network computation. Astrom teaches that any artificial neural network can be implemented in a shader. Astrom further teaches the benefits to implementing the neural network in a shader ([p. 20 Sec. 9] “This thesis shows that by using compute shader we can cut down on train-ing time for artificial neural networks when using big networks and many datasets.”). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 37, the combination of Parker, Morein, and Astrom teaches The method of claim 36, comprising: detecting the condition associated with input data of a neural network computation;(Astrom [p. 10] "each time the compiler finds an identifier with the same name as the macro name then it replaces that with the macro definition" [p. 11] "In this implementation we have a layer buffer that holds a number of layers defined in a shader macro and each layer consist of the layer size, the first neuron index and the first connection index for the first neuron." compiler finding macro identifier interpreted as a condition for shader compilation associated with input data of a neural network computation)
compiling a modified shader based on the condition, the modified shader to configure the GPU to perform a modified neural network computation; and performing the modified neural network computation via the modified shader. (Astrom [p. 10] "each time the compiler finds an identifier with the same name as the macro name then it replaces that with the macro definition" [p. 11] "In this implementation we have a layer buffer that holds a number of layers defined in a shader macro and each layer consist of the layer size, the first neuron index and the first connection index for the first neuron." compiler finding macro identifier interpreted as a condition for shader compilation associated with input data of a performed modified neural network computation).
Regarding claim 43, the combination of Parker and Morein teaches The data processing system of claim 42.
However, the combination of Parker and Morein doesn't explicitly teach, wherein the condition is associated with input data of a neural network computation and the circuitry is configured to:
detect the condition associated with input data of a neural network computation;
based on the condition, compile a modified shader that is to configure the GPU to perform a modified neural network computation; and perform the modified neural network computation via the modified shader.
Astrom, in the same field of endeavor, teaches The data processing system of claim 42, wherein the condition is associated with input data of a neural network computation and the circuitry is configured to: ([p. 1] "In this thesis I look into how one can train and run an artificial neural network using Compute Shader and what kind of performance can be expected.")
detect the condition associated with input data of a neural network computation; ([p. 10] "each time the compiler finds an identifier with the same name as the macro name then it replaces that with the macro definition" [p. 11] "In this implementation we have a layer buffer that holds a number of layers defined in a shader macro and each layer consist of the layer size, the first neuron index and the first connection index for the first neuron." compiler finding macro identifier interpreted as a condition for shader compilation associated with input data of a neural network computation)
based on the condition, compile a modified shader that is to configure the GPU to perform a modified neural network computation; and perform the modified neural network computation via the modified shader.([p. 10] "each time the compiler finds an identifier with the same name as the macro name then it replaces that with the macro definition" [p. 11] "In this implementation we have a layer buffer that holds a number of layers defined in a shader macro and each layer consist of the layer size, the first neuron index and the first connection index for the first neuron." compiler finding macro identifier interpreted as a condition for shader compilation associated with input data of a performed modified neural network computation).
The combination of Parker and Morein as well as Astrom are directed towards GPU based accelerators. Therefore, the combination of Parker and Morein as well as Astrom are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Parker and Morein with the teachings of Astrom by using the graph shader for neural network computation. Astrom teaches that any artificial neural network can be implemented in a shader. Astrom further teaches the benefits to implementing the neural network in a shader ([p. 20 Sec. 9] “This thesis shows that by using compute shader we can cut down on train-ing time for artificial neural networks when using big networks and many datasets.”). This motivation for combination also applies to the remaining claims which depend on this combination.
Claims 31, 32, 39, 40, and 44 are rejected under U.S.C. §103 as being unpatentable over the combination of Parker and Morein and in further view of Kubisch (US20140292771A1).
Regarding claim 31, the combination of Parker and Morein teaches The apparatus of claim 30.
However, the combination of Parker and Morein doesn't explicitly teach wherein the dynamically compiled shader is a shader associated with a second draw operation and the condition is to be detected in a first draw operation that proceeds the second draw operation.
Kubisch, in the same field of endeavor, teaches the dynamically compiled shader is a shader associated with a second draw operation and the condition is to be detected in a first draw operation that proceeds the second draw operation. ([¶0008] "The method includes receiving a draw call from a software application, evaluating graphics state information included in the draw call, generating via a shader program modified graphics state information, generating a modified draw call that includes the modified graphics state information, and causing the modified draw call to be executed within a graphics processing pipeline.").
The combination of Parker and Morein as well as Kubisch are directed towards dynamic GPU shader compilation. Therefore, the combination of Parker and Morein as well as Kubisch are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Parker and Morein with the teachings of Kubisch by dynamically modifying shader draw calls. Kubisch provides as additional motivation for combination ([Abstract] “By performing the computations associated with generating the modified draw call on-the-fly within the processing subsystem, the draw-call shader decreases the amount of system memory required to render graphics while increasing the overall processing efficiency of the graphics processing pipeline.”). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 32, the combination of Parker, Morein, and Kubisch teaches The apparatus of claim 31, wherein the CU is configured to submit a call to the GrPU to assist with compilation of the dynamically compiled shader.(Morein [¶0014] " if the cache entry does not exist in the hardware state and shader vector, the compiler compiles the abstract state vector to thereupon produce the hardware state and shader vector. The state and shader cache generates a cache entry for the abstract state vector which was not contained therein and when the compiler has completed the compilation process" [¶0019] " The shader assembler receives the abstract state vector whereupon an intermediate form for the vertex shader and an intermediate form for the pixel shader is created by combining the input intermediate forms with the abstract render state." See also FIG. 3).
Regarding claim 39, the combination of Parker and Morein teaches The method of claim 38.
However, the combination of Parker and Morein doesn't explicitly teach, wherein the dynamically compiled shader is a shader associated with a second draw operation and the condition was detected in a first draw operation that proceeded the second draw operation.
Kubisch, in the same field of endeavor, teaches the dynamically compiled shader is a shader associated with a second draw operation and the condition was detected in a first draw operation that proceeded the second draw operation. ([¶0008] "The method includes receiving a draw call from a software application, evaluating graphics state information included in the draw call, generating via a shader program modified graphics state information, generating a modified draw call that includes the modified graphics state information, and causing the modified draw call to be executed within a graphics processing pipeline.").
The combination of Parker and Morein as well as Kubisch are directed towards dynamic GPU shader compilation. Therefore, the combination of Parker and Morein as well as Kubisch are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Parker and Morein with the teachings of Kubisch by dynamically modifying shader draw calls. Kubisch provides as additional motivation for combination ([Abstract] “By performing the computations associated with generating the modified draw call on-the-fly within the processing subsystem, the draw-call shader decreases the amount of system memory required to render graphics while increasing the overall processing efficiency of the graphics processing pipeline.”). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 40, the combination of Parker, Morein, and Kubisch teaches The method of claim 39, comprising submitting a call by the CU to the GrPU to assist with compilation of the dynamically compiled shader.(Morein [¶0014] " if the cache entry does not exist in the hardware state and shader vector, the compiler compiles the abstract state vector to thereupon produce the hardware state and shader vector. The state and shader cache generates a cache entry for the abstract state vector which was not contained therein and when the compiler has completed the compilation process" [¶0019] " The shader assembler receives the abstract state vector whereupon an intermediate form for the vertex shader and an intermediate form for the pixel shader is created by combining the input intermediate forms with the abstract render state.").
Regarding claim 44, the combination of Parker and Morein teaches The data processing system of claim 42.
However, the combination of Parker and Morein doesn't explicitly teach, wherein the condition is associated with an availability of a conditional simplification of the dynamically compiled shader, the dynamically compiled shader is a shader associated with a second draw operation, and the condition is to be detected in a first draw operation that proceeds the second draw operation..
Kubisch, in the same field of endeavor, teaches the condition is associated with an availability of a conditional simplification of the dynamically compiled shader, the dynamically compiled shader is a shader associated with a second draw operation, and the condition is to be detected in a first draw operation that proceeds the second draw operation.([¶0008] "The method includes receiving a draw call from a software application, evaluating graphics state information included in the draw call, generating via a shader program modified graphics state information, generating a modified draw call that includes the modified graphics state information, and causing the modified draw call to be executed within a graphics processing pipeline.").
The combination of Parker and Morein as well as Kubisch are directed towards dynamic GPU shader compilation. Therefore, the combination of Parker and Morein as well as Kubisch are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Parker and Morein with the teachings of Kubisch by dynamically modifying shader draw calls. Kubisch provides as additional motivation for combination ([Abstract] “By performing the computations associated with generating the modified draw call on-the-fly within the processing subsystem, the draw-call shader decreases the amount of system memory required to render graphics while increasing the overall processing efficiency of the graphics processing pipeline.”). This motivation for combination also applies to the remaining claims which depend on this combination.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Munshi (US20150347108A1) is directed towards a GPU accelerator which dynamically compiles shaders on the GPU.
Grover (US20150331700A1) is directed towards a GPU accelerator which explicitly compiles on the GPU.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SIDNEY VINCENT BOSTWICK whose telephone number is (571)272-4720. The examiner can normally be reached M-F 7:30am-5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached on (571)270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SIDNEY VINCENT BOSTWICK/Examiner, Art Unit 2124
