DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Terminal Disclaimer
The terminal disclaimer filed on 08/13/2026 is approved on 08/24/2026.
Response to Amendment
The office action is responsive to the amendment received 08/13/2026.
In the response to the Non-Final Office Action 08/06/2026, applicant states that claims 1-20 are pending, with claims 1, 11, and 15 being independent claims. Claims 6 and 19 are currently amended.
Claims 6 and 19 have been amended. In summary, claims 1-20 are pending in current application.
Response to Arguments
Applicant's arguments filed 08/13/2026 have been fully considered but they are not persuasive.
The terminal disclaimer filed on 08/13/2026 is approved on 08/24/2026. Therefore, the double patenting rejection of claims 1, and 3-20 are hereby withdrawn.
Regarding to claim 1, the applicant argues that Umuroglu and Ng do not disclose a graphics processor. The arguments have been fully considered, but they are not persuasive. The examiner cannot concur with the applicant for following reasons:
Umuroglu discloses “a graphics processor”. For example, in paragraph [0019], Umuroglu teaches the neural networks classify and execute incoming graphic images; Umuroglu further teaches a neural network implementation can classify and process input data sets at 100 million images per second; Umuroglu further more teaches a neural network implementation inside a programmable integrated circuit and GPUs are graphic processors; In Fig. 1 and paragraph [0023-0024], Umuroglu teaches a mapping of a binary neural network 101 onto a hardware implementation in a programmable integrated circuit (IC); Umuroglu further teaches a programmable integrated circuit executes one or more images. In Fig. 4 and paragraphs [0032-0033], Umuroglu teaches a neural network circuit, i.e. a specialized microprocessor, commonly called a Neural Processing Unit, NPU; Umuroglu further teaches an input data set, e.g., a graphic image, is divided into a set of N input feature maps each having a height H and a width W. In paragraph [0037], Umuroglu teaches the control circuit 130 loads a next set of binary weights and a next set of threshold values corresponding to a next layer of the binary neural network 101 into the memory circuit(s) 120 for use by the layer circuit(s) 120. In Fig. 9 and paragraph [0039], Umuroglu teaches digital signal processing blocks; Umuroglu further teaches some FPGAs include dedicated processor blocks. In Fig. 9 and paragraph [0043], Umuroglu teaches that processor block 10 includes various components ranging from a single microprocessor to a complete programmable processing system of microprocessor(s), memory controllers, peripherals, and the like.
Ng discloses “graphics processor and graphics cores”. For example, in paragraph [0025], Ng teaches a graphics processing unit; Ng further teaches GPU. In Fig. 2 and paragraph [0031], Ng teaches GPU and FPGA; Ng further teaches an acceleration circuit is hosted on any type of hardware system, e.g., a GPU; Ng furthermore teaches an acceleration circuit or kernel accelerator circuit is hosted on GPU. In Fig. 2 and paragraph [0032], Ng teaches the host 105 includes a processor 110 and a memory 115; Ng further teaches the processor 110 represents any number of processing elements that each includes any number of processing cores. In paragraph [0074], Ng teaches assigning additional processing cores in the host to execute the pre-processing stage. In Fig. 9 and paragraph [0085], Ng teaches the hardware 904 includes various other conventional devices and peripherals of a computing system, such as graphics cards, universal serial bus (USB) interfaces, and the like.
PNG
media_image1.png
178
344
media_image1.png
Greyscale
. In paragraph [0086], Ng teaches the programmable IC 928 can be an FPGA or the like or an SoC having an FPGA or the like.
Regarding to claim 1, the applicant argues that the cited Umuroglu-Ng combination therefore does not teach or suggest "the programmable neural network unit addressable by cores within the block of graphics cores.” The arguments have been fully considered, but they are not persuasive. The examiner cannot concur with the applicant for following reasons:
Ng discloses “the programmable neural network unit addressable by cores within the block of graphics cores”. For example, in Fig. 2 and paragraph [0031], Ng teaches GPU and FPGA; Ng further teaches an acceleration circuit is hosted on any type of hardware system—e.g., a GPU; Ng furthermore teaches an acceleration circuit or kernel accelerator circuit is hosted on GPU. In Fig. 2 and paragraph [0032], Ng teaches the host 105 includes a processor 110 and a memory 115; the processor 110 represents any number of processing elements that each includes any number of processing cores. In Fig. 2 and paragraph [0033], Ng teaches the memory 115 includes the neural network application 120 executed by the processor 110; Ng further teaches the neural network application 120 establishes the neural network with any number of layers;
PNG
media_image2.png
479
654
media_image2.png
Greyscale
.
Regarding to claim 1, the applicant argues that the Chen-Santodomingo combination does not teach or suggest "the programmable neural network unit is to configure one or more neural network hardware blocks with a meta-shader neural network, the meta-shader neural network to generate a texture for one of multiple types of terrain." The arguments have been fully considered, but they are not persuasive. The examiner cannot concur with the applicant for following reasons:
Umuroglu in view of Chen and Santodomingo discloses "the programmable neural network unit is to configure one or more neural network hardware blocks with a meta-shader neural network, the meta-shader neural network to generate a texture for one of multiple types of terrain."
Umuroglu discloses “wherein the programmable neural network unit is to configure one or more neural network hardware blocks”. For example, in paragraph [0021], Umuroglu teaches weights and thresholds are infrequently updated and configured; Umuroglu further teaches the weights and thresholds are configured to generate specialized hardware neurons. In Fig. 1 and paragraph [0023], Umuroglu teaches in a programmable integrated circuit, each layer 102 includes neurons 104 having inputs, and outputs. In Fig. 3 and paragraph [0030], Umuroglu teaches the count circuit 304 is configured to count the number of the X logic signals output by the XNOR circuit 302 that have a predefined logic state. In Fig. 6 and paragraph [0036], Umuroglu teaches in the programmable IC 118, the layer circuit 120 is configured to implement the layer 102-1, then the layer 102-2, and so on until implementing the layer 102-n; the programmable IC 118 includes more than one layer circuit 120. In Fig. 7 and paragraph [0037], Umuroglu teaches using the programmable IC 118; the layer circuits 120 process the binary inputs thereto.
Chen discloses “one or more neural network hardware blocks with a meta-shader neural network”. For example, in col. 2, lines 30-40, Chen teaches the multistage NN system may store neural networks as shader programs, i.e. meta-shader neural network, on a memory of a GPU; Chen further teaches the weights of the neural networks are stored as shader objects or textures on the memory of the GPU. In col. 5, lines 55-65, Chen teaches the multistage NN client 210 stores and executes shaders on the multistage NN server 117, which is configured to implement neural network shaders. In Fig. 7 and col. 10, lines 55-67, Chen teaches the CPU 700 includes an application 710, which may include code configured to execute one or more shaders 720 stored on the GPU 705. In Fig. 7, col. 11, lines 1-20, Chen teaches the components of the GPU 705 are components of a graphics platform, such as OpenGL. Chen further teaches the GPU 705 is located on a machine different from the client device. Fig. 8, and col. 11, lines 40-65, Chen teaches the configuration engine 615 stores one or more shaders on the GPU; Chen further teaches each neural network to be implemented may have a shader program stored on the GPU and weights stored as texture data on the GPU.
Chen further discloses “the meta-shader neural network to generate a texture for one of multiple types of texture”. For example, in col. 2, lines 35-45, Chen teaches the weights of the neural networks are stored as shader objects or textures on the memory of the GPU. In col. 10, lines 20-30, Chen teaches the training engine 610 manages generating neural network models to generate weights which are stored as object data, e.g., texture data, on GPU memory; Chen further teaches neural network models generate texture data. In col. 11, lines 1-10, Chen teaches the application 710 interfaces with the one or more shaders 720 and objects 725, e.g., textures, on the GPU 705. In col. 11, lines 54-65, Chen teaches each neural network to be implemented has a shader program stored on the GPU and weights stored as texture data on the GPU; Chen further teaches storing the output as texture data. In col. 12, lines 1-10, Chen teaches shader adds two input textures element by element and stores the result into the output texture. In col. 13, lines 36-60, Chen teaches the shader engine 655 applies style transfer using a neural network to the head texture stored in the object memory 660; Chen further teaches generating the modified texture; Chen furthermore teaches the modified texture is stored as a new texture. In col. 15, lines 10-35, Chen teaches creating an OpenGL texture; Chen further teaches performing texture-based NN operations in OpenGL.
Santodomingo discloses “multiple types of texture are multiple types of terrain”. For example, in Fig. 6 and paragraph [0041], Santodomingo teaches various types of terrain classes. In paragraph [0044], Santodomingo teaches a suburban terrain class is at a lower priority than an urban terrain class; Santodomingo further teaches two types of terrain. In paragraph [0045], Santodomingo teaches the selected terrain class and another terrain class are different types; Santodomingo furthermore teaches two types of terrain.
Regarding to claim 1, the applicant argues that there are not teaching and motivation for combining Chen and Santodomingo and the proposed combination results from using Applicant's disclosure as a roadmap. The arguments have been fully considered, but they are not persuasive. The examiner cannot concur with the applicant for following reasons:
In response to applicant's argument that there are not teaching and motivation for combining Chen and Santodomingo and the proposed combination results from using Applicant's disclosure as a roadmap, the examiner recognizes that the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). Chen teaches generating weights which are stored as texture data on GPU memory, modifying images, and displaying modified images. Chen further teaches a fragment shader processes fragments to generate one or more colors and/or a depth value as fragment outputs. Chen furthermore teaches vertex shaders are shaders in a render pipeline that render vertices of a scene, e.g., 3D scene. Santodomingo teaches determining the size, shape, texture, and other characteristics, modifying the texture value, and generating graphics data to display an alternate type of 3D object, such as a vegetation object with textures. Santodomingo further teaches efficiently rendering a graphic object based on conditions and rendering scenery objects. Both Chen and Santodomingo process textures, render a scene, and display images. In response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, the motivation for combining Chen and Santodomingo would have been to render objects with realistic lighting depending on current conditions; to render scenery objects that overlay the terrain data; to improve rendering; to associate different types and sizes of scenery objects with each terrain class; to adjust the texture color values on the wall and roof surfaces of the building; to determine lighting for the vegetation object and to adjust the texture color values as taught by Santodomingo in paragraphs [0007], [0033], [0035], [0045-0045], [0047], and [0054]. The examiner recognizes “the law does not require that the references be combined for the reasons contemplated by the inventor” {In re Beattie, 974 F.2d 1309, 1314 (Fed. Cir. 1992)(prior citations omitted)). The Examiner’s articulated reasoning provides a rational underpinning to support the legal conclusion of obviousness. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398,418 (2007).
Regarding to claim 2, the applicant argues that applicant argues that cited arts fail to teach or suggest “an input parameter to a neural network that causes generation of terrain texture data”. The arguments have been fully considered, but are not persuasive. The examiner cannot concur with the applicant for following reasons:
What claimed is: “wherein the meta-shader neural network is to generate a texture for one of multiple types of terrain based on one or more input parameters to the meta-shader neural network”.
Chen discloses “wherein the meta-shader neural network is to generate a texture for one of multiple types of texture based on one or more input parameters to the meta-shader neural network”. For example, in col. 11, lines 54-65, Chen teaches each neural network to be implemented has a shader program stored on the GPU and weights stored as texture data on the GPU; Chen further teaches storing the output as texture data. In col. 12, lines 1-10, Chen teaches shader adds two input textures element by element and stores the result into the output texture. In col. 13, lines 36-60, Chen teaches the shader engine 655 applies style transfer using a neural network to the head texture stored in the object memory 660; Chen further teaches generating the modified texture; Chen furthermore teaches the modified texture is stored as a new texture. In col. 15, lines 10-35, Chen teaches creating an OpenGL texture; Chen further teaches performing texture-based NN operations in OpenGL.
Santodomingo discloses “terrain”. For example, in paragraph [0033], Santodomingo teaches rendering scenery objects that overlay the terrain data. In Fig. 6 and paragraph [0041], Santodomingo teaches various types of terrain classes; Santodomingo further teaches texture is terrain texture. In paragraph [0044], Santodomingo teaches a suburban terrain class is at a lower priority than an urban terrain class; Santodomingo further teaches two types of terrain; Santodomingo furthermore teaches two types textures. In paragraph [0045], Santodomingo teaches the selected terrain class and another terrain class are different types; Santodomingo further teaches two types of terrain texture.
In summary, the combination of Chen and Santodomingo discloses “wherein the meta-shader neural network is to generate a texture for one of multiple types of terrain based on one or more input parameters to the meta-shader neural network”.
Regarding to claim 9, the applicant argues that “The filed specification distinguishes this arrangement: a trained machine-learning model generates visibility information on a per-object basis, and completely obstructed objects are sent to the graphics pipeline for culling. Spec. III [0235]-[0236], Fig. 27.” The arguments have been fully considered. However, the claim 9 does not include “a trained machine-learning model generates visibility information on a per-object basis, and completely obstructed objects are sent to the graphics pipeline for culling”.
Regarding to claim 9, the applicant argues that the combination of Chen and Hakura does not teach the claim limitations of claim 9. The arguments have been fully considered, but are not persuasive. The examiner cannot concur with the applicant for following reasons:
Chen discloses “wherein the programmable neural network unit is to determine visibility for a geometry operation via the neural network hardware block and the neural network hardware block is to determine visibility on a per-object basis”. For example, in col. 10, lines 35-55, Chen teaches the multistage NN server 117 comprises an interface engine 650, a shader engine 655, and an object memory 660; Chen further teaches a fragment shader processes fragments, i.e. geometry operation, to generate one or more colors and a depth value, i.e. related to visibility, as fragment outputs; Chen further more teaches Vertex shaders are shaders in a render pipeline that render vertices of a scene. In col. 12, lines 35-50, Chen teaches the image identified at operation 815 is part of a pipeline implementing real-time neural network image effects, e.g., implemented by the shaders 720 on the GPU 705. In Fig. 12 and col. 14, lines 33-45, Chen teaches once the shader processes of the multistage NN server 117 are complete, the interface engine 650 transmits the modified image 1200 to the display engine 625 for display;
PNG
media_image3.png
504
368
media_image3.png
Greyscale
; Chen further teaches the visibility is determined as illustrated in Fig. 12.
In same field of endeavor, Hakura discloses “to determine visibility for a geometry culling operation and to determine visibility on a per-object basis”. For example, in paragraph [0032], Hakura teaches implementing a graphics rendering pipeline to perform various operations related to generating pixel data based on graphics data. In paragraph [0040], Hakura teaches image rendering operations; Hakura further teaches tessellation shader, vertex shader, geometry shader, and/or pixel/fragment shading programs. In paragraph [0067], Hakura teaches the rasterizer 370 reads the meshlets 360, scans the graphics primitives, and transmits fragments and coverage data to the pixel shading unit 380; Hakura further teaches the rasterizer 385 is configured to perform z culling and other z-based optimizations. In paragraph [0072], Hakura teaches the graphics processing pipeline 320 includes cull, and clip unit (VPC). In paragraph [0079], Hakura teaches the mesh shader 330 has culled all the graphics primitives processed by the mesh shader 330. In paragraph [0100], Hakura teaches in hierarchical culling, evaluation of an imposter is performed in a first stage, and finer evaluation of graphics primitives is performed in a second stage. In paragraph [0158], Hakura teaches the graphics processing pipeline performs primitive culling operations. In paragraph [0177], Hakura teaches the second plurality of execution threads performs one or more culling operations on a plurality of graphics primitives specified via the first indirect index buffer and the first vertex buffer. In paragraph [0178], Hakura teaches a graphics processing pipeline renders a first frame and a second frame based on the first vertex buffer, the first indirect index buffer, a second vertex buffer, a second indirect index buffer, and a shading program.
Regarding to claim 10, the applicant argues that “The filed specification describes using coarse vertex data and resulting fine vertex data to train a neural network for AI-based tessellation”. The arguments have been fully considered. However, the claim 10 does not include “using coarse vertex data and resulting fine vertex data to train a neural network for AI-based tessellation”.
Regarding to claim 10, the applicant argues that the cited Hakura passages do not teach that claimed neural-network configuration. The arguments have been fully considered, but are not persuasive. The examiner cannot concur with the applicant for following reasons:
What claimed is: “to configure the programmable neural network unit”.
Umuroglu discloses “further comprising to configure the programmable neural network unit”. For example, in paragraph [0005], Umuroglu teaches a programmable integrated circuit (IC) includes a programmable fabric configured to implement: a layer of hardware neurons, the layer including a plurality of inputs, a plurality of outputs, a plurality of weights, and a plurality of threshold values. In paragraph [0021], Umuroglu teaches weights and thresholds are infrequently updated; Umuroglu further teaches the weights and thresholds are used to generate specialized hardware neurons. In Fig. 1 and paragraph [0023], Umuroglu teaches in a programmable integrated circuit, i.e. programmable IC, each layer 102 includes neurons 104 having synapses 106, e.g., inputs, and activations 108, e.g., outputs. In Fig. 6 and paragraph [0036], Umuroglu teaches in the programmable IC 118, the layer circuit 120 is configured to implement the layer 102-1, then the layer 102-2, and so on until implementing the layer 102-n; Umuroglu further teaches the programmable IC 118 includes more than one layer circuit 120. In Fig. 7 and paragraph [0037], Umuroglu teaches using the programmable IC 118; Umuroglu further teaches the layer circuit(s) 120 process the binary inputs thereto.
In same field of endeavor, Hakura discloses “a tessellation module to generate tessellated output based on coarse input data”. For example, in paragraph [0040], Hakura teaches image rendering operations include tessellation shader, vertex shader, geometry shader, and/or pixel/fragment shading programs, general compute operations, etc. In paragraph [0094], Hakura teaches the task shader 650 performs read/load operations, general compute operations, vertex shading operations, tessellation operations, geometry shading operations, and write/store operations. In paragraph [0100], Hakura teaches programmable tessellation patterns.
Claim 11 and claim 15 are not allowable due to similar reasons as discussed above.
Claims 2-8, 12-14, and 16-20 are not allowable due to the similar reasons as discussed above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, and 11-18 are rejected under 35 U.S.C. 103 as being unpatentable over Umuroglu (US 20180039886 A1) in view of Ng (US 20190114533 A1), in view of Chen (US 10482565 B1), and further in view of Santodomingo (US 20040263512 A1).
Regarding to claim 1 (Original), Umuroglu discloses a graphics processor ([0019]: the neural networks classify incoming data sets, e.g., images; a programmable integrated circuit, FPGA and GPUs are graphic processor; Fig. 1; [0023]: a programmable integrated circuit; [0037]: the control circuit 130 loads a next set of binary weights and a next set of threshold values corresponding to a next layer of the binary neural network 101 into the memory circuit(s) 120 for use by the layer circuit(s) 120; [0039]: digital signal processing blocks; some FPGAs also include dedicated processor blocks; [0043]: a complete programmable processing system of microprocessors) comprising:
a block of graphics compute units ([0019]: a programmable integrated circuit, FPGA and GPUs include a block of graphic compute units; [0039]: digital signal processing blocks; some FPGAs also include dedicated processor blocks; [0043]: a complete programmable processing system of microprocessors); and
circuitry including a programmable neural network unit (Fig. 1; [0023]: a programmable integrated circuit includes a programmable binary neural network 101;
PNG
media_image4.png
407
573
media_image4.png
Greyscale
Fig. 1; [0026]: the programmable hardware implementation of the binary neural network 101 includes layer circuits 120; Fig. 1; [0027]: the programmable IC 118 includes circuits to facilitate operation of the layer circuits 120; the programmable IC 118 also includes memory circuits 128 and a control circuit 130), the programmable neural network unit including one or more neural network hardware blocks (Fig. 1; [0026]: the programmable hardware implementation of the binary neural network 101 includes layer circuits 120; Fig. 1; [0027]: the programmable IC 118 includes circuits to facilitate operation of the layer circuits 120; the programmable IC 118 also includes memory circuits 128 and a control circuit 130; Fig. 2; [0028]: a layer circuit 120A of a neural network circuit;
PNG
media_image5.png
308
517
media_image5.png
Greyscale
), wherein a neural network hardware block includes circuitry to perform neural network operations and activation operations for a layer of a neural network (Fig. 1; [0026]: the programmable hardware implementation of the binary neural network 101 includes layer circuits 120; the activations 108 are mapped to the output connections 126; [0027]: the programmable IC 118 includes circuits to facilitate operation of the layer circuits 120; Fig. 7; [0037]: using the programmable IC 118, the layer circuits 120 process the binary inputs thereto), wherein the programmable neural network unit is to configure one or more neural network hardware blocks (Umuroglu; [0021]: weights and thresholds are infrequently updated and configured; the weights and thresholds are configured to generate specialized hardware neurons; Fig. 1; [0023]: in a programmable integrated circuit, each layer 102 includes neurons 104 having inputs, and outputs; Fig. 3; [0030]: the count circuit 304 is configured to count the number of the X logic signals output by the XNOR circuit 302 that have a predefined logic state; Fig. 6; [0036]: in a the programmable IC 118, the layer circuit 120 is configured to implement the layer 102-1, then the layer 102-2, and so on until implementing the layer 102-n; the programmable IC 118 includes more than one layer circuit 120; Fig. 7; [0037]: using the programmable IC 118; the layer circuits 120 process the binary inputs thereto).
Umuroglu fails to explicitly disclose:
compute units are graphics cores; and
the programmable neural network unit addressable by cores within the block of graphics cores;
one or more neural network hardware blocks with a meta-shader neural network, the meta-shader neural network to generate a texture for one of multiple types of terrain.
In same field of endeavor, Ng teaches:
compute units are graphics cores ([0025]: a graphics processing unit; GPU; [0031]: GPU; Fig. 2; [0032]: the host 105 includes a processor 110 and a memory 115; the processor 110 represents any number of processing elements that each includes any number of processing cores; [0074]: assign additional processing cores in the host to execute the pre-processing stage; [0085]: graphics cards); and
the programmable neural network unit addressable by cores within the block of graphics cores ( Fig. 2; [0031]: the combination of interfacing a neural network accelerator 165, a neural network application, FPGA, and programmable logic is the programmable neural network unit; GPU and FPGA; Fig. 2; [0032]: the host 105 includes a processor 110 and a memory 115; the processor 110 represents any number of processing elements that each includes any number of processing cores; Fig. 2; [0033]: the memory 115 includes the neural network application 120 executed by the processor 110; the neural network application 120 establishes the neural network with any number of layers;
PNG
media_image2.png
479
654
media_image2.png
Greyscale
);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Umuroglu to include compute units are graphics cores; and the programmable neural network unit addressable by cores within the block of graphics cores as taught by Ng. The motivation for doing so would have been to improve the performance of the neural network application; to increase the number of layers in the neural network 100; to execute the neural network application 120 by the processor 110 as taught by Ng in paragraphs [0006], [0030], and [0033].
Umuroglu in view of Ng fails to explicitly disclose:
one or more neural network hardware blocks with a meta-shader neural network, the meta-shader neural network to generate a texture for one of multiple types.
In same field of endeavor, Chen teaches:
one or more neural network hardware blocks with a meta-shader neural network (col. 2, lines 30-40: the multistage NN system may store neural networks as shader programs, i.e. meta-shader neural network, on a memory of a GPU; the weights of the neural networks are stored as shader objects or textures on the memory of the GPU; col. 5, lines 55-65: the multistage NN client 210 stores and executes shaders on the multistage NN server 117, which is configured to implement neural network shaders), the meta-shader neural network to generate a texture for one of multiple types of texture (col. 10, lines 20-30: the training engine 610 manages generating neural network models to generate weights which are stored as object data, e.g., texture data, on GPU memory; neural network models generate texture data; col. 11, lines 54-65: each neural network to be implemented has a shader program stored on the GPU and weights stored as texture data on the GPU; store the output as texture data; col. 12, lines 1-10: shader adds two input textures element by element and stores the result into the output texture; col. 13, lines 36-60: the shader engine 655 applies style transfer using a neural network to the head texture stored in the object memory 660; generate the modified texture; the modified texture is stored as a new texture).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Umuroglu in view of Ng to include one or more neural network hardware blocks with a meta-shader neural network, the meta-shader neural network to generate a texture for one of multiple types as taught by Chen. The motivation for doing so would have been to generate weights which are stored as object data, e.g., texture data, on GPU memory by neural network models; to store the output as texture data; to generate the modified texture by shader engine using a neural network as taught by Chen in col. 10, lines 20-30, col. 11, lines 54-65, and col. 13, lines 36-60.
Umuroglu in view of Ng and Chen fails to explicitly disclose: multiple types of texture are multiple types of terrain.
In same field of endeavor, Santodomingo teaches:
multiple types of texture are multiple types of terrain (Fig. 6; [0041]: various types of terrain classes; [0044]: a suburban terrain class is at a lower priority than an urban terrain class; two types of terrain; [0045]: the selected terrain class and another terrain class are different types; two types of terrain).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Umuroglu in view of Ng and Chen to include multiple types of texture are multiple types of terrain as taught by Santodomingo. The motivation for doing so would have been to improve rendering; to associate different types and sizes of scenery objects with each terrain class as taught by Santodomingo in paragraphs [0035], and [0045-0045].
Regarding to claim 2 (Original), Umuroglu in view of Ng, Chen, and Santodomingo discloses the graphics processor of claim 1, wherein the meta-shader neural network is to generate a texture for one of multiple types of texture based on one or more input parameters to the meta-shader neural network (Chen; col. 11, lines 54-65: each neural network to be implemented has a shader program stored on the GPU and weights stored as texture data on the GPU; store the output as texture data; col. 12, lines 1-10: shader adds two input textures element by element and stores the result into the output texture; col. 13, lines 36-60: the shader engine 655 applies style transfer using a neural network to the head texture stored in the object memory 660; generate the modified texture; the modified texture is stored as a new texture);
Umuroglu in view of Ng, Chen, and Santodomingo further discloses terrain (Santodomingo; Fig. 6; [0041]: various types of terrain classes; [0044]: a suburban terrain class is at a lower priority than an urban terrain class; two types of terrain; [0045]: the selected terrain class and another terrain class are different types; two types of terrain).
Same motivation of claim 1 is applied here.
Regarding to claim 3 (Original), Umuroglu in view of Ng, Chen, and Santodomingo discloses the graphics processor of claim 1, wherein the one or more neural network hardware blocks include a source data buffer, a neural network operations and activation operations block, and an output data buffer (Umuroglu; Fig. 1; [0027]: the programmable IC 118 includes circuits to facilitate operation of the layer circuits 120, including memory circuit(s) 128 and a control circuit 130; memory circuits 128 stores binary inputs and binary outputs for the layer circuits 120; Fig. 7; [0037]: the control circuit 130 loads a first set of binary weights and a first set of threshold values corresponding to a first layer of the binary neural network 101 into the memory circuits 128 for use by the layer circuits 120; the layer circuits 120 process the binary inputs thereto; the control circuit 130 stores the outputs of the layer circuits 120 in the memory circuits 128; the binary outputs of the layer circuits 120 are provided as the final outputs).
Regarding to claim 4 (Original), Umuroglu in view of Ng, Chen, and Santodomingo discloses the graphics processor of claim 3, wherein the neural network operations and activation operations block is programmably configurable (Umuroglu; [0021]: weights and thresholds are infrequently updated and configured; the weights and thresholds are configured to generate specialized hardware neurons; Fig. 1; [0023]: in a programmable integrated circuit, each layer 102 includes neurons 104 having inputs, and outputs; Fig. 3; [0030]: the count circuit 304 is configured to count the number of the X logic signals output by the XNOR circuit 302 that have a predefined logic state; Fig. 6; [0036]: in a the programmable IC 118, the layer circuit 120 is configured to implement the layer 102-1, then the layer 102-2, and so on until implementing the layer 102-n; the programmable IC 118 includes more than one layer circuit 120; Fig. 7; [0037]: using the programmable IC 118; the layer circuits 120 process the binary inputs thereto).
Regarding to claim 5 (Original), Umuroglu in view of Ng, Chen, and Santodomingo discloses the graphics processor of claim 4, wherein the programmable neural network unit includes a block programming unit (Umuroglu; Fig. 1; [0023]: a programmable integrated circuit; Fig. 1; [0026]: the programmable hardware implementation of the binary neural network 101 includes layer circuits 120) to configure layer state information for the one or more neural network hardware blocks (Umuroglu; [0021]: weights and thresholds are infrequently updated; the weights and thresholds are configured to generate specialized hardware neurons; Fig. 2; [0028]: data inputs of each of the hardware neurons 202 receive logic signals supplying binary inputs; the binary inputs are a set of the binary inputs 110 or a set of activations from a previous layer; receive X logic signals;
PNG
media_image6.png
320
530
media_image6.png
Greyscale
; [0029]: each neuron 202 receives the same or a different set of binary weights.), the layer state information associated with one or more layers of a neural network to be processed by the programmable neural network unit (Umuroglu; [0021]: weights and thresholds are infrequently updated; the weights and thresholds are used to generate specialized hardware neurons; Fig. 1; [0023]: each neuron 104 supplies binary output 116 through an activation 108; [0029]: each neuron 202 receives the same or a different set of binary weights).
Regarding to claim 11 (Original), Umuroglu discloses a method ([0019]: the neural networks classify incoming data sets, e.g., images; a programmable integrated circuit, FPGA and GPUs are graphic processor; Fig. 1; [0023]: a programmable integrated circuit; [0037]: the control circuit 130 loads a next set of binary weights and a next set of threshold values corresponding to a next layer of the binary neural network 101 into the memory circuit(s) 120 for use by the layer circuit(s) 120; [0039]: digital signal processing blocks; some FPGAs also include dedicated processor blocks; [0043]: a complete programmable processing system of microprocessors; microprocessors are a graphics processor) comprising:
The rest claim limitations are similar to claim limitations recited in claim 1 and claim 2. Therefore, same rational used to reject claim 1 and claim 2 is also used to reject claim 11.
Regarding to claim 12 (Original), Umuroglu in view of Ng, Chen, and Santodomingo discloses the method of claim 11,
The claim limitations are similar to claim limitations recited in claim 3 and claim 4. Therefore, same rational used to reject claim 3 and claim 4 are also used to reject claim 12.
Regarding to claim 13 (Original), Umuroglu in view of Ng, Chen, and Santodomingo discloses the method of claim 12,
The claim limitations are similar to claim limitations recited in claim 5. Therefore, same rational used to reject claim 5 is also used to reject claim 13.
Regarding to claim 14 (Original), Umuroglu in view of Ng, Chen, and Santodomingo discloses the method of claim 13,
The rest claim limitations are similar to claim limitations recited in claim 8. Therefore, same rational used to reject claim 8 is also used to reject claim 14.
Regarding to claim 15 (Original), Umuroglu discloses a data processing system ([0019]: the neural networks classify incoming data sets, e.g., images; a programmable integrated circuit, FPGA and GPUs are graphic processor; Fig. 1; [0023]: a programmable integrated circuit; [0037]: the control circuit 130 loads a next set of binary weights and a next set of threshold values corresponding to a next layer of the binary neural network 101 into the memory circuit(s) 120 for use by the layer circuit(s) 120; [0039]: digital signal processing blocks; some FPGAs also include dedicated processor blocks; [0043]: a complete programmable processing system of microprocessors; microprocessors are a graphics processor) comprising:
a memory device (Fig. 1; [0027]: the programmable IC 118 includes circuits to facilitate operation of the layer circuits 120, and also includes memory circuits 128 and a control circuit 130); and
an accelerator device coupled with the memory device, the accelerator device includes a processing cluster and circuitry including a programmable neural network unit, the programmable neural network unit (Fig. 1; [0027]: the programmable IC 118 includes circuits to facilitate operation of the layer circuits 120, including memory circuits 128 and a control circuit 130;
PNG
media_image7.png
442
605
media_image7.png
Greyscale
; Fig. 7; [0037]: the control circuit 130 loads a first set of binary weights and a first set of threshold values corresponding to a first layer of the binary neural network 101 into the memory circuit(s) 128 for use by the layer circuits 120), the graphics processor including :
the rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject claim 15.
Regarding to claim 16 (Original), Umuroglu in view of Ng, Chen, and Santodomingo discloses the data processing system of claim 15,
The rest claim limitations are similar to claim limitations recited in claim 3. Therefore, same rational used to reject claim 3 is also used to reject claim 16.
Regarding to claim 17 (Original), Umuroglu in view of Ng, Chen, and Santodomingo discloses the data processing system of claim 16,
The rest claim limitations are similar to claim limitations recited in claim 4. Therefore, same rational used to reject claim 4 is also used to reject claim 17.
Regarding to claim 18 (Original), Umuroglu in view of Ng, Chen, and Santodomingo discloses the data processing system of claim 17,
The rest claim limitations are similar to claim limitations recited in claim 5. Therefore, same rational used to reject claim 5 is also used to reject claim 18.
Claims 6-8 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Umuroglu (US 20180039886 A1) in view of Ng (US 20190114533 A1), in view of Chen (US 10482565 B1), in view of Santodomingo (US 20040263512 A1), and further in view of Vantrease (US 20190294959 A1).
Regarding to claim 6 (Currently Amended), Umuroglu in view of Ng, Chen, and Santodomingo discloses the graphics processor of claim 4, wherein the programmable neural network unit includes a weights buffer to store weights associated with one or more layers of a neural network to be processed by the programmable neural network unit (Umuroglu; Fig. 3; [0027]: memory circuits 128 store binary inputs, binary weights, threshold values, and binary outputs for the layer circuits 120; [0029]: each neuron 202 receives the same or a different set of binary weights; Fig. 7; [0037]: loads a first set of binary weights and a first set of threshold values corresponding to a first layer of the binary neural network 101 into the memory circuits 128 for use by the layer circuit(s) 120).
Umuroglu in view of Ng, Chen, and Santodomingo fails to explicitly disclose a weights cache and cache weights.
In same field of endeavor, Vantrease teaches a weights cache and cache weights ([0045]: provide caching of data used for computations at computing engine 324; the data is cached at state buffer 322; the caching reduces the effect of memory access bottleneck; pre-fetch a set of weights to computing engine 324; Fig. 3A; [0047]: state buffer 322 may pre-fetch and cache a set of weights for one neural network layer; [0097]: field programmable gate arrays, i.e., FPGAs; level 1 (L1) caches, and level 2 (L2) caches).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Umuroglu in view of Ng, Chen, and Santodomingo to include a weights cache and cache weights as taught by Vantrease. The motivation for doing so would have been to reduce the effect of memory access bottleneck on the performance of computing engine 324; to pre-fetch and cache a set of weights for one neural network layer as taught by Vantrease in paragraphs [0045] and [0047].
Regarding to claim 7 (Original), Umuroglu in view of Ng, Chen, Santodomingo, and Vantrease discloses the graphics processor of claim 6, wherein the programmable neural network unit includes multiple neural network hardware blocks (Umuroglu; Fig. 1; [0023]: a programmable integrated circuit (IC); the binary neural network includes one or more layers 102; each layer 102 includes neurons 104 having synapses 106, e.g., inputs, and activations 108,e.g., outputs; Fig. 1; [0026]: the hardware implementation of the binary neural network 101 includes layer circuits 120; each layer circuit 120 includes input connections 122, hardware neurons 124, and output connections 126; Fig. 1; [0027]: the programmable IC 118 includes circuits to facilitate operation of the layer circuit(s) 120, including memory circuit(s) 128 and a control circuit 130).
Regarding to claim 8 (Original), Umuroglu in view of Ng, Chen, Santodomingo, and Vantrease discloses the graphics processor of claim 7, wherein multiple neural network hardware blocks are respectively associated with one or more layers of the neural network to be processed by the programmable neural network unit (Umuroglu; Fig. 1; [0023]: a programmable integrated circuit (IC); the binary neural network includes one or more layers 102; each layer 102 includes neurons 104 having synapses 106, e.g., inputs, and activations 108, e.g., outputs; Fig. 6; [0036]: the binary neural network 101 includes n layers 102-1 through 102-n; Fig. 7; [0037]: using the programmable IC 118, the layer circuits 120 process the binary inputs thereto).
Regarding to claim 19 (Currently Amended), Umuroglu in view of Ng, Chen, and Santodomingo discloses the data processing system of claim 17,
The rest claim limitations are similar to claim limitations recited in claim 6. Therefore, same rational used to reject claim 6 is also used to reject claim 19.
Regarding to claim 20 (Original), Umuroglu in view of Ng, Chen, and Santodomingo discloses the data processing system of claim 19,
The rest claim limitations are similar to claim limitations recited in claim 7 and claim 8. Therefore, same rational used to reject claim 7 and claim 8 are also used to reject claim 20.
Claims 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Umuroglu (US 20180039886 A1) in view of Ng (US 20190114533 A1), in view of Chen (US 10482565 B1), in view of Santodomingo (US 20040263512 A1), and further in view of Hakura (US 20190236829 A1).
Regarding to claim 9 (Original), Umuroglu in view of Ng, Chen, and Santodomingo discloses the graphics processor of claim 1, wherein the programmable neural network unit is to determine visibility for a geometry operation via the neural network hardware block and the neural network hardware block is to determine visibility on a per-object basis (Chen; col. 10, lines 35-55: the multistage NN server 117 comprises an interface engine 650, a shader engine 655, and an object memory 660; a fragment shader processes fragments, i.e. geometry operation, to generate one or more colors and a depth value, i.e. related to visibility, as fragment outputs; Vertex shaders are shaders in a render pipeline that render vertices of a scene; Fig. 12; col. 14, lines 33-45: once the shader processes of the multistage NN server 117 are complete, the interface engine 650 transmits the modified image 1200 to the display engine 625 for display;
PNG
media_image3.png
504
368
media_image3.png
Greyscale
; the visibility is determined as illustrated in Fig. 12).
Umuroglu in view of Ng, Chen, and Santodomingo fails to explicitly disclose:
a geometry culling operation.
In same field of endeavor, Hakura teaches:
to determine visibility for a geometry culling operation and to determine visibility on a per-object basis ([0040]: image rendering operations; tessellation shader, vertex shader, geometry shader, and/or pixel/fragment shading programs; [0067]: the rasterizer 385 is configured to perform z culling and other z-based optimizations; [0072]: the graphics processing pipeline 320 includes cull, and clip unit (VPC); [0079]: the mesh shader 330 has culled all the graphics primitives processed by the mesh shader 330; [0100]: hierarchical culling; [0158]: the graphics processing pipeline performs primitive culling operations; [0177]: the second plurality of execution threads performs one or more culling operations on a plurality of graphics primitives specified via the first indirect index buffer and the first vertex buffer).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Umuroglu in view of Ng, Chen, and Santodomingo to include to determine visibility for a geometry culling operation and to determine visibility on a per-object basis as taught by Hakura. The motivation for doing so would have been to perform z culling and other z-based optimizations; to perform primitive culling operations as taught by Hakura in paragraphs [0067] and [0158].
Regarding to claim 10 (Original), Umuroglu in view of Ng, Chen, and Santodomingo discloses the graphics processor of claim 1, further comprising to configure the programmable neural network unit (Umuroglu; [0021]: weights and thresholds are infrequently updated; the weights and thresholds are used to generate specialized hardware neurons; Fig. 1; [0023]: in a programmable integrated circuit, i.e. programmable IC, each layer 102 includes neurons 104 having synapses 106, e.g., inputs, and activations 108, e.g., outputs; Fig. 6; [0036]: in a the programmable IC 118, the layer circuit 120 is configured to implement the layer 102-1, then the layer 102-2, and so on until implementing the layer 102-n; the programmable IC 118 includes more than one layer circuit 120; Fig. 7; [0037]: using the programmable IC 118; the layer circuit(s) 120 process the binary inputs thereto).
Umuroglu in view of Ng, Chen, and Santodomingo fails to explicitly disclose:
a tessellation module to generate tessellated output based on coarse input data.
In same field of endeavor, Hakura teaches a tessellation module to generate tessellated output based on coarse input data ([0040]: image rendering operations include tessellation shader, vertex shader, geometry shader, and/or pixel/fragment shading programs, general compute operations, etc.; [0094]: the task shader 650 performs read/load operations, general compute operations, vertex shading operations, tessellation operations, geometry shading operations, and write/store operations; [0100]: programmable tessellation patterns).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Umuroglu in view of Ng, Chen, and Santodomingo to include a tessellation module to generate tessellated output based on coarse input data as taught by Hakura. The motivation for doing so would have been to perform z culling and other z-based optimizations; to perform primitive culling operations as taught by Hakura in paragraphs [0067] and [0158].
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hai Tao Sun whose telephone number is (571)272-5630. The examiner can normally be reached 9:00AM-6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 5712727642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAI TAO SUN/Primary Examiner, Art Unit 2616
Prosecution Insights