DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Rejections - 35 USC § 103
1 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
2 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
3 Claim(s) 1-2, 4, 7-10, 13-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kakarlapudi et al. (US 20170316601 A1) in view of Paluri et al. (US 20190197667 A1).
4 Regarding claim 1, Kakarlapudi teaches a method comprising:
generating first fragment information for an image, wherein the first fragment information has a first fragment resolution; rendering, using the first fragment information, the image, wherein the image has a first image resolution ([0080] reciting “In an embodiment, the step of rendering respective images representing the different resolutions of the scene in an embodiment comprises identifying from the geometry list prepared at the first (e.g. highest) resolution for a rendering tile, the geometry (e.g. primitives) to be processed for the rendering tile, performing any geometry setup operations (e.g. primitive setup operations) necessary for processing the identified geometry, rasterising the identified geometry to graphics fragments for rendering, and then rendering (shading) the graphics fragments to provide output rendered graphics fragment data for the geometry for the tile.”);
generating second fragment information for the image, wherein the second fragment information has a second fragment resolution ([0105] reciting “…in an embodiment, the tile is generated at the, e.g., first resolution, and then generated at the, e.g., second resolution, before moving on to generating the next tile (which may then again, e.g., be generated at the first resolution, followed by the second resolution), and so on.”); and
generating, , using the image and the second fragment information, a transformed image, wherein the transformed image has a second image resolution ([0105] reciting “For example, where for a given tile, two different resolution images are to be generated, in an embodiment, the tile is generated at the, e.g., first resolution, and then generated at the, e.g., second resolution, before moving on to generating the next tile (which may then again, e.g., be generated at the first resolution, followed by the second resolution), and so on. This can help to improve the rendering efficiency, as it may avoid, e.g., the need to repeatedly reload the same geometry lists for a tile.”).
5 Kakarlapudi does not explicitly teach generating, using at least one machine learning model…
6 Paluri teaches generating, using at least one machine learning model ([0004] reciting “Disclosed herein are systems and methods for computing a high-resolution depth image (i.e., an upsampled depth image) of an input image having depth information captured at a lower resolution using a trained depth prediction model, e.g., using the trained depth prediction model to upsample the depth information captured at the lower resolution.”; [0028] reciting “In one embodiment, to generate a high resolution depth image, the computing device 150 may include a transformation module 125, a feature extraction module 130, a model application module 135, a scene reconstruction module 140, a model training module 145, as well as a training data store 175.”)…
7 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi) to incorporate the teachings of Paluri to provide a method to include a machine learning model for generating transformed images, utilizing the generation type methods provided by Kakarlapudi. Doing so would allow the models to be trained to minimize an error between the higher resolution depth image and the depth information as stated by Paluri ([Abstract] recited).
8 Regarding claim 2, Kakarlapudi in view of Paluri teaches the method of claim 1 (see claim 1 rejection above), wherein the second fragment information comprises at least one of depth information, albedo information, normal information, or specular information ([0080] reciting “…and then rendering (shading) the graphics fragments to provide output rendered graphics fragment data for the geometry for the tile. The rendering process can, and in an embodiment does, also include any other processing that the graphics processor or the graphics processing pipeline may perform when rendering geometry for output (e.g. for display), such as performing hidden surface removal operations, depth testing, stencil testing, blending, texture mapping, etc.”; [0203] reciting “During a normal graphics rendering operation, the renderer will modify the (e.g.) colour (red, green and blue, RGB) and transparency (alpha, a) data associated with each fragment so that the fragments can be displayed correctly.”).
9 Regarding claim 4, Kakarlapudi in view of Paluri teaches the method of claim 1, wherein (see claim 1 rejection above): firstsecondsecond
10 Paluri from claim 1 can further teach the limitations, specifically the image is applied to a first portion of the machine learning model; and the second fragment information is applied to a second portion of the machine learning model ([0012] reciting “The machine learning model is trained to accurately generate a high resolution depth image. During the real-time phase, the trained machine learning model can be applied to a depth image with an initial resolution and subsequently generates an upsampled depth image with a resolution greater than the initial resolution.”).
11 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi in view of Paluri) to incorporate additional teachings of Paluri to provide a method that incorporates getting a type of portion from a machine learning model, utilizing the first and second images and fragment information provided by Kakarlapudi in view of Paluri. Doing so would allow the models to be trained to minimize an error between the higher resolution depth image and the depth information as stated by Paluri ([Abstract] recited).
12 Regarding claim 7, Kakarlapudi in view of Paluri teaches the method of claim 1 (see claim 1 rejection above), wherein: the second fragment information comprises channel information (Kakarlapudi; [0105] reciting “For example, where for a given tile, two different resolution images are to be generated, in an embodiment, the tile is generated at the, e.g., first resolution, and then generated at the, e.g., second resolution, before moving on to generating the next tile (which may then again, e.g., be generated at the first resolution, followed by the second resolution), and so on.”; [0203] reciting “During a normal graphics rendering operation, the renderer will modify the (e.g.) colour (red, green and blue, RGB) and transparency (alpha, a) data associated with each fragment so that the fragments can be displayed correctly.”);
13 Paluri from claim 1 can further teach the limitations, specifically the machine learning model is configured to process a portion of the channel information ([0059] reciting “The trained model 250 outputs a combined image 275 that, as depicted in FIG. 2, includes four image channels. As an example, the four image channels may be a red color image 260, a green color image 262, a blue color image 264, and an upsampled depth image 266.”).
14 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi in view of Paluri) to incorporate additional teachings of Paluri to provide a method to configure a type of portion of a channel, utilizing the fragment information provided by Kakarlapudi in view of Paluri. Doing so would allow the models to be trained to minimize an error between the higher resolution depth image and the depth information as stated by Paluri ([Abstract] recited).
15 Regarding claim 8, Kakarlapudi in view of Paluri teaches the method of claim 1 (see claim 1 rejection above), second fragment information ([0105] reciting “For example, where for a given tile, two different resolution images are to be generated, in an embodiment, the tile is generated at the, e.g., first resolution, and then generated at the, e.g., second resolution, before moving on to generating the next tile (which may then again, e.g., be generated at the first resolution, followed by the second resolution), and so on.”)
16 Paluri from claim 1 can further teach the limitations, specifically wherein the machine learning model is configured to process the second fragment information for a portion of the image ([0049] reciting “Therefore, the model training module 145 provides the feature vectors as input to the machine learning model. In some embodiments, the received feature vectors may derive from color images and low resolution depth images that have undergone a transformation process to eliminate positional offsets, as described previously in reference to the transformation module 125. In another embodiment, the color information and low resolution depth information provided as input to the machine learning model are the images themselves e.g., color images captured by the color sensors 110 and low resolution depth images captured by the depth sensor 115.”).
17 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi in view of Paluri) to incorporate additional teachings of Paluri to provide a type of information to process from a type of image portion, utilizing the type of second fragment information provided by Kakarlapudi. Doing so would allow the models to be trained to minimize an error between the higher resolution depth image and the depth information as stated by Paluri ([Abstract] recited).
18 Regarding claim 9, Kakarlapudi in view of Paluri teaches the method of claim 8 (see claims 1 and 8 rejections above), second fragment information ([0105] reciting “For example, where for a given tile, two different resolution images are to be generated, in an embodiment, the tile is generated at the, e.g., first resolution, and then generated at the, e.g., second resolution, before moving on to generating the next tile (which may then again, e.g., be generated at the first resolution, followed by the second resolution), and so on.”)
19 Paluri from claim 1 can further teach the limitations, specifically wherein the machine learning model is configured to process the second fragment information using sparse convolution ([0034] reciting “As an example, if the machine learning model is a convolutional neural network (CNN), the CNN can extract features from color images captured by the color sensors 110 and low resolution depth images captured by the depth sensor 115. The CNN convolves the pixels of the low resolution depth images or the color images with a patch (e.g., an N×N patch) to identify convolved features.”).
20 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi in view of Paluri) to incorporate additional teachings of Paluri to provide a type of information to process from a type of image portion as well a type of space convolution, utilizing the type of second fragment information provided by Kakarlapudi. Doing so would allow the models to be trained to minimize an error between the higher resolution depth image and the depth information as stated by Paluri ([Abstract] recited).
21 Regarding claim 10, Kakarlapudi teaches a system comprising:
a graphics processing pipeline configured to render, using first fragment information having a first fragment resolution, an image having a first image resolution ([0080] reciting “In an embodiment, the step of rendering respective images representing the different resolutions of the scene in an embodiment comprises identifying from the geometry list prepared at the first (e.g. highest) resolution for a rendering tile, the geometry (e.g. primitives) to be processed for the rendering tile, performing any geometry setup operations (e.g. primitive setup operations) necessary for processing the identified geometry, rasterising the identified geometry to graphics fragments for rendering, and then rendering (shading) the graphics fragments to provide output rendered graphics fragment data for the geometry for the tile. The rendering process can, and in an embodiment does, also include any other processing that the graphics processor or the graphics processing pipeline may perform when rendering geometry for output (e.g. for display), such as performing hidden surface removal operations, depth testing, stencil testing, blending, texture mapping, etc.”); and
to generate, using the image and second fragment information, a transformed image having a second image resolution; wherein the second fragment information has a second fragment resolution ([0105] reciting “For example, where for a given tile, two different resolution images are to be generated, in an embodiment, the tile is generated at the, e.g., first resolution, and then generated at the, e.g., second resolution, before moving on to generating the next tile (which may then again, e.g., be generated at the first resolution, followed by the second resolution), and so on. This can help to improve the rendering efficiency, as it may avoid, e.g., the need to repeatedly reload the same geometry lists for a tile.”).
22 Kakarlapudi does not explicitly teach a machine learning model configured to generate …
23 Paluri teaches a machine learning model configured to generate ([0004] reciting “Disclosed herein are systems and methods for computing a high-resolution depth image (i.e., an upsampled depth image) of an input image having depth information captured at a lower resolution using a trained depth prediction model, e.g., using the trained depth prediction model to upsample the depth information captured at the lower resolution.”; [0028] reciting “In one embodiment, to generate a high resolution depth image, the computing device 150 may include a transformation module 125, a feature extraction module 130, a model application module 135, a scene reconstruction module 140, a model training module 145, as well as a training data store 175.”)…
24 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi) to incorporate the teachings of Paluri to provide a method to include a machine learning model for generating transformed images, utilizing the generation type methods provided by Kakarlapudi. Doing so would allow the models to be trained to minimize an error between the higher resolution depth image and the depth information as stated by Paluri ([Abstract] recited).
25 Regarding claim 13, Kakarlapudi in view of Paluri teaches the system of claim 10 (see claim 10 rejection above), wherein the machine learning model operates using a driver.
26 Paluri from claim 10 can further teach the limitations, specifically wherein the machine learning model operates using a driver ([0028] reciting “Thus, a module can be implemented in hardware, firmware, and/or software. In one embodiment, program modules are stored on a storage device of the computing device 150, loaded into a memory of the computing device 150, and executed by a processor of the computing device 150.”; [0033] reciting “During the training phase, the feature extraction module 130 can provide a feature vector for each image (e.g., each color image and each depth image) to the model training module 145 to be provided as inputs to train a machine learning model. During the real-time phase, the feature extraction module 130 provides a feature vector for each image to the model application module 135 to be provided as inputs to the trained machine learning model.”).
27 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi in view of Paluri) to incorporate additional teachings of Paluri to provide a type of driver for the machine learning models that are provided from Kakarlapudi in view of Paluri. Doing so would allow the models to be trained to minimize an error between the higher resolution depth image and the depth information as stated by Paluri ([Abstract] recited).
28 Regarding claim 14, Kakarlapudi in view of Paluri teaches the system of claim 10 (see claim 10 rejection above), wherein the second fragment information is generated using the graphics processing pipeline (Kakarlapudi; [0080] reciting “The rendering process can, and in an embodiment does, also include any other processing that the graphics processor or the graphics processing pipeline may perform when rendering geometry for output (e.g. for display), such as performing hidden surface removal operations, depth testing, stencil testing, blending, texture mapping, etc.”; [0105] reciting “For example, where for a given tile, two different resolution images are to be generated, in an embodiment, the tile is generated at the, e.g., first resolution, and then generated at the, e.g., second resolution, before moving on to generating the next tile (which may then again, e.g., be generated at the first resolution, followed by the second resolution), and so on. This can help to improve the rendering efficiency, as it may avoid, e.g., the need to repeatedly reload the same geometry lists for a tile.”; [0179] reciting “Any programmable, shading stages of the graphics processing pipeline such as the vertex shader, fragment shader, etc., can be implemented as desired and in any suitable manner, and can perform any desired and suitable shading, e.g. vertex shading, fragment shading, etc., functions, respectively and as appropriate.”).
29 Regarding claim 15, Kakarlapudi in view of Paluri teaches the system of claim 14 (see claims 10 and 14 rejections above), wherein: the first fragment information is generated using a first pass of the graphics processing pipeline; and the second fragment information is generated using a second pass of the graphics processing pipeline (Kakarlapudi; [0105] reciting “For example, where for a given tile, two different resolution images are to be generated, in an embodiment, the tile is generated at the, e.g., first resolution, and then generated at the, e.g., second resolution, before moving on to generating the next tile (which may then again, e.g., be generated at the first resolution, followed by the second resolution), and so on. This can help to improve the rendering efficiency, as it may avoid, e.g., the need to repeatedly reload the same geometry lists for a tile.”; [0118] reciting “Thus, in an embodiment, the different resolution images in the set of plural images that are rendered are treated as respective texture maps to be input to the graphics processor, and the graphics processor then performs a rendering pass where it uses those texture maps to provide the overall final, “foveated”, output image, e.g. to be displayed.”).
30 Regarding claim 16, Kakarlapudi in view of Paluri teaches the system of claim 10 (see claim 10 rejection above), wherein the second fragment information is stored in a buffer (Kakarlapudi; [0105] reciting “For example, where for a given tile, two different resolution images are to be generated, in an embodiment, the tile is generated at the, e.g., first resolution, and then generated at the, e.g., second resolution, before moving on to generating the next tile (which may then again, e.g., be generated at the first resolution, followed by the second resolution), and so on. This can help to improve the rendering efficiency, as it may avoid, e.g., the need to repeatedly reload the same geometry lists for a tile.”; [0182] reciting “In an embodiment, the various functions of the technology described herein are carried out on a single graphics processing platform that generates and outputs the rendered fragment data that is, e.g., written to a frame buffer for a display device.”).
31 Regarding claim 17, Kakarlapudi in view of Paluri teaches the system of claim 10 (see claim 10 rejection above), : a first portion configured to process at least a portion of the second fragment information; and a second portion configured to generate, using the image and an output of the first portion, the transformed image (Kakarlapudi; [0047] reciting “In another embodiment, however, the render target sub-regions may be sized such that they correspond to portions of rendering tiles. For example, the sub-regions may be sized such that they to correspond to (i.e. encompass) half a tile, or a quarter of a tile, or one and a half tiles, etc.”; [0105] reciting “For example, where for a given tile, two different resolution images are to be generated, in an embodiment, the tile is generated at the, e.g., first resolution, and then generated at the, e.g., second resolution, before moving on to generating the next tile (which may then again, e.g., be generated at the first resolution, followed by the second resolution), and so on. This can help to improve the rendering efficiency, as it may avoid, e.g., the need to repeatedly reload the same geometry lists for a tile.”).
32 Paluri from claim 10 can further teach the limitations, specifically wherein the machine learning model comprises: a first portion configured to process at least a portion of the second fragment information; and a second portion configured to generate, using the image and an output of the first portion, the transformed image ([0012] reciting “The machine learning model is trained to accurately generate a high resolution depth image. During the real-time phase, the trained machine learning model can be applied to a depth image with an initial resolution and subsequently generates an upsampled depth image with a resolution greater than the initial resolution.”; [0028] reciting “In one embodiment, to generate a high resolution depth image, the computing device 150 may include a transformation module 125, a feature extraction module 130, a model application module 135, a scene reconstruction module 140, a model training module 145, as well as a training data store 175.”).
33 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi in view of Paluri) to incorporate additional teachings of Paluri to provide a method to incorporate a machine learning model in relation to finding a type of portion(s) based on certain fragment information that is provided by Kakarlapudi in view of Paluri. Doing so would allow the models to be trained to minimize an error between the higher resolution depth image and the depth information as stated by Paluri ([Abstract] recited).
34 Regarding claim 18, Kakarlapudi teaches a method comprising:
generating fragment information for an image, wherein the image has a first resolution, and the fragment information has a second resolution ([0080] reciting “In an embodiment, the step of rendering respective images representing the different resolutions of the scene in an embodiment comprises identifying from the geometry list prepared at the first (e.g. highest) resolution for a rendering tile, the geometry (e.g. primitives) to be processed for the rendering tile, performing any geometry setup operations (e.g. primitive setup operations) necessary for processing the identified geometry, rasterising the identified geometry to graphics fragments for rendering, and then rendering (shading) the graphics fragments to provide output rendered graphics fragment data for the geometry for the tile.”; [0105] reciting “…in an embodiment, the tile is generated at the, e.g., first resolution, and then generated at the, e.g., second resolution, before moving on to generating the next tile (which may then again, e.g., be generated at the first resolution, followed by the second resolution), and so on.”); and
generating, , using the image and the fragment information, a transformed image ([0062] reciting “The geometry sorting (listing) process should, and in an embodiment does, operate on (use) appropriately transformed (vertex shaded) positions for the vertices of the geometry (e.g. primitives) that are to be rendered.”; [0105] reciting “For example, where for a given tile, two different resolution images are to be generated, in an embodiment, the tile is generated at the, e.g., first resolution, and then generated at the, e.g., second resolution, before moving on to generating the next tile (which may then again, e.g., be generated at the first resolution, followed by the second resolution), and so on. This can help to improve the rendering efficiency, as it may avoid, e.g., the need to repeatedly reload the same geometry lists for a tile.”; [0209] reciting “The vertex shading operation operates to transform the attributes for each vertex into a desired form for the subsequent graphics processing operations.”).
35 Kakarlapudi does not explicitly teach generating, using at least one machine learning model …
36 Paluri teaches generating, using at least one machine learning model ([0004] reciting “Disclosed herein are systems and methods for computing a high-resolution depth image (i.e., an upsampled depth image) of an input image having depth information captured at a lower resolution using a trained depth prediction model, e.g., using the trained depth prediction model to upsample the depth information captured at the lower resolution.”; [0028] reciting “In one embodiment, to generate a high resolution depth image, the computing device 150 may include a transformation module 125, a feature extraction module 130, a model application module 135, a scene reconstruction module 140, a model training module 145, as well as a training data store 175.”)…
37 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi) to incorporate the teachings of Paluri to provide a method to include a machine learning model for generating transformed images, utilizing the generation type methods provided by Kakarlapudi. Doing so would allow the models to be trained to minimize an error between the higher resolution depth image and the depth information as stated by Paluri ([Abstract] recited).
38 Regarding claim 19, Kakarlapudi in view of Paluri teaches the method of claim 18 (see claim 18 rejection above), wherein the generating the fragment information is performed using a graphics processing pipeline (Kakarlapudi; [0080] reciting “The rendering process can, and in an embodiment does, also include any other processing that the graphics processor or the graphics processing pipeline may perform when rendering geometry for output (e.g. for display), such as performing hidden surface removal operations, depth testing, stencil testing, blending, texture mapping, etc.”; [0179] reciting “Any programmable, shading stages of the graphics processing pipeline such as the vertex shader, fragment shader, etc., can be implemented as desired and in any suitable manner, and can perform any desired and suitable shading, e.g. vertex shading, fragment shading, etc., functions, respectively and as appropriate.”).
39 Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kakarlapudi et al. (US 20170316601 A1) in view of Paluri et al. (US 20190197667 A1) as of claim 1, further in view of Saleh et al. (US 20200202594 A1).
40 Regarding claim 3, Kakarlapudi in view of Paluri teaches the method of claim 1 (see claim 1 rejection above), but does not explicitly teach wherein the rendering is performed at a shading rate corresponding to an image resolution that is lower than the first image resolution.
41 Saleh teaches wherein the rendering is performed at a shading rate corresponding to an image resolution that is lower than the first image resolution ([0037] reciting “The shading rate may be one of a sub-sample shading rate, a one-to-one shading rate, or a super-sample shading rate. A sub-sample shading rate means that the resolution of pixel shading is lower than the resolution of the render target (but not the resolution of the samples). A one-to-one shading rate means that the resolution of pixel shading is the same as the resolution of the render target.”).
42 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi in view of Paluri) to incorporate the teachings of Saleh to provide a method that incorporates a shading rate for a resolution to be lower than another resolution, utilizing the first resolutions and shading methods provided by Kakarlapudi in view of Paluri. Doing so would determine one or more shading rates as stated by Saleh ([0037] recited).
43 Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kakarlapudi et al. (US 20170316601 A1) in view of Paluri et al. (US 20190197667 A1) as of claim 1 and 4, further in view of Kamenetskaya et al. (US 20210103852 A1).
44 Regarding claim 5, Kakarlapudi in view of Paluri teaches the method of claim 4 (see claims 1 and 4 rejection above), but does not explicitly teach wherein the second portion of the machine learning model processes at least a portion of the second fragment information in parallel with the rendering.
45 Kamenetskaya teaches wherein the second portion of the machine learning model processes at least a portion of the second fragment information in parallel with the rendering ([Abstract] reciting “Methods, systems, and devices for workload balancing for machine learning are described. Generally, a device may determine a size of a level one cache of a texture processor, identify a portion of input activation data for an iterative machine-learning process, and load the portion of input activation data into the level one cache. The device may allocate, based at least in part on a texture processor to shading processor arithmetic logic unit (ALU) resource ratio”).
46 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi in view of Paluri) to incorporate the teachings of Kamenetskaya to provide a method that renders parallelly between certain types of portions from a machine learning model, utilizing the second portions and models provided by Kakarlapudi in view of Paluri (Kakarlapudi; [0105] reciting “For example, where for a given tile, two different resolution images are to be generated, in an embodiment, the tile is generated at the, e.g., first resolution, and then generated at the, e.g., second resolution, before moving on to generating the next tile (which may then again, e.g., be generated at the first resolution, followed by the second resolution), and so on.”). Doing so would allow workload balancing as stated by Kamenetskaya ([Abstract] recited).
47 Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kakarlapudi et al. (US 20170316601 A1) in view of Paluri et al. (US 20190197667 A1) as of claim 1 and 4, further in view of Bourd et al. (US 20250238996 A1).
48 Regarding claim 6, Kakarlapudi in view of Paluri teaches the method of claim 4 (see claims 1 and 4 rejections above), wherein an output of the second portion of the machine learning model has a lower dimensionality than the second fragment information.
49 Bourd teaches wherein an output of the second portion of the machine learning model has a lower dimensionality than the second fragment information ([0012] reciting “In one example, the method further includes establishing the initial feature field neural network and the initial opacity field neural network. In one example, the method further includes ingesting the plurality of two-dimensional (2D) images for machine learning (ML) training.”; [0039] reciting “In one example, the latent representation of an entity has a lower dimensionality (e.g., simpler representation) that the entity itself. That is, the latent representation may use simpler elements (e.g., triangle meshes, textures, etc.) to represent the entity.”).
50 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi in view of Paluri) to incorporate the teachings of Bourd to provide a method to determine a lower dimensionality based on types of portions, utilizing the second fragment information provided by Kakarlapudi in view of Paluri. Doing so would allow a type of latent representation to use simpler elements (e.g., triangle meshes, textures, etc.) to represent certain types of entities as stated by Bourd ([0039] recited).
51 Claim(s) 11-12 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kakarlapudi et al. (US 20170316601 A1) in view of Paluri et al. (US 20190197667 A1) as of claim 1 and 4, further in view of Calidas et al. (US 20220058476 A1).
52 Regarding claim 11, Kakarlapudi in view of Paluri teaches the system of claim 10 (see claim 10 rejection above), wherein the using the graphics processing pipeline (Kakarlapudi; [0204] reciting “FIG. 2 shows the main elements and pipeline stages of the graphics processor 3 that are relevant to the operation of the present embodiment. As will be appreciated by those skilled in the art there may be other elements and stages of the graphics processor (graphics processing pipeline) that are not illustrated in FIG. 2.”).
53 Although Kakarlapudi in view of Paluri could teach wherein the machine learning model operates using the graphics processing pipeline (specifically the machine learning model) (Paluri; [0004] reciting “Disclosed herein are systems and methods for computing a high-resolution depth image (i.e., an upsampled depth image) of an input image having depth information captured at a lower resolution using a trained depth prediction model, e.g., using the trained depth prediction model to upsample the depth information captured at the lower resolution.”), prior art from Calidas can further teach the limitations.
54 Calidas teaches wherein the machine learning model operates using the graphics processing pipeline ([0043] reciting “According to an aspect, the graphics processing pipeline 107 of the processing unit 120 includes one or more arithmetic logic units (ALUs), and specifically one or more graphics texture ALUs that are configured to perform texture filtering to determine texture colors for texture mapped pixels based on colors of nearby texels (i.e., pixels of the texture). As will be described in detail below, the processing unit 120 can execute machine-learning shaders that may cause the one or more graphics texture ALUs to perform machine-learning operations according to an exemplary aspect.”).
55 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi in view of Paluri) to incorporate the teachings of Calidas to provide a clearer method where the machine learning model operates with a graphics processing pipeline, utilizing the models and the pipeline methods provided by Kakarlapudi in view of Paluri. Doing so would allow methods like perform texture filtering to determine texture colors for texture mapped pixels based on colors of nearby texels as stated by Calidas ([0043] recited).
56 Regarding claim 12, Kakarlapudi in view of Paluri and Calidas teaches the system of claim 11 (see claims 10-11 rejections above),
57 Calidas from claim 11 can further teach the limitations, specifically wherein the machine learning model operates using a shader in the graphics processing pipeline ([0043] reciting “According to an aspect, the graphics processing pipeline 107 of the processing unit 120 includes one or more arithmetic logic units (ALUs), and specifically one or more graphics texture ALUs that are configured to perform texture filtering to determine texture colors for texture mapped pixels based on colors of nearby texels (i.e., pixels of the texture). As will be described in detail below, the processing unit 120 can execute machine-learning shaders that may cause the one or more graphics texture ALUs to perform machine-learning operations according to an exemplary aspect.”).
58 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi in view of Paluri) to incorporate the teachings of Calidas to provide a clearer method where the machine learning model operates with a type of shader in a graphics processing pipeline, utilizing the models and the pipeline methods provided by Kakarlapudi in view of Paluri, which can also provide a specific shader (Kakarlapudi; [0179] reciting “Any programmable, shading stages of the graphics processing pipeline such as the vertex shader, fragment shader, etc., can be implemented as desired and in any suitable manner, and can perform any desired and suitable shading, e.g. vertex shading, fragment shading, etc., functions, respectively and as appropriate.”). Doing so would allow methods like perform texture filtering to determine texture colors for texture mapped pixels based on colors of nearby texels as stated by Calidas ([0043] recited).
59 Regarding claim 20, Kakarlapudi in view of Paluri teaches the method of claim 18 (see claim 18 rejection above), wherein the using a graphics processing pipeline.
60 Although Kakarlapudi in view of Paluri could teach wherein the machine learning model operates using a graphics processing pipeline (specifically the machine learning model) (Paluri; [0004] reciting “Disclosed herein are systems and methods for computing a high-resolution depth image (i.e., an upsampled depth image) of an input image having depth information captured at a lower resolution using a trained depth prediction model, e.g., using the trained depth prediction model to upsample the depth information captured at the lower resolution.”), prior art from Calidas can further teach the limitations.
61 Calidas teaches wherein the machine learning model operates using a graphics processing pipeline ([0043] reciting “According to an aspect, the graphics processing pipeline 107 of the processing unit 120 includes one or more arithmetic logic units (ALUs), and specifically one or more graphics texture ALUs that are configured to perform texture filtering to determine texture colors for texture mapped pixels based on colors of nearby texels (i.e., pixels of the texture). As will be described in detail below, the processing unit 120 can execute machine-learning shaders that may cause the one or more graphics texture ALUs to perform machine-learning operations according to an exemplary aspect.”).
62 It would have been obvious to one with ordinary skill before the effective filing date of the claimed invention, to have modified the method (taught by Kakarlapudi in view of Paluri) to incorporate the teachings of Calidas to provide a clearer method where the machine learning model operates with a graphics processing pipeline, utilizing the models and the pipeline methods provided by Kakarlapudi in view of Paluri. Doing so would allow methods like perform texture filtering to determine texture colors for texture mapped pixels based on colors of nearby texels as stated by Calidas ([0043] recited).
Conclusion
63 Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNNY TRAN LE whose telephone number is (571)272-5680. The examiner can normally be reached Mon-Thu: 7:30am-5pm; First Fridays Off; Second Fridays: 7:30am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHNNY T LE/ Examiner, Art Unit 2614
/KENT W CHANG/ Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights