Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 5/28/2026 has been entered.
Response to Amendment
This action is in response to the amendment filed on May 28th, 2026. Claims 1,10, and 17 have been amended. Applicant’s amendments to the claims have overcome the 35 USC 103 rejections as set forth in the final office action mailed April 4th, 2026. Applicant’s amendments to the claims have overcome the 35 USC 103 rejections previously set forth, however, new rejections have been issued, as necessitated by amendment.
Response to Arguments
Applicant’s remaining arguments regarding the prior § 103 rejection are moot in view of the amendments, and the Examiner has introduced new grounds of rejections based on new references to address amended limitations.
Regarding arguments to independent claims 10 and 17, they have been amended in an analogous manner to claim 1, and, for the reasons discussed above, the prior § 103 rejections of claims 10 and 17 are not maintained and new grounds of rejection are set forth below.
Applicant’s arguments regarding the dependent claims being in condition for allowance due to the reasons related to the corresponding independent claims are not persuasive because the independent claims are not allowed, therefore the dependent claims remain rejected.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-5, 8-13, 16-18, and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Tuzel (US9836820B2), Hobson (US20220092740A1), Sunkavalli (US20190340810A1), and Sakaguchi (US20110206296A1).
Regarding claim 1, Tuzel teaches a method comprising: receiving, by a processing device, an input having a first resolution (Tuzel; ¶0020, describes a computer system with a processor and memory, Tuzel; ¶0025, describes the image (input) for upsampling being received from the imaging device and, Tuzel; ¶0021, describes the received image as a low-resolution image (first resolution) to be upsampled. This teaches receiving an input, by a processing device, having a first resolution), generating, by the processing device, a bilinearly upsampled texel by isolating a texel from the input and increasing a size and a resolution of the texel; (Tuzel; ¶0028, describes an interpolation-based upsampling stream that produces a smooth upsampled image, the interpolation using fixed weights such as nearest neighbor, bilinear, or cubic interpolation and, Tuzel; ¶0037, describes the interpolation stream mapping one pixel (texel) in the low-resolution input to several pixels in the high-resolution image by multiplying each input pixel by a weight factor initialized bilinearly. This teaches obtaining an input picture element/pixel (isolating a texel from the input) and increasing its size and resolution by bilinear interpolation (increase a size and a resolution of the texel).), generating, by the processing device, an updated texel having a second resolution that is higher than the first resolution based on the bilinearly upsampled texel using a machine learning model trained on training data, (Tuzel; Fig. 3 shows the smooth upsampled image and the global details concatenated and processed by a CNN (machine learning model) to produce a high-resolution output (updated texel having a second resolution. Tuzel; ¶0042, describes the local refinement network as a CNN, Tuzel; ¶0045, describes that the fully connected network, the interpolation, and the convolution are concurrently trained to reduce an error between upsampled images and corresponding high-resolution images (trained on training data), and, Tuzel; ¶0046, describes training using pairs of low-resolution and high-resolution images to produce the network weights. This teaches generating a higher-resolution output from the bilinearly upsampled input using a machine learning model trained on training data, the output resolution being higher than the input resolution.), generating, by the processing device for display in a user interface, an output based on the updated texel having the second resolution, the output having a resolution that is higher than the first resolution (Tuzel; ¶0024 describes a display interface connecting the computer system to a display device (display in a user interface). Tuzel; Figs. 2A and 3 show the high-resolution output image, the higher-resolution output generated by the convolutional network (based on the updated texel having the second resolution), being output for display in a user interface, the output being a higher-resolution image than the input. This teaches generating, for display in a user interface, an output based on the updated texel and having a resolution higher than the first resolution.)
However, Tuzel does not explicitly teach that the input and output are digital material model comprising texels.
Hobson teaches digital material models comprising texels (Hobson; ¶0005, describes computer graphic texture comprising texels and having a first resolution and, Fig. 7 and ¶0005-6, describe generating a texture with a higher resolution using a machine learning engine. This teaches the input and output as digital material models comprising texels).
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify the super-resolution pipeline of Tuzel to the textures (texels) of Hobson because both describe resolution enhancement of sampled image/texture data using machine learning and combining them would yield the predictable improvement of higher-resolution material textures for rendering.
However, Tuzel in view of Hobson does not explicitly disclose rendering, by the processing device, one or more radiance images from the input digital material model, each of the one or more radiance images corresponding to exposure of the input digital material model to a different light position and the machine learning model trained on training data, including the one or more radiance images, to generate texels.
Sunkavalli teaches rendering, by the processing device, one or more radiance images from the input digital material model, each of the one or more radiance images corresponding to exposure of the input digital material model to a different light position (Sunkavalli; ¶0062, describes a light transport function mapping incident illumination to outgoing radiance at a pixel and, ¶0071-0072, describes texturizing synthetic objects using SVBRDF and rendering a set of training digital images in which each training digital image portrays the object illuminated from a different lighting direction. This teaches rendering radiance images from a digital material at different light positions.) and the machine learning model trained on training data, including the one or more radiance images, to generate texels (Sunkavalli; ¶0074, describes providing the set of training digital images to the object relighting neural network and training the network by backpropagation. This teaches training a machine learning model on training data that includes the rendered radiance images.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify the super-resolution pipeline of Tuzel in view of Hobson with the radiance-image-rendering and training of Sunkavalli. The motivation for such a combination would have been to provide the benefit of generating more accurate and higher resolution digital material models.
However, Tuzel in view of Hobson and Sunkavalli does not disclose explicitly that the training data includes upsampled version of the one or more radiance images.
Sakaguchi teaches a machine learning model trained on upsampled versions of input images (Sakaguchi; ¶0007, describes enlarging an input image by a factor N (upsampling) using a pixel interpolation method such as bicubic or spline interpolation and, Sakaguchi; ¶0015, describes enlarging the training low-resolution image by the factor N (upsampled version) to generate an enlarged training image used to produce the training data. This teaches training the model on upsampled versions of the input images.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify the system of Tuzel in view of Hobson and Sunkavalli with the upsampled training images of Sakaguchi. The motivation for such a combination would have been to provide the benefit of improved reconstruction of fine detail in the higher-resolution output.
Claim 10, has similar limitations as of Claim(s) 1, therefore it is rejected under the same rationale as Claim(s) 1, except claim 10 further recites “a memory component”, “a processing device coupled to the memory component”, and that the machine learning model is “trained on training data to generate texels based on radiances corresponding to different light positions”.
Tuzel teaches a system with a memory component that stores instructions and a processor coupled to the memory to perform the operations (pg. 3 Fig. 1, shows a computer system including a processor and memory and, pg. 13 col. 3 lines 7-18, describe the system where the processor is connected to the memory to perform the operations).
However, Tuzel does not explicitly describe training on data based on radiances corresponding to different light positions.
Hobson describes (¶0065) training the machine learning model using ground truth texture blocks and (¶0052) where the textures include physically based rendering (PBR) properties that characterize material light response, including albedo, which characterizes diffuse lighting response, and (¶0055) reflectance which represents specular lighting response, and (¶0051) describes that light is modelled as diffuse or specular with specular lighting being view dependent. This teaches training on texture data that includes radiance information corresponding to different light positions and viewing angles. It would have been obvious to one of ordinary skill in the art, before the effective filing date, to further modify the training approach of Tuzel in view of Hobson and Sunkavalli with the PBR texture training data of Hobson because Hobson explicitly teaches that physically based rendering textures capture material light response properties (albedo, reflectance) that vary with light position and viewing angle, to give the benefit of training a model that accurately produces material appearance under varying lighting conditions.
Claim 17, has similar limitations as of Claims 1 and 10, therefore it is rejected under the same rationale as Claims 1 and 10, except claim 17 further recites “non-transitory computer-readable storage medium storing executable instructions”.
As previously discussed in claim 10, Tuzel discloses a system with a memory component, including RAM, ROM, flash, and other suitable memory types, to store the instructions and a processor coupled to the memory to perform the operations (pg. 3 Fig. 1, shows a computer system including a processor and memory and, pg. 13 col. 3 lines 7-18, describe the system where the processor is connected to the memory to perform the operations). This teaches a non-transitory computer-readable storage medium storing executable instructions.
Regarding claim 4, Tuzel in view of Hobson, Sunkavalli, and Sakaguchi teaches the method of claim 1, wherein the input digital material model having the first resolution includes at least one of a base color map, a normal map, a metallic map, a roughness map, or a height map (Hobson; ¶0005, describes a first data structure representing textures having a first resolution (see claim 1), ¶0052, describes that PBR textures include albedo which characterizes diffuse lighting response and, ¶0053, describes the input includes a normal map that defines surface normal detail. This teaches the input digital material model having the first resolution includes at least a normal map).
Claim 12, has similar limitations as of Claim(s) 4, therefore it is rejected under the same rationale as Claim(s) 4.
Claim 20, has similar limitations as of Claim(s) 4, therefore it is rejected under the same rationale as Claim(s) 4.
Regarding claim 5, Tuzel in view of Hobson, Sunkavalli, and Sakaguchi teaches the method of claim 1, wherein the machine learning model includes a multilayer perceptron model (Tuzel; pg 13, col. 4 line 41- col. 5 line 7, describe that the machine learning model includes a fully connected neural network with multiple hidden layers. A fully connected neural network with multiple layers is a multilayer perceptron. This teaches the machine learning model includes a multilayer perceptron model).
Claim 13, has similar limitations as of Claim(s) 5, therefore it is rejected under the same rationale as Claim(s) 5.
Regarding claim 8, Tuzel in view of Hobson, Sunkavalli, and Sakaguchi teaches the method of claim 1, further comprising performing parameter optimization on the output digital material model. Tuzel describes (Tuzel; pg. 14 col. 6 lines 51-60) after generating a high-resolution output image using the upsampling network, applying a post-processing operation. This teaches optimizing the high-resolution output image to adjust its value based on consistency, or, performing parameter optimization on the high-resolution output image.
Claim 16, has similar limitations as of claim 8, therefore it is rejected under the same rationale as claim 8.
Regarding claim 2, Tuzel in view of Hobson, Sunkavalli, and Sakaguchi teaches the method of claim 1, including texel-based material models upsampled by an image upsampler. Tuzel (Tuzel; pg. 13 lines 27-40) describes upsampling images for use in training a machine learning model and Hobson (Hobson; ¶0005) describes computer graphic textures comprising texels. Together, this teaches applying an image upsampler to texel-based material models. wherein the training data includes renderings computed from texels upsampled by an image upsampler at different light positions. Sunkavalli discloses (¶0072) that each training digital image portrays the object illuminated from a different lighting direction and (¶0073) that ground-truth images are rendered from different, new lighting directions. Sunkavalli describes (¶0071-74) generating rendered training digital images and ground-truth images of texturized synthetic objects, each rendered under different lighting directions. This teaches training data including renderings of textured surfaces at different light positions.
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to apply and/or modify the training of Tuzel in view of Hobson, with the multi-lighting-direction rendering technique of Sunkavalli to generate training renderings from the upsampled texel-based material models, as doing so would improve accuracy in various lighting conditions.
Claim 11, has similar limitations as of Claim(s) 2, therefore it is rejected under the same rationale as Claim(s) 2.
Claim 18, has similar limitations as of Claim(s) 2, therefore it is rejected under the same rationale as Claim(s) 2.
Regarding claim 9, Tuzel in view of Hobson, Sunkavalli, and Sakaguchi teaches the method of claim 1, wherein the output digital material model is applied to a three-dimensional geometry. Sunkavalli describes (Figs. 5A, 5B and ¶0075-76) generating digital objects such as a cube, a cylinder, or an ellipsoid as 3D shapes, then (¶0077) texturizing the digital object using a random texture crop to obtain a training set of texturized digital objects. This teaches applying a texel-based texture or digital material model to the surface of a three-dimensional geometry.
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to apply the high-resolution texel-based digital material model generated as taught by Tuzel in view of Hobson, Sunkavalli, and Sakaguchi to a three-dimensional geometry as further taught by Sunkavalli, in order to use the upsampled material model in 3D rendering pipelines.
Claims 3 are rejected under 35 U.S.C. 103 as being unpatentable over Tuzel (US9836820B2), Hobson (US20220092740A1), Sunkavalli (US20190340810A1), Sakaguchi (US20110206296A1), and Upchurch (US11403811B1).
Regarding claim 3, Tuzel in view of Hobson, Sunkavalli, and Sakaguchi teaches the method of claim 2, but does not explicitly disclose wherein the renderings are generated using a microfacet model.
Upchurch discloses (pg. 20 col. 16 lines 16-35) generating reconstruction images by evaluating a real-time rendering approximation of a micro-facet scattering function at each pixel and explains that “Micro-facet scattering functions describe how light scatters from a rough surface” and that micro-facet scattering models replace a detailed microsurface with a simplified macrosurface. This teaches generating renderings using a microfacet model.
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify the rendering approach of Sunkavalli with the microfacet scattering model of Upchurch in order to accurately simulate light scattering from rough surfaces.
Claim 19, has similar limitations as of Claim 3, therefore it is rejected under the same rationale as Claim 3.
Claims 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Tuzel (US9836820B2), Hobson (US20220092740A1), Sunkavalli (US20190340810A1), Sakaguchi (US20110206296A1), and Kim (US11562597B1)
Regarding claim 6, Tuzel in view of Hobson, Sunkavalli, and Sakaguchi teaches the method of claim 1, wherein the texel having the second resolution is generated, but the do not explicitly disclose using a transformer model.
Kim describes (Kim; Fig. 2A and 2B, and pg. 17 col. 8 line 49 - pg. 18 col. 9, line 18) an encoder-decoder neural network for processing image frames to generate output images and that the encoder network may be implemented as a transformer model. This teaches using a transformer-based neural network for generating output image data from image inputs. It would have been obvious to one of ordinary skill in the art, before the effective filing date, to implement the machine learning model of Tuzel in view of Hobson using a transformer-based architecture as taught by Kim, because doing so would have been a simple substitution of one known element for another to obtain predictable results.
Claim 14, has similar limitations as of claim 6, therefore it is rejected under the same rationale as claim 6.
Regarding claim 7, Tuzel in view of Hobson, Sunkavalli, Sakaguchi, and Kim teaches the method of claim 6, wherein the transformer model includes filters optimized for the one or more radiance images.
As previously discussed in claim 2, Sunkavalli describes (¶0072-73) generating rendered training digital images and ground-truth images of texturized synthetic objects, each rendered under different lighting directions and (¶0074) training an object-relighting neural network by backpropagating a loss to modify its parameters. Sunkavalli also describes (¶0062) the light transport function maps incident illumination from a direction to outgoing radiance at each pixel. This teaches optimizing learned network parameters using the one or more radiance images.
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to train the transformer model of Kim, as applied to the system of Tuzel in view of Hobson, using the multi-lighting direction rendering and training of Sunkavalli. The motivation for such a combination wo9uld have been to provide the benefit of optimizing the learned model parameters using the appearance information of the material to improve generation of higher resolution texels.
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Tuzel (US9836820B2), Hobson (US20220092740A1), Sunkavalli (US20190340810A1), Sakaguchi (US20110206296A1), and Zhao (Zhao, Ziping, and Daniel P. Palomar. "Sparse reduced rank regression with nonconvex regularization." 2018 IEEE statistical signal processing workshop (SSP). IEEE, 2018.).
Regarding claim 21, Tuzel in view of Hobson, Sunkavalli, and Sakaguchi teaches the method of claim 1. However, Tuzel in view of Hobson and Sunkavalli does not explicitly disclose wherein generating the texel having the second resolution comprises applying a sparsity-inducing loss function that retains material property signals.
Zhao; ABST and pg. 1-2, teaches sparsity inducing functions and regularization. Sunkavalli teaches training using images of an object illuminated from different lighting directions and comparing a generated image with a ground truth image using a loss function (Sunkavalli; ¶0022), and applying the loss function and backpropagating the loss to modify parameters (Sunkavalli; ¶0060-0061). Thus, it would have been obvious to apply Zhao’s sparsity-inducing loss function in the radiance-image based texel generation framework in a way that retains material property related signals.)
It would have been obvious to one of ordinary skill in the art before the effective filing date, to modify the method of Tuzel in view of Hoson and Sunkavalli with the sparsity-inducing loss function of Zhao. The motivation for such a combination would have been to provide the benefit of generating higher resolution texels with better material appearance.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAN F KALHORI whose telephone number is (571)272-5475. The examiner can normally be reached Mon-Fri 8:30-5:30 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, DEVONA E FAULK can be reached at (571) 272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAN F KALHORI/Examiner, Art Unit 2618
/DEVONA E FAULK/Supervisory Patent Examiner, Art Unit 2618
Prosecution Insights