DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 18, 19, 27, 28, and 33 provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 8, 14, 15, and 20 of copending Application No. 19/756,849 (reference application). Although the claims at issue are not identical, they are not patentably distinct from each other because the claims are obvious variants.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
Claim 18 of the instant application corresponds to claim 8 of the co-pending application as follows:
Instant Application
Co-pending Application
A computer-implemented method comprising:
A computer-implemented method comprising:
receiving at a user device, first texture data representing a plurality of computer graphic textures at a first resolution, the texture data being usable to render an object on a display of the user device;
receiving at least a first data structure representing at least one computer graphic texture, the first data structure having a first resolution… using the second data structure to render an object on at least one display
processing the first texture data using a set of machine learning models to generate second texture data representing the plurality of computer graphic textures at a second, higher resolution, wherein each machine learning model in the set of machine learning models is configured to convert instances of texture data from one resolution to another higher resolution;
processing the first data structure using at least one neural network to generate a second data structure representing the computer graphic texture, the second data structure having a second resolution… wherein the second resolution is higher than the first resolution
rendering the object on the display of the user device using the second texture data
using the second data structure to render an object on at least one display
The co-pending application claim does not explicitly recite that the reception of data and display is at a “user device”, but the term “user device” is so broad as to encompass just about any computer, and the co-pending application explicitly states that the method is computer implemented. Thus, performing processing and display at a computer that is a “user device” would be extremely well-known in the art and would not serve to patentably distinguish the claims. Further the co-pending application claim recites performing processing with at least one neural network while the instant application recites machine learning models. It is noted that a neural network would be considered a type of machine learning model and thus there is no patentable distinction in these terms either.
The instant and co-pending application claim numbers correspond to each other as follows:
Instant
18
19
27
28
33
Co-pending
8
14
15
20
15
Claims 20, 21, 23, 24, 29, 30, 32, 34, 35, and 37 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 8 and 15 of copending Application No. 19/756,849 in view of Fuller (U.S. Publication 2019/0304140).
This is a provisional nonstatutory double patenting rejection.
As to claim 20, claim 8 of the co-pending application does not disclose, but Fuller discloses wherein the first texture data is a mipmap sequence of compressed first texture data comprising a sequence of elements of first texture data, each element having been compressed at a particular level of compression, wherein an increasing ordering of elements in the mipmap sequence corresponds with increasingly compressed first texture data (p. 6, section 0060; as the compressed mipmap increases from 128x128 to 512x512, compression as measured by elements that represent a larger amount of elements increases; for example the 128x128 mipmap represents 16,384 elements missing in the 256x256 mipmap while the 512x512 mipmap represents the 1024x1024 mipmap with 262,144 elements missing), and further comprising:
identifying an element of first texture data at a particular level of compression as the first texture data for processing using the set of machine learning models based at least upon a measure of pixel density for rendering the object on the display of the user device (p. 6, section 0060; density of the mipmap is used to determine what to process for the final rendering mipmaps; for example, to produce the 256x256 mipmap level, the 128x128 density level is chosen while to produce the 1024x1024 mipmap level, the 512x512 density level is chosen). The motivation for this is to enable better compression and less storage usage while still allowing enhanced access speed (p. 1, section 0003). It would have been obvious to one skilled in the art to modify claim 8 of the co-pending application to have the first texture data be a mipmap sequence of compressed first texture data comprising a sequence of elements of first texture data, each element having been compressed at a particular level of compression, wherein an increasing ordering of elements in the mipmap sequence corresponds with increasingly compressed first texture data and identify an element of first texture data at a particular level of compression as the first texture data for processing using the set of machine learning models based at least upon a measure of pixel density for rendering the object on the display of the user device in order to enable better compression and less storage usage while still allowing enhanced access speed as taught by Fuller.
As to claim 21, claim 8 of the co-pending application does not disclose, but Fuller discloses compressing the second texture data using block compression and maintaining the compressed second texture data as a new element of first texture data in the mipmap sequence at the particular level of compression corresponding with the second, higher resolution (p. 6, sections 0057-0061; the upscaled/reconstructed MIP texture images are block compressed images compressed at their upscaled resolutions; for example, block compressed reconstructed 1024x1024 images correspond to upscaling the 512x512 texture to 1024x1024). Motivation for the combination is given in the Double Patenting rejection to claim 20 above.
As to claim 23, claim 8 of the co-pending application does not disclose, but Fuller discloses wherein the set of machine learning models comprises one or more neural network types (p. 3-4, section 0035; models can comprise at least two different neural network types, such as CNNs and GANs). Motivation for the combination is given in the Double Patenting rejection to claim 20 above.
As to claim 24, claim 8 of the co-pending application does not disclose, but Fuller discloses wherein each machine learning model in the set of machine learning models has been trained through operations comprising:
obtaining a set of ground truth outputs comprising a second instance of texture data at a third resolution; obtaining, for each ground truth output in the set of ground truth outputs, a corresponding training input comprising a first instance of texture data at a fourth lower resolution (p. 3, section 0034; actual high resolution images are input to the model, representing ground truth, and corresponding lower resolution images are also input);
processing each training input to generate a set of predicted outputs for the training inputs (p. 3, section 0034; the training involves how to take the lower resolution input and generate higher resolution predicted output); and
updating one or more values of the set of parameters of the machine learning model based at least on a discrepancy between the set of ground truth outputs and the set of predicted outputs for the training inputs (p. 3, section 0034-p. 4, section 0035; the models are optimized to reduce artifacts and quality loss in reconstructed images vs. original/ground truth images). Motivation for the combination is given in the Double Patenting rejection to claim 20 above.
As to claims 29 and 34, see the Double Patenting rejection of claim 20 above. Claims 29 and 34 relate to claim 15 of the co-pending application in a similar manner.
As to claims 30 and 35, see the Double Patenting rejection of claim 21 above. Claims 30 and 35 relate to claim 15 of the co-pending application in a similar manner.
As to claims 32 and 37, see the Double Patenting rejection of claim 24 above. Claims 32 and 37 relate to claim 15 of the co-pending application in a similar manner.
Claims 22, 31, and 36 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 8 and 15 of copending Application No. 19/756,849 in view of Fuller and Ha (U.S. Publication 2021/0074052).
This is a provisional nonstatutory double patenting rejection.
As to claim 22, claim 8 of the co-pending application does not disclose, but Fuller discloses wherein the set of machine learning models comprises:
a neural network configured to convert instances of albedo data from one resolution to instances of albedo data at another higher resolution (p. 2, sections 0022-0024; p. 3-4, section 0035; the machine learning network, which can be a neural network, upscales diffuse texture data into a higher resolution MIP level; albedo data characterizes diffuse lighting characteristics, and thus the network trained for the “diffuse” texture type would read on the neural network converting albedo data); and
a neural network configured to convert instances of reflectance data from one resolution to instances of reflectance data at another higher resolution (p. 2, sections 0022-0024; p. 3-4, section 0035; the machine learning network, which can be a neural network, upscales light map or texture content data into a higher resolution MIP level; reflectance data is characterized in a light map or in data about particular content types like metal, and thus the network trained for any of these maps would read on the neural network converting reflectance data), with motivation to combine similar to that given in the Double Patenting rejection of claim 20.
Fuller discloses a neural network configured to convert instances from one resolution to instances at another higher resolution, as well as roughness and normal textures (p. 2, sections 0022-0024; p. 3-4, section 0035; p. 4, section 0038). Fuller does not expressly disclose, but Ha does disclose processing normal data and roughness data in the same neural network (fig. 3; fig. 4; p. 4, sections 0067-0069; p. 4, section 0073-p. 5, section 0075; the normal data and the glossiness data, which would read on “roughness” data since it characterizes the same variable in an inverse manner, are processed together using the same neural network/machine learning model). The motivation for this is to render 3D scenes under various rendering conditions through a realistic simulation. It would have been obvious to one skilled in the art to modify the co-pending application claim and Fuller to process normal data and roughness data in the same neural network in order to render 3D scenes under various rendering conditions through a realistic simulation as taught by Ha.
As to claims 31 and 36, see the Double Patenting rejection of claim 22 above. Claims 31 and 36 relate to claim 15 of the co-pending application in a similar manner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 33-37 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the claims are directed to “computer readable media” which would encompass a claim that is directed to nothing more than a transitory signal. Transitory signals are not processes, machines, manufactures or compositions of matter and thus are not patent-eligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 18-21, 23, 24, 27-30, 32-35, and 37 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Fuller (U.S. Publication 2019/0304140).
As to claim 18, Fuller discloses a computer-implemented method comprising:
receiving at a user device, first texture data representing a plurality of computer graphic textures at a first resolution, the texture data being usable to render an object on a display of the user device (p. 4, section 0038; p. 5, section 0046; p. 6, sections 0057-0060; p. 6, section 0066; a plurality of textures are received at user device such as a computer, at, for example, a 512x512 resolution, to render a textured scene to display);
processing the first texture data using a set of machine learning models to generate second texture data representing the plurality of computer graphic textures at a second, higher resolution, wherein each machine learning model in the set of machine learning models is configured to convert instances of texture data from one resolution to another higher resolution (p. 2, section 0022; p. 6, sections 0057-0060; a number of ML models can be used to generate higher resolution textures of different types; for example, a 512x512 texture can be converted into a 1024x1024 texture); and
rendering the object on the display of the user device using the second texture data (p. 3, section 0029; p. 6, section 0066; the reconstructed/upscaled texture is used to render objects in a scene on a display).
As to claim 19, Fuller discloses wherein the first texture data has been compressed using block compression (p. 4, section 0043; p. 7, section 0075).
As to claim 20, Fuller discloses wherein the first texture data is a mipmap sequence of compressed first texture data comprising a sequence of elements of first texture data, each element having been compressed at a particular level of compression, wherein an increasing ordering of elements in the mipmap sequence corresponds with increasingly compressed first texture data (p. 6, section 0060; as the compressed mipmap increases from 128x128 to 512x512, compression as measured by elements that represent a larger amount of elements increases; for example the 128x128 mipmap represents 16,384 elements missing in the 256x256 mipmap while the 512x512 mipmap represents the 1024x1024 mipmap with 262,144 elements missing), and further comprising:
identifying an element of first texture data at a particular level of compression as the first texture data for processing using the set of machine learning models based at least upon a measure of pixel density for rendering the object on the display of the user device (p. 6, section 0060; density of the mipmap is used to determine what to process for the final rendering mipmaps; for example, to produce the 256x256 mipmap level, the 128x128 density level is chosen while to produce the 1024x1024 mipmap level, the 512x512 density level is chosen).
As to claim 21, Fuller discloses compressing the second texture data using block compression and maintaining the compressed second texture data as a new element of first texture data in the mipmap sequence at the particular level of compression corresponding with the second, higher resolution (p. 6, sections 0057-0061; the upscaled/reconstructed MIP texture images are block compressed images compressed at their upscaled resolutions; for example, block compressed reconstructed 1024x1024 images correspond to upscaling the 512x512 texture to 1024x1024).
As to claim 23, Fuller discloses wherein the set of machine learning models comprises one or more neural network types (p. 3-4, section 0035; models can comprise at least two different neural network types, such as CNNs and GANs).
As to claim 24, Fuller discloses wherein each machine learning model in the set of machine learning models has been trained through operations comprising:
obtaining a set of ground truth outputs comprising a second instance of texture data at a third resolution; obtaining, for each ground truth output in the set of ground truth outputs, a corresponding training input comprising a first instance of texture data at a fourth lower resolution (p. 3, section 0034; actual high resolution images are input to the model, representing ground truth, and corresponding lower resolution images are also input);
processing each training input to generate a set of predicted outputs for the training inputs (p. 3, section 0034; the training involves how to take the lower resolution input and generate higher resolution predicted output); and
updating one or more values of the set of parameters of the machine learning model based at least on a discrepancy between the set of ground truth outputs and the set of predicted outputs for the training inputs (p. 3, section 0034-p. 4, section 0035; the models are optimized to reduce artifacts and quality loss in reconstructed images vs. original/ground truth images).
As to claim 27, see the rejection to claim 18. Further, Fuller discloses a computing device comprising a memory configured to store instructions (fig. 1, element 34; p. 8, section 0087-p. 9, section 0088) and a processor to execute the instructions to perform operations (fig. 1, element 32; p. 8, section 0087-p. 9, section 0088).
As to claim 28, see the rejection to claim 19.
As to claim 29, see the rejection to claim 20.
As to claim 30, see the rejection to claim 21.
As to claim 32, see the rejection to claim 24.
As to claim 33, see the rejections to claims 18 and 27.
As to claim 34, see the rejection to claim 20.
As to claim 35, see the rejection to claim 21.
As to claim 37, see the rejection to claim 24.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 22, 31, and 36 are rejected under 35 U.S.C. 103 as being unpatentable over Fuller in view of Ha (U.S. Publication 2021/0074052).
As to claim 22, Fuller discloses wherein the set of machine learning models comprises:
a neural network configured to convert instances of albedo data from one resolution to instances of albedo data at another higher resolution (p. 2, sections 0022-0024; p. 3-4, section 0035; the machine learning network, which can be a neural network, upscales diffuse texture data into a higher resolution MIP level; albedo data characterizes diffuse lighting characteristics, and thus the network trained for the “diffuse” texture type would read on the neural network converting albedo data); and
a neural network configured to convert instances of reflectance data from one resolution to instances of reflectance data at another higher resolution (p. 2, sections 0022-0024; p. 3-4, section 0035; the machine learning network, which can be a neural network, upscales light map or texture content data into a higher resolution MIP level; reflectance data is characterized in a light map or in data about particular content types like metal, and thus the network trained for any of these maps would read on the neural network converting reflectance data);
Fuller discloses a neural network configured to convert instances from one resolution to instances at another higher resolution, as well as roughness and normal textures (p. 2, sections 0022-0024; p. 3-4, section 0035; p. 4, section 0038). Fuller does not expressly disclose, but Ha does disclose processing normal data and roughness data in the same neural network (fig. 3; fig. 4; p. 4, sections 0067-0069; p. 4, section 0073-p. 5, section 0075; the normal data and the glossiness data, which would read on “roughness” data since it characterizes the same variable in an inverse manner, are processed together using the same neural network/machine learning model). The motivation for this is to render 3D scenes under various rendering conditions through a realistic simulation. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Fuller to process normal data and roughness data in the same neural network in order to render 3D scenes under various rendering conditions through a realistic simulation as taught by Ha.
As to claim 31, see the rejection to claim 22.
As to claim 36, see the rejection to claim 22.
Conclusion
Claims 25 and 26 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
As to claim 25, Fuller discloses obtaining ground truth outputs and corresponding training inputs, as noted in the rejection to claim 24. However, the ground truth outputs in Fuller or other combinable prior art are not disclosed as obtained from texture block compression data, and generation of training inputs is not disclosed as compressing ground truth outputs using block compression.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON M RICHER whose telephone number is (571)272-7790. The examiner can normally be reached 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON M RICHER/ Primary Examiner, Art Unit 2617
