Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 31 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as failing to set forth the subject matter which the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the applicant regards as the invention. Claim 31 appears to be indefinite and incomplete. Claim recites “and the fetched texels ” which is an incomplete sentence and is likely an error. Correction is necessary to proceed with examination.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4, 8-12, 14, 16-20, 22 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Li et al. (U.S. Patent Publication No. 2019/0251734).
Regarding claim 1, Li discloses a method, comprising: at a device: decompressing at least a portion of a single texture representation of a set of textures into at least a portion of a plurality of textures included in the set of textures [Li: 0039 “At block 206 , a consistent texture atlas for each frame of a segment is generated , and texture coordinates are re - projected so that the vertices of the source mesh of the CMS are mapped to the consistent texture atlas . As used herein , a " consistent texture atlas , ” or “ CTA , " defines a single texture - coordinate space that can include all of the textures used in a frame”][Li: 0046 “As described above , the compressed data for a segment includes a 2D video file containing the CIA for the segment and a bin file containing the compressed vertex information for the segment . At block 804 , the rendering system can regenerate image data for frame F ; in the com pressed CIA video file , e . g . , using conventional techniques for regenerating in a frame in a 2D video file generated using standard 21 ) video compression techniques”](teaches a CTA as a single texture coordinate space that can include all textures used in a frame, further teaches that the CTA is contained in a compressed CTA video file and that the rendering system regenerates image data from that compressed CTA video file); and outputting the at least a portion of the plurality of textures (Li: Abstract “The compressed data files can be decompressed and used to render display able images”)[Li: 0046 “At block 808 , the rendering system can render an image for frame F ; using the regenerated mesh and the regenerated CIA . Process 800 can be repeated for any number of frames ; frames can be rendered successively to generate animation”](teaches that the decompressed/regenerated CTA is used by the rendering system to render displayable images, using the regenerated CTA in the rendering pipeline is an output of the decompressed texture data because the texture data is made available to render the final image).
Regarding claim 2, Li discloses the method of claim 1, wherein the single texture representation is a compressed representation of the plurality of textures included in the set of textures [Li: 0039 “At block 206 , a consistent texture atlas for each frame of a segment is generated , and texture coordinates are re - projected so that the vertices of the source mesh of the CMS are mapped to the consistent texture atlas . As used herein , a " consistent texture atlas , ” or “ CTA , " defines a single texture - coordinate space that can include all of the textures used in a frame”].
Regarding claim 3, Li discloses the method of claim 2, wherein the compressed representation is generated using at least one compression method [Li: 0006 “After a segment having a CMS is identified , a consistent texture atlas ( CTA ) is constructed from the texture map ( s ) associated with the object ( s ) in the segment , and the texture ( s ) from the CTA is ( are ) re - projected onto the mesh of the CMS . The CTA is then compressed , e . g . , using standard video compression techniques”](teaches that the CTA which is the single texture representation is compressed using standard video compression techniques).
Regarding claim 4, Li discloses the method of claim 2, wherein the compressed representation is generated using at least two compression methods [Li: 0006 “After a segment having a CMS is identified , a consistent texture atlas ( CTA ) is constructed from the texture map ( s ) associated with the object ( s ) in the segment , and the texture ( s ) from the CTA is ( are ) re - projected onto the mesh of the CMS . The CTA is then compressed , e . g . , using standard video compression techniques . The CMS is also compressed , e . g . , by defining vertex trajectory vectors that can be clustered and compressed based on principal com ponent analysis”](teaches 2 different compression techniques).
Regarding claim 8, Li discloses the method of claim 1, wherein the set of textures represents a material [Li: 0023 “Texture maps can be used to represent various attributes that may vary across the surface of the mesh , including color , surface normal , lighting effects , sur face properties ( e . g . , reflectivity , transparency ) , and so on , and any number of textures representing any number of attributes may be associated with a single mesh”].
Regarding claim 9, Li discloses the method of claim 8, wherein each texture of the plurality of textures represents a different property of the material [Li: 0023 “Texture maps can be used to represent various attributes that may vary across the surface of the mesh , including color , surface normal , lighting effects , sur face properties ( e . g . , reflectivity , transparency ) , and so on , and any number of textures representing any number of attributes may be associated with a single mesh”].
Regarding claim 10, Li discloses the method of claim 1, wherein at least one texture of the plurality of textures includes a plurality of channels [Li: 0023 “Texture maps can be used to represent various attributes that may vary across the surface of the mesh , including color , surface normal , lighting effects , sur face properties ( e . g . , reflectivity , transparency ) , and so on , and any number of textures representing any number of attributes may be associated with a single mesh”].
Regarding claim 11, Li discloses the method of claim 1, wherein the decompressing is performed in hardware [Li: 0057 “Network interface 908 can be implemented using any combination of hardware and software components that together enable communication with other computer systems”].
Regarding claim 12, Li discloses the method of claim 11, wherein the decompressing is performed in a processing unit driver [Li: 0054 “Processing subsystem 902 can include one or more general purpose programmable processors capable of executing program code instructions to perform various operations , including operations described herein”].
Regarding claim 14, Li discloses the method of claim 1, wherein the decompressing is performed in system software [Li: 0055 “Storage subsystem 904 can include a combination of volatile and nonvolatile storage elements ( e . g . , DRAM , SRAM , flash memory , magnetic disk , optical disk , etc . ) . Portions of storage subsystem 904 may be used to store program code to be executed by processing subsystem 904 . Examples of program code can include compression module 910 ( e . g . , code implementing process 200 of FIG . 2 ) and / or decompression module 912 ( e . g . ; code implementing process 800 of FIG . 8 )”].
Regarding claim 16, Li discloses the method of claim 1, wherein the decompressing is performed at install time when an application has been downloaded and installed onto a storage device of a computer [Li: 0055 “Storage subsystem 904 can include a combination of volatile and nonvolatile storage elements ( e . g . , DRAM , SRAM , flash memory , magnetic disk , optical disk , etc . ) . Portions of storage subsystem 904 may be used to store program code to be executed by processing subsystem 904 . Examples of program code can include compression module 910 ( e . g . , code implementing process 200 of FIG . 2 ) and / or decompression module 912 ( e . g . ; code implementing process 800 of FIG . 8 )”].
Regarding claim 17, Li discloses the method of claim 1, wherein the at least a portion of the single texture representation is streamed to a memory for decompression [Li: 0007 “The compressed data file format can be suitable for streaming video presentations . Specifically , the format allows the data for each segment to be sent independently of data for any other segments , and therefore it is not necessary for the receiving device to wait for all of the data to be received before beginning to render and display images , nor is it necessary for the receiving device to begin rendering with any particular segment”].
Regarding claim 18, Li discloses the method of claim 17, wherein a portion of the single texture representation is streamed to the memory while a portion of the plurality of textures is being rendered [Li: 0007 “The compressed data file format can be suitable for streaming video presentations . Specifi cally , the format allows the data for each segment to be sent independently of data for any other segments , and therefore it is not necessary for the receiving device to wait for all of the data to be received before beginning to render and display images , nor is it necessary for the receiving device to begin rendering with any particular segment”][Li: 0052 “At the receiving system , a given frame can be rendered as soon as all of its data has been received . For instance , process 800 can be applied to regenerate one or more frames of a given segment without using data from any previous segments and without waiting for any data from subsequent segments . Thus , a receiving system can begin rendering and displaying systems at any point in a stream .”].
Regarding claim 19, Li discloses the method of claim 1, wherein the single texture representation enables random access such that only a portion of the single texture representation is decompressed [Li: 0052 “At the receiving system , a given frame can be rendered as soon as all of its data has been received . For instance , process 800 can be applied to regenerate one or more frames of a given segment without using data from any previous segments and without waiting for any data from subsequent segments . Thus , a receiving system can begin rendering and displaying systems at any point in a stream .”].
Regarding claim 20, Li discloses the method of claim 1, further comprising, at the device: compressing the output at least a portion of the plurality of textures into a select compressed format [Li: 0048 “The resulting compressed data for a segment can be transmitted as soon as it is generated , even if subsequent frames are still being generated . In applications where stored data is streamed , the data can be stored in compressed format and delivered to a receiving device , either on demand or in accordance with a schedule”].
Regarding claim 22, Li discloses the method of claim 20, wherein the select compressed format is supported by a processing unit that performs rendering [Li: 0055 “portions of storage subsystem 904 may be used to store program code to be executed by processing subsystem 904”].
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5-7, 13, 15, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (U.S. Patent Publication No. 2019/0251734), in view of Fuller et al. (U.S. Patent Publication No. 2019/0304140).
Regarding claim 5, Li discloses the method of claim 1, but fails to explicitly disclose wherein the single texture representation is learned using a neural network.
However, Fuller discloses wherein the single texture representation is learned using a neural network [Fuller: 0022 “Different machine learning networks and / or trained models may be created for different texture types”].
Li and Fuller are considered to be analogous to the claimed invention because they are in the same field of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Li to incorporate Fuller’s teachings of utilizing a neural network. The motivation for such a combination would provide the benefit of reducing texture storage size while efficiently generating texture data for rendering.
Regarding claim 6, Li discloses the method of claim 5, but fails to explicitly disclose wherein the single texture representation is further generated by applying an entropy encoding to an output of the neural network.
However, Fuller discloses wherein the single texture representation is further generated by applying an entropy encoding to an output of the neural network [Fuller: 0040 “Machine learning model 31 may use adversarial machine learning networks , such as , but not limited to , convolutional neural networks and generative adversarial networks ( GANs ) to generate the trained machine learning model 18”][Fuller: 0041 “the trained machine learning model 18 may decompress the identified textures 17 into block compressed textures usable by the GPU by predicting the components of a blocked compressed texture ( e . g . , the modes , shapes , endpoints , and / or indices ) for the identified textures 17 and / or a region of the texture 17”](teaches that a neural network based trained machine learning model predicts components of a block compressed texture including modes, shapes, endpoints, and indices).
Li and Fuller are considered to be analogous to the claimed invention because they are in the same field of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Li to incorporate Fuller’s teachings of utilizing a neural network. The motivation for such a combination would provide the benefit of reducing texture storage size while efficiently generating texture data for rendering.
Regarding claim 7, Li discloses the method of claim 1, but fails to explicitly disclose wherein the single texture representation is a pyramid of a plurality of feature levels, wherein each feature level of the plurality of feature levels includes a plurality of grids.
However, Fuller discloses wherein the single texture representation is a pyramid of a plurality of feature levels, wherein each feature level of the plurality of feature levels includes a plurality of grids [Fuller: 0060 “the trained machine learning model 18 may generate a hardware compatible compressed reconstructed MIP chain 20 that includes compressed versions of the 1024x1024 first image , the 512x512 second image 202 , the 256x256 third image , the 128x128 fourth image 204 , the 64x64 fifth image , a 32x32 sixth image , and a 16x16 seventh image”](teaches a reconstructed MiP chain having multiple compressed images at different resolutions , a MiP chain is a texture pyramid because it contains progressively smaller texture levels which corresponds to the claimed pyramid of feature levels and grid).
Li and Fuller are considered to be analogous to the claimed invention because they are in the same field of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Li to incorporate Fuller’s teachings of generating a reconstructed MiP chain having multiple compressed texture levels at different resolutions. The motivation for such a combination would provide the benefit of representing texture data at multiple resolutions for efficient rendering and reduced memory usage.
Regarding claim 13, Li discloses the method of claim 12, but fails to explicitly disclose wherein the processing unit is a graphics processing unit.
However, Fuller discloses wherein the processing unit is a graphics processing unit [Fuller: 0006 “The computer device may include a graphics processing unit ( GPU)”].
Li and Fuller are considered to be analogous to the claimed invention because they are in the same field of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Li to incorporate Fuller’s teachings of utilizing a GPU. The motivation for such a combination would provide the benefit of having more compute power.
Regarding claim 15, Li discloses the method of claim 1, but fails to explicitly disclose wherein the decompressing is performed at load time when loading graphics data to memory for use by a processing unit during rendering.
However, Fuller discloses wherein the decompressing is performed at load time when loading graphics data to memory for use by a processing unit during rendering [Fuller: 0006 “The computer device may include a graphics processing unit ( GPU)”].
Li and Fuller are considered to be analogous to the claimed invention because they are in the same field of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Li to incorporate Fuller’s teachings of utilizing a GPU. The motivation for such a combination would provide the benefit of having more compute power.
Regarding claim 21, Li discloses the method of claim 20, but fails to explicitly disclose wherein the select compressed format is a select block compression format.
However, Fuller discloses wherein the select compressed format is a select block compression format (interpreted as texture compression where the texture is divided into block and each block is encoded into compressed block data) [Li: 0057 “At 504 , method 500 may include decompressing the hardware incompatible compressed textures directly into hardware compatible compressed textures usable by the GPU . The trained machine learning model 18 may receive and / or otherwise access the modified MIP chains 14 and / or the hardware incompatible compressed textures 16 and may decompress the hardware incompatible compressed textures 16 and / or the modified MIP chains 14 directly into hardware compatible compressed textures 22 for use with application 10. For example , the trained machine learning model 18 may decompress the compressed textures 16 directly into block compressed textures usable by the GPU”].
Li and Fuller are considered to be analogous to the claimed invention because they are in the same field of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Li to incorporate Fuller’s teachings of decompressing texture data directly into block compressed textures. The motivation for such a combination would provide the benefit of storing texture data into a GPU efficient block compression format.
Claims 23-24 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (U.S. Patent Publication No. 2019/0251734), in view of Akenine Moller et al. (U.S. Patent Publication No. 2022/0392140).
Regarding claim 23, Li discloses the method of claim 1, but fails to explicitly disclose further comprising, at the device: interleaving the output at least a portion of the plurality of textures to form at least one interleaved texture.
However, Akenine Moller discloses further comprising, at the device: interleaving the output at least a portion of the plurality of textures to form at least one interleaved texture (interpreted as arranging data from multiple textures together such as alternating blocks from different textures)[Akenine Moller: 0111 “multiple textures that would otherwise be accessed separately during ray tracing are interleaved into a single, interleaved texture that can be used to access the multiple textures together”][Akenine Moller: 0023 “an interleaved texture can include alternating blocks from multiple textures. Such an interleaved texture can be generated when the multiple textures are being loaded into memory”].
Li and Akenine Moller are considered to be analogous to the claimed invention because they are in the same field of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Li to incorporate Akenine Moller’s teachings of interleaving multiple textures into a single interleaved texture. The motivation for such a combination would provide the benefit of reducing texture bandwidth.
Regarding claim 24, Li discloses the method of claim 23, but fails to disclose further comprising, at the device: compressing the output at least a portion of the plurality of textures into at least one select compressed format to form two or more compressed textures; wherein the interleaving is applied to the two or more compressed textures to form the at least one interleaved texture.
However, Akenine Moller discloses further comprising, at the device: compressing the output at least a portion of the plurality of textures into at least one select compressed format to form two or more compressed textures; wherein the interleaving is applied to the two or more compressed textures to form the at least one interleaved texture [Akenine Moller: 0084 “the compressed textures are interleaved together to generate mipmap level 1 740 and mipmap level 2 750”].
Li and Akenine Moller are considered to be analogous to the claimed invention because they are in the same field of image processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Li to incorporate Akenine Moller’s teachings of interleaving multiple textures into a single interleaved texture. The motivation for such a combination would provide the benefit of reducing texture bandwidth.
Allowable Subject Matter
Claims 25-30 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHMED TAHA whose telephone number is (571)272-6805. The examiner can normally be reached 8:30 am - 5 pm, Mon - Fri.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, XIAO WU can be reached at (571)272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AHMED TAHA/Examiner, Art Unit 2613
/XIAO M WU/Supervisory Patent Examiner, Art Unit 2613
Prosecution Insights