DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 6, 7, 10-13, 16, 17, 19, and 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Bi et al. (US 2022/0335636; hereinafter “Bi”).
Regarding claim 1, Bi discloses An image rendering method (“renders images of a scene,” abstract), comprising: obtaining target camera association information and target light source association information (“specify an arbitrary viewing location and lighting location,” para. 23); determining target voxel association information of each voxel in a target object based on the target camera association information, the target light source association information, and a pre-trained target object attribute determination model (“trains a network to learn a volume representation of the scene … the volume representation may comprise, opacity, surface normal, and reflectance voxel grids,” para. 20; “reconstruct per-voxel reflectance and handle complex materials with high glossiness,” para. 25); and rendering a target structure image corresponding to the target object based on the target voxel association information of each voxel (“The neural rendering framework thus enables rendering with complex view-dependent and light-dependent shading effects including specularities, occlusions, and shadows,” para. 25).
Regarding claim 2, Bi discloses wherein the camera association information comprises a camera position and orientation information, and the light source association information comprises a light source position and a lighting direction of a light source (“allows a user to provide input adjusting the viewing location and lighting location,” para. 33; “to render images under different viewing and lighting directions,” para. 35).
Regarding claim 3, Bi discloses determining at least one to-be-processed sampling point on each light ray, by processing, based on a preset sampling point determination procedure (“Points are sequentially sampled along a given ray,” para. 43), the target camera association information and the target light source association information (“computes the reflected radiance from the reflectance function and the incoming light … the accumulated opacity from the camera,” para. 45); obtaining to-be-selected voxel association information corresponding to each sampling point, by processing the at least one to-be-processed sampling point based on the target object attribute determination model (“computes reflected radiance at x using its local surface normal n(x) and the reflectance parameters R(x) of a given surface reflectance model,” para. 42); and determining the target voxel association information of each voxel based on the to-be-selected voxel association information corresponding to each sampling point (“reconstruct per-voxel reflectance,” para. 25).
Regarding claim 6, Bi discloses determining internal structure information corresponding to the target object based on the target voxel association information of each voxel (“the volume representation may comprise … surface normal,” para. 20); and rendering the target structure image corresponding to the target object based on the internal structure information (“The neural rendering framework thus enables rendering with complex view-dependent and light-dependent shading effects including specularities, occlusions, and shadows,” para. 25).
Regarding claim 7, Bi discloses obtaining the target object attribute determination model through training (“the scene reconstruction system trains a network to learn a volume representation,” para. 3); wherein obtaining the target object attribute determination model through training comprises: obtaining a plurality of training samples, wherein each training sample comprises a to-be-trained association parameter and a theoretical rendered image corresponding to the to-be-trained association parameter, and the to-be-trained association parameter comprises a to-be-trained camera parameter and a to-be-trained light source parameter (“Given a set of input images of a scene, the training module trains a network to learn a volume representation of the scene. The scene can be any real scene with any number of objects. The set of input images comprises images taken from different viewpoints of the scene. As will be described in further detail below, the images may be taken with collocated viewing and lighting to facilitate the training process,” para. 31); obtaining an actual rendered image corresponding to the to-be-trained association parameter, by inputting, for each training sample, to-be-trained association parameter in a current training sample into a to-be-trained object attribute determination model (“images rendered using the volume representation,” para. 31); determining an error value based on the actual rendered image and a theoretical rendered image in the current training sample; calculating a preset loss function in the to-be-trained object attribute determination model based on the error value, and performing parameter correction on the to-be-trained object attribute determination model; and obtaining the target object attribute determination model, by converging the preset loss function as a training objective (“the network may be trained using a loss function that minimizes a difference between the input images and images rendered using the volume representation,” para. 31; “During training, the rendering module can render images of the scene with the same viewing and lighting location of the input images to provide rendered images for comparison with the input images when evaluating the loss function,” para. 32).
Regarding claim 10, it is rejected using the same citations and rationales described in the rejection of claim 1, with the additional limitations of An electronic device, comprising: at least one processor; and a storage means, configured to store at least one program, wherein when the at least one program is executed by the at least one processor, the at least one processor is caused to (“a processor executing instructions stored in memory,” Bi, para. 58).
Regarding claim 11, it is rejected using the same citations and rationales described in the rejection of claim 1, with the additional limitations of A non-transitory computer-readable storage medium, storing a computer program, wherein the program, when executed by a processor, implements (“a processor executing instructions stored in memory,” Bi, para. 58).
Regarding claims 12, 13, 16, and 17, they are rejected using the same citations and rationales described in the rejections of claims 2, 3, 6, and 7, respectively.
Regarding claims 19 and 20, they are rejected using the same citations and rationales described in the rejections of claims 2 and 3, respectively.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4, 8, 14, 18, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Bi in view of Zheng et al. (“Neural Relightable Participating Media Rendering”; hereinafter “Zheng”).
Regarding claim 4, Bi does not disclose wherein the target voxel association information at least comprises color information and light quantity information of a voxel.
In the same art of neural rendering, Zheng teaches wherein the target voxel association information at least comprises color information and light quantity information of a voxel (Fig. 2 illustrates, for each sample point, an output of albedo a which represents color information and spherical harmonic coefficients
c
l
m
which represent light quantity information).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Zheng to Bi. The motivation would have been that it “achieves better visual quality” (Zheng, pg. 2, para. 2).
Regarding claim 8, Bi discloses determining information of at least one to-be-trained sampling point corresponding to the to-be-trained association parameter; obtaining to-be-trained … information which are outputted by the to-be-trained object attribute determination model and correspond to the at least one to-be-trained sampling point, by inputting the information of the at least one to-be-trained sampling point into the to-be-trained object attribute determination model; and rendering, based on the to-be-trained … information, an actual rendered image corresponding to the target object (“During training, pixels may be randomly sampled from the captured images and the ray marching (using known camera calibration) is performed to get the rendered pixel colors Lk of pixel k; the approach may supervise them with the ground truth colors Ĺk in the captured images using a L2 loss,” para. 54).
Bi does not disclose the to-be-trained information being color information and light quantity information.
In the same art of neural rendering, Zheng teaches the to-be-trained information being color information and light quantity information (Fig. 2 illustrates an output of albedo a which represents color information and spherical harmonic coefficients
c
l
m
which represents light quantity information; “we can end-to-end learn a neural representation … During each training iteration, we trace primary rays through the media … We optimize the parameters of Fθ, Rϑ, and Sϕ by minimizing a rendering loss,” pg. 6, sec. 4.3).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Zheng to Bi. The motivation would have been that it “achieves better visual quality” (Zheng, pg. 2, para. 2).
Regarding claims 14 and 18, they are rejected using the same citations and rationales described in the rejections of claims 4 and 8, respectively.
Regarding claim 21, it is rejected using the same citations and rationales described in the rejection of claim 4.
Allowable Subject Matter
Claims 5 and 15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: The known prior art does not teach or render obvious, when considered in the context of the parent and intervening claim limitations, wherein the light quantity information comprises a reflection brightness value of light rays irradiating on the voxel, and a refraction brightness value of the light rays refracted from an interior of the voxel.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ryan McCulley whose telephone number is (571)270-3754. The examiner can normally be reached Monday through Friday, 8:00am - 4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN MCCULLEY/Primary Examiner, Art Unit 2611