DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This Office Action is in response to Applicant’s amendment/response filed on 13 February 2026, which has been entered and made of record.
Response to Arguments
Applicant's arguments filed 13 February 2026 have been fully considered but they are not persuasive.
Applicant argues “the lighting described in Lombardi is referring to the lighting during image capture of the subject. By contrast, the claims clarify that the compressed lighting representation corresponds to lighting characteristics for a different environment than the physical environment in which the subject is captured” (Remarks, pg. 7). The Examiner respectfully disagrees.
Lombardi discloses “while traditional systems may require a prediction of albedo, specular, etc., to enable relighting, the disclosed systems may directly predict a shaded appearance as output” (para. 43). In other words, the autoencoder of Lombardi directly predicts a “relighted” image, meaning an image having a different lighting environment than an original image.
The Examiner acknowledges that Lombardi measures the lighting of the capture environment; this is for conditioning/training the neural network to learn different lighting configurations: “The autoencoder may be a conditional autoencoder, and the autoencoder may be configured to condition at least one variable that is associated with each image” (para. 5); “condition the network on illumination” (para. 39); “conditioning may be performed by concatenating a conditioning variable to the first latent vector z” (para. 74); “the decoder may translate the first latent vector z … into geometry and appearance texture” (para. 78). Thus, the output of the autoencoder can be conditioned on a desired lighting. This enables the “relighting” described in para. 43 of Lombardi. Lombardi further discloses “The systems described herein may also use the dynamic light conditions as input in the model, which may allow the systems described herein to prescribe a particular lighting environment when rendering the avatar” (para. 62). The “particular lighting environment” refers to a relighted lighting environment, which can be different from the captured lighting environment. Lombardi uses the plurality of captured lighting configurations to train an autoencoder how to reproduce different lighting environments, and the trained autoencoder can be used to perform image relighting.
Any remaining arguments are considered moot based on the foregoing.
Double Patenting
The previous nonstatutory double patenting rejection is withdrawn in view of the approved Terminal Disclaimer.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6-11, 13-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Lombardi et al. (US 2019/0213772; hereinafter “Lombardi”) in view of Sunkavalli et al. (US 2020/0302684; hereinafter “Sunkavalli”).
Regarding claim 1, Lombardi discloses A non-transitory computer readable medium comprising computer readable instructions executable by one or more processors (“non-transitory-type media,” para. 94) to: obtain a request to represent an avatar of a subject (“render avatar geometry and texture,” para. 65) in a requested scene (“provide photorealistic rendering of dynamic and socially interactive scenes,” para. 2) having first lighting characteristics; obtain a lighting representation for the first lighting characteristics (“prescribe a particular lighting environment when rendering the avatar,” para. 62; “enable relighting,” para. 43), wherein the subject is captured in a physical environment having second lighting characteristics (“record the subject from a plurality of cameras,” para. 60; “Light sources may provide various light conditions during the capture operations,” para. 61; “enable relighting,” para. 43); apply the lighting representation and a geometric representation of the subject to a network to obtain a texture map corresponding to the avatar, wherein the network is configured to generate texture data from lighting and geometric data (“use the dynamic light conditions as input in the model,” para. 62; “encode … the geometry information,” para. 52; “use the latent vector to infer … an inferred view-dependent texture of the subject,” para. 52; “prescribe a particular lighting environment when rendering the avatar,” para. 62); cause the avatar of the subject to be rendered in a view of the requested scene based on the texture map and the geometric representation of the subject (“render reconstructed images of the subject to provide an avatar, for example, using the inferred geometry and the inferred view-dependent texture,” para. 50).
Lombardi does not disclose using a compressed representation of lighting for an environment.
In the same art of graphics rendering, Sunkavalli teaches using a compressed representation of lighting for an environment (“When generating location-specific-lighting parameters, the lighting estimation system can generate spherical-harmonic coefficients that indicate lighting conditions,” para. 30).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Sunkavalli to Lombardi. The motivation would have been “to realistically depict changes in lighting” (Sunkavalli, para. 30).
Regarding claim 2, the combination of Lombardi and Sunkavalli renders obvious wherein the compressed representation comprises a compressed representation of lighting and color in the requested scene (“renders pixels for the virtual object that reflect lighting, shading, or appropriate color hues indicated by the location-specific-lighting parameters,” Sunkavalli, para. 47; see claim 1 for motivation to combine).
Regarding claim 3, the combination of Lombardi and Sunkavalli renders obvious wherein the compressed lighting representation corresponds to at least one selected from a group consisting of spherical harmonic coefficients, spherical gaussians, and spherical wavelets for the requested scene (“When generating location-specific-lighting parameters, the lighting estimation system can generate spherical-harmonic coefficients that indicate lighting conditions,” Sunkavalli, para. 30; see claim 1 for motivation to combine).
Regarding claim 4, the combination of Lombardi and Sunkavalli renders obvious obtain one or more images of the subject (“record the subject from a plurality of cameras,” Lombardi, para. 60); and obtain a latent representation of a geometry of the subject based on the one or more images (“encode … the geometry information,” Lombardi, para. 52).
Regarding claim 6, the combination of Lombardi and Sunkavalli renders obvious wherein the avatar of the subject is further rendered based on a head pose of the subject (“tracked head pose,” Lombardi, para. 45; “a viewer's point of view, relative to the position and orientation of the avatar,” Lombardi, para. 37).
Regarding claim 7, the combination of Lombardi and Sunkavalli renders obvious wherein the avatar is rendered by applying the texture map to the geometric representation of the subject (“use the reconstructed texture maps and the reconstructed three-dimensional mesh of the subject to render a reconstructed image of the subject, thus providing a data-driven avatar of the subject,” Lombardi, para. 43).
Regarding claims 8-11, 13, and 14, they are rejected using the same citations and rationales described in the rejections of claims 1-4, 6, and 7, respectively.
Regarding claims 15-18 and 20, they are rejected using the same citations and rationales described in the rejections of claims 1-4 and 6, respectively.
Claims 5, 12, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Lombardi and Sunkavalli, and further in view of Rafii et al. (US 2018/0114264; hereinafter “Rafii”).
Regarding claim 5, the combination of Lombardi and Sunkavalli does not disclose wherein the requested scene is selected by a user.
In the same art of relighting objects in computer graphics scenes, Rafii teaches wherein the requested scene is selected by a user (“proper global relighting of the object,” para. 49; “multiple potential scenes may be automatically generated by the system, and the user may select one or more of these scenes,” para. 86).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Rafii to the combination of Lombardi and Sunkavalli. The motivation would have been “for a more accurate depiction of these objects in the 3D environments” (Rafii, para. 110).
Regarding claims 12 and 19, they are rejected using the same citations and rationales described in the rejection of claim 5.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ryan McCulley whose telephone number is (571)270-3754. The examiner can normally be reached Monday through Friday, 8:00am - 4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached on (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN MCCULLEY/Primary Examiner, Art Unit 2611
Prosecution Insights