DETAILED ACTION
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1 and 17 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly regards as the invention.
In claim 1, the limitation of “a selection of a direction for virtually illuminating the object and different from illumination of the object in the digital images” is grammatically unclear. For example, it is unclear what is different from the object in the digital images. The language can be understood as “a selection”, “a direction” or “the object” that is different from the object in the digital images. The use of the “and” does not have a proper parallel form.
Claim 17 has similar issue.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-15 and 17-21 are rejected under 35 U.S.C. 103 as being unpatentable over Purushwalkam (US 12,586,335) in view of Sun (US 11,657,546) and further in view of Balatzis (US 2023/0,186,437).
Referring to claims 1, 10 and 17, Purushwalkam discloses a method comprising:
receiving, by a processing device (fig. 4, device 400),
digital images (fig. 3, input image source 102, input view 302, generated views 303/304/305) depicting an object (fig. 1, object 101; fig. 3, broccoli; 7:65-8:14) from different angles (6:3-13, different camera pose) and
a selection of a direction (6:3-13, lighting direction) for virtually illuminating the object and different from illumination of the object in the digital images (fig. 3, different illuminating among views 302/303/304/305; 5:28-33, different viewpoint images 303/304/305 with corresponding radiance fields have different illuminating result of the chair from input view 302);
determining, by the processing device, a geometry (3:62-67, geometry of object, i.e., surface shape of the object) of the object using a machine learning model (fig. 2, 3D model 110, diffusion model 130; 15:41-45, neural network model) based on the digital images;
determining, by the processing device, parameters (6:3-13, rendering parameters) corresponding to the direction (6:3-13, lighting direction) based on the geometry of the object (fig. 2, constrained 3D representation 116 of input image 102 viewpoints); and
rendering (fig. 2, rendering 120), by the processing device, a reconstructed virtual object (fig. 1, visualization 117) that is a virtual representation of the object based on the geometry (fig. 8C, visualization of the target 3D object).
Sun discloses determining, by the processing device (fig. 12, system 1200), illumination parameters (fig. 1A, segmentation mask 16, coefficients 24-1/2402, albedo map 22, matrix 28, shading map 34) corresponding to the illumination direction based on the geometry of the object, the illumination parameters indicating how portions of the geometry of the object interacts with light from the illumination direction (fig. 1A, segmentation mask 12, illumination coefficients 24-1, albedo map 22, matrix 28 and shading mask 34 provides illuminating information of the input image 14 based on object’s surface shape); and
rendering (fig. 1B, produce a relight image 114), by the processing device, a reconstructed virtual object (fig. 1B, the image based on albedo map and shading map 114) that is a representation (fig. 1A, relight image 36) of the object based on the geometry and illuminated based on the illumination parameters (fig. 1A, relight image 36 based on segmentation mask 16, coefficients 24-1/2402, albedo map 22, matrix 28, shading map 34).
Balatzis discloses receiving digital images depicting an object from different angles (fig. 4A, left and right cam captures) (alternate interpretation).
Purushwalkam, Sun and Balatzis are analogous art because they are from from the same field of endeavor in reconstructing three-dimensional images. Before the time of the filing, it would have been obvious to a person of ordinary skill in the art, having the teaching of Purushwalkam, Sun and Balatzis before him or her to modify the reconstructing a three-dimensional object from image of Purushwalkam to include the relighting of the object of Sun, and to obtain digital images from different cameras; thereafter the reconstructed a three-dimensional object from multiple viewpoints is presented with relighting. The suggestion and/or motivation for doing so would be obtaining the advantage of more realistic lighting result (1:42-51; 2:65-3-8) as suggested by Sun and acquire improved 3D scan by Balatzis (paras.0004-0006). Therefore, it would have been obvious to combine Purushwalkam with Sun and Balatzis to obtain the invention as specified in the instant application claims.
As to claims 2, 11 and 18, Sun discloses the method of claim 1, wherein the selection of the illumination direction is indicated by an environment map (fig. 1A, albedo map 22, shading map 34) specifying a type (fig. 1A, masking 16 or shading 34) of illumination for virtually illuminating the object. (See TSM analysis above).
As to claims 3, 12 and 19, Purushwalkam discloses the method of claim 1, wherein the reconstructed virtual object is for positioning in a virtual three-dimensional environment (1:27-44, 3D virtual and augmented reality environment).
As to claims 4 and 13, Purushwalkam discloses the method of claim 1, wherein the reconstructed virtual object is a three-dimensional Gaussian (6:14-18, Gaussian distribution image).
As to claims 5 and 14, Purushwalkam discloses the method of claim 1, wherein the machine learning model includes a first transformer model (fig. 2, 3D model 110) trained on prior virtual object reconstructions (5:45-50, trained 3D model; fig. 8B, pretrained diffusion model 820; 10:58-64, iteratively trained) to reconstruct the geometry of the object.
As to claims 6 and 15, Purushwalkam discloses the method of claim 5, wherein the determining the illumination parameter is performed using a second transformer model (fig. 2, diffusion model 130) to denoise (6:40-43, diffusion model denoises) illuminated views of the reconstructed virtual object.
As to claim 7, Purushwalkam discloses the method of claim 6, wherein the first transformer model and the second transformer model are trained on images of objects from multiple known directions (fig. 3, input view 302, views 303/304/305). Sun discloses illumination (fig. 1A, relight image 36). (See TSM analysis above).
As to claim 8, Purushwalkam discloses the method of claim 1, wherein the different angles represent known camera poses (fig. 8A, first camera pose 802, second camera pose 816) relative to the object.
As to claim 9, Sun discloses the method of claim 1, wherein the determining the illumination parameters is based on initial illumination (fig. 1A, input image 14) of the object depicted in the digital images. (See TSM analysis above).
As to claim 21, Puruschwalkam discloses the method of claim 1, wherein the portions of the geometry include an edge or a corner (fig. 3, surface and shape of broccoli) of the object.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f), because the claim limitations uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitations are:
“the reconstructed virtual object is configured for positioning” in claims 3, 12 and 19,
“a second transformer model configured to denoise” in claim 6,
“means for receiving digital images” in claim 17,
“means for determining a geometry” in claim 17,
“means for determining illumination parameters” in claim 17, and
“means for rendering a reconstructed virtual object” in claim 17.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f).
Response to Arguments
Applicant’s arguments have been fully considered, but they are not deemed to be persuasive.
Applicant argues that Purushwalkam receives “an input image” rather than “input images” (pp.9-10).
Examiner considered “input images” with those images from plurality of viewpoints. Although there is one input image source 102, the field information 104/116 derived images of multiple viewpoints. The claim scope does not exclude such an extra stage for input images. Nevertheless, examiner provided Balatzis for teaching multiple input images.
Applicant argues that Sun does not disclose “illuminating parameters indicating how portions of the geometry of the object interacts with light from the illumination direction” (pp.11-12).
Initially, the dictionary definition of “geometry” is simply a surface or shape of an object. All cited prior art discloses a captured image with illuminated object. Sun discloses coefficients, map and matrix to describe lightings of objects in the input image 14 to become a relighted image 36 as shown in fig. 1A.
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See M.P.E.P 707.05(c).
US 12,347,076 discloses joint neural denoising of geometry shaped surfaces and volumes. The denoising provides relighting results.
Applicant’s amendment necessitated the new grounds of rejection presented in this Office action. Accordingly, this action is made final. See MPEP §706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire in THREE MONTHS from the mailing date of this action. In the event a first reply is filled within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date of the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136 (a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than six months from the date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to examiner Cheng-Yuan Tseng whose telephone number is (571)272-9772, and fax number is (571)273-9772. The examiner can normally be reached on Monday through Friday from 09:00 to 17:30 Eastern Time. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached on (571)272-2330. The fax phone number for the organization where this application or proceeding is assigned is (571)273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at (866)217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call (800)786-9199 (IN USA OR CANADA) or (571)272-1000.
/CHENG YUAN TSENG/Primary Examiner, Art Unit 2615
Prosecution Insights