DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-7, 12-13, and 15-25 are currently pending in the present application, with claims 1, 12 and 13 being independent.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 04/08/2025 have been considered by the examiner.
Specification
The abstract of the disclosure is objected to because the form and legal phraseology often used in patent claims should be avoided. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Claim Objections
Claim 1 and 5 are objected to because of the following informalities:
The “depth image” recited in claim 1, line 4 should be amended to “the corresponding depth image” for consistency.
The typographical error in claim 5 does not clearly distinguish whether claim 5 is dependent from claim 3 or 4. For examination purposes, the examiner is interpreting that claim 5 depends from claim 3.
Claim 18 objected to under 37 CFR 1.75 as being a substantial duplicate of claim 5. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 13 and 21-25 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim(s) 13 and 21-25 is/are drawn to a computer-readable storage medium, wherein the storage medium stores a computer program, where the computer-readable storage medium can be transitory, i.e., is not explicitly limited, as disclosed, as only being non-transitory computer-readable storage medium, therefore, fail(s) to fall within a statutory category of invention.
A claim directed to a computer-readable storage medium, wherein the storage medium stores a computer program embodied is non-statutory, where the computer-readable storage medium can be a signal, a carrier wave, or a data structure, per se, which are non-statutory as noted, infra. A claim directed to a signal, a carrier wave, or a data structure, per se, is non-statutory because it is not:
A process, or
A machine, or
A manufacture, or
A composition of matter.
Applicant should note that adding "non- transitory" to the claim to limit a claimed computer-readable storage medium to being statutory would be acceptable.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1 and 12-13 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhao et al. "Pointar: Efficient lighting estimation for mobile augmented reality." In European Conference on Computer Vision, pp. 678-693. Cham: Springer International Publishing, 2020, hereinafter referred to as “Zhao”.
Regarding claim 1, Zhao discloses an image processing method, comprising:
obtaining an environmental keyframe image and a corresponding depth image captured by an extended reality device (Fig. 2 and Pg. 681, Section 3; The first stage starts with an operation f(C, D, I) that generates a point cloud Po at observation o. This operation takes three inputs: (i) an RGB image, represented as C, (ii) the corresponding depth image, represented as D, and (iii) the mobile camera intrinsic I…);
determining a lighting model based on the environmental keyframe image and the depth image (Pg. 680, Section 1; PointAR takes the input of an RGB-D image and a 2D pixel coordinate…and outputs the 2nd degree spherical harmonics (SH) coefficients (i.e., a compact lighting representation of diffuse irradiance map) at a world position. Pg. 681, Section 3; We formulate the lighting estimation in mobile augmented reality as a SH coefficients regression problem as h: h(g(f(C, D, I), r)) = Sr…For the second stage, we formulate the lighting estimation as a point cloud based learning problem h that takes an incomplete point cloud Pr and outputs 2nd degree SH coefficients Sr); and
rendering an extended reality object to be rendered based on the lighting model (Fig. 2; The estimated lighting information is then used to render 3D objects, which are then combined with the original camera view into a 2D frame. Pg. 680, Section 1; The estimated SH coefficients can be directly used for rendering 3D objects, even under spatially variant lighting conditions. Pg. 681, Section 2; The estimated lighting information will then be used by the rendering engine to relit the 3D object…).
Regarding claim 12, claim 12 is the device claim (Section 1-2; mobile device. Section 5; GPU memory) of method claim 1, and is accordingly rejected using substantially similar rationale as to that which is set for with respect to claim 1.
Regarding claim 13, claim 13 is the CRM claim (Section 1-2; mobile device. Section 5; GPU memory) of method claim 1, and is accordingly rejected using substantially similar rationale as to that which is set for with respect to claim 1.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 15, and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al. "Pointar: Efficient lighting estimation for mobile augmented reality." In European Conference on Computer Vision, pp. 678-693. Cham: Springer International Publishing, 2020, hereinafter referred to as “Zhao”, in view of Byun et al. "Fast and Accurate Reconstruction of Pan-Tilt RGB-D Scans via Axis Bound Registration." arXiv preprint arXiv:1812.00240 (2018)., hereinafter referred to as “Byun”.
Regarding claim 2, Zhao discloses capturing an environmental scene to obtain the environmental keyframe image and the corresponding depth image (Fig. 2 and Pg. 681, Section 3; The first stage starts with an operation f(C, D, I) that generates a point cloud Po at observation o. This operation takes three inputs: (i) an RGB image, represented as C, (ii) the corresponding depth image, represented as D, and (iii) the mobile camera intrinsic I…).
Zhao does not disclose upon rotation of the extended reality device by a preset angle each time.
In the same art of RGB-D environmental capture, Byun discloses upon rotation of the extended reality device by a preset angle each time (Fig. 1,3 and Pg. 3, Section 3; pan and tilt angles of the local frame are denoted by α and β).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Zhao’s mobile AR lighting-estimation system to capture RGB-D environmental observations at preset angular rotations, as taught by Byun. Doing so provides a more geometrically consistent environmental coverage for reconstructing the scene. Such modifications predictably improve the spatial reliability of capturing RGB-D environmental data using a known RGB-D scanning technique (Byun Abstract; The pan-tilt RGB-D camera rotates and scans the entire scene in an automated fashion… to realize fast and accurate registration of acquired point clouds… a more accurate registration can be achieved).
Regarding claim 15, claim 15 has similar limitations as of claim 2, except it is a device claim (Section 1-2; mobile device. Section 5; GPU memory), therefore it is rejected under the same rationale as claim 2.
Regarding claim 21, claim 21 has similar limitations as of claim 2, except it is a CRM claim (Section 1-2; mobile device. Section 5; GPU memory), therefore it is rejected under the same rationale as claim 2.
Claim(s) 3-4, 6, 16-17, 19, 22-23, and 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al. "Pointar: Efficient lighting estimation for mobile augmented reality." In European Conference on Computer Vision, pp. 678-693. Cham: Springer International Publishing, 2020, hereinafter referred to as “Zhao”, in view of Ogura et al. "Illumination Estimation and Relighting using an RGB-D Camera." In VISAPP (2), pp. 305-312. 2015, hereinafter referred to as “Ogura”.
Regarding claim 3, Zhao discloses wherein the determining the lighting model based on the environmental keyframe image and the depth image comprises:
processing the environmental keyframe image to obtain a radiance map (Pg. 680, Section 1; PointAR takes the input of an RGB-D image…and outputs the 2nd degree spherical harmonics (SH) coefficients (i.e., compact lighting representation of diffuse irradiance map) at a world position. Fig. 3; We use l2 loss on both estimated SH coefficients and HDIR irradiance map reconstructed from spherical harmonics coefficients for evaluation. Fig. 6; Row 3 shows the irradiance map generated from the environment map using spherical harmonics convolution), and
calculating based on a preset internal parameter matrix and pixel coordinates of the radiance map to obtain a direction vector of a texture model (Pg. 685; Section 4.3; we leveraged the pinhole camera model and camera intrinsic of each photo in the dataset…Then we calculated the 3D point cloud coordinates (x, y, z) as: … where z is the depth value in the RGB-D photo, u and v are the photo pixel coordinates, fx and fy are the vertical and horizontal camera focal length, cx and cy are the photo optical center…T is determined by using the pixel coordinates of each rendering position on observation image from the Neural Illumination dataset in order to calculate a vector to the locale point…Pg. 686, Section 4.4; T is the number of texels in the irradiance environment map);
determining a pixel radiance value based on the radiance map (Fig. 6 and Pg. 685, Section 4.2; (1) …diffuse irradiance learning).
Zhao does not disclose processing the depth image to obtain a normal vector map; obtaining a normal vector based on the normal vector map; calculating based on the pixel radiance value, the direction vector, and the normal vector to obtain a lighting intensity and a lighting color of each azimuth coordinate of the texture model; and determining the lighting model based on the lighting intensity and the lighting color of each azimuth coordinate of the texture model
In the same art of RGB-D illumination estimation and relighting, Ogura discloses processing the depth image to obtain a normal vector map (Pg. 306, Section 3; we obtain normal map from a depth image);
obtaining a normal vector based on the normal vector map (Pg. 307, Section 3.1; we obtain the normal vector by calculating a cross product of two vectors from the neighbor points. Normal vector N(u) at a point u = (u, v) … (3) N(u) = (V(u+1), v) - V(u, v)) X (V(u, v+1) - V(u, v)), V(u, v) is a vertex map corresponding to a camera coordinate);
calculating based on the pixel radiance value, the direction vector, and the normal vector to obtain a lighting intensity and a lighting color of each azimuth coordinate of the texture model (Pg. 306, Section; Illumination environment is estimated from pixel intensity, normal map and surface reflectance. Pg. 307-308, Section 3.2; The irradiance E(x) observed at a point x is given by an integral on the distant sphere 𝛺 … (4)… L(w) is incoming light intensity along the direction vector w = (𝛳, 𝛗) and n(x) is a normal vector at a point x. max((w *n(x)), 0) shows a dot product of a normal vector and incoming light direction…we are interested in estimating the incoming light L(w)…(5)…in this paper, we consider equation (5) in RGB color space to apply it to color illumination estimation. The color pixel intensity I(x) is written as (6) …); and
determining the lighting model based on the lighting intensity and the lighting color of each azimuth coordinate of the texture model (Pg. 307-308, Section 3.2; we are interested in estimating the incoming light L(w)…The illumination is approximated with Spherical Harmonics (SH) to reduce the calculating. The illumination is shown with SH basis function y and the coefficients c…we consider equation (5) in RGB color space to apply it to color illumination estimation. Fig. 1 and Pg. 308, Section 3.4)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Ogura’s depth normal-map and inverse-illumination calculations into Zhao’s RGB-D lighting estimation system. Using the depth image to obtain surface normal, as taught by Ogura, would allow Zhao’s system to distinguish real scene geometry and how light interacts with that geometry. This would improve the resulting lighting model to be physically consistent with the observed environment, rather than relying only on image brightness or learned point-cloud features, and the modification would have been a predictable use of known RGB-D inverse lighting techniques to improve realistic rendering of virtual objects and visual coherence in AR rendering.
Regarding claim 4, Zhao in view of Ogura discloses the image processing method according to claim 3, and Zhao further discloses wherein before the processing the depth image to obtain the normal vector map, the image processing method further comprises:
obtaining a shooting position corresponding to the radiance map and the depth map (Pg. 681, Section 3; The first stage starts with an operation f(C, D, I) that generates a point cloud Po at observation o…Then g(Po, r) takes both Po and the rendering position r, and leverages a linear translation T to generated a point cloud Pr centered at r…); and
in response to the shooting position being not a preset standard position, converting coordinates of the radiance map and of the depth image to obtain a new radiance map and a new depth image, and using the new radiance map as the radiance map and using the new depth image as the depth image (Pg. 681, Section 3; In essence, this transformation simulates the process of re-centering the camera from user's current position o to the rendering position r. Pg. 685, Section 4.3; we applied a linear translation T to Po to transform the view at observation position to the rendering position…To represent the rendering position for a 3D object, we used a scale factor. This allows us (i) to compensate for the position different between the placement and the ground truth locations; (ii) to account for the potentially inaccurate depth information…we also used a rotation operation that aligns the recentered point cloud Po with ground truth environments maps in our dataset. Both the recenter and rotation operations are needed during the inference to achieve spatially-variant estimation and to account for the geometry surface and camera pose).
Zhao and Ogura are combined for the reasons set forth above with respect to claim 3.
Regarding claim 6, Zhao in view of Ogura discloses the image processing method according to claim 3, but Zhao does not disclose wherein the processing the depth image to obtain the normal vector map comprises calculating, based on depth values and pixel coordinates of the depth image, a normal vector on a three-dimensional space object corresponding to each pixel to obtain the normal vector map.
In the same art of RGB-D illumination estimation and relighting, Ogura discloses wherein the processing the depth image to obtain the normal vector map comprises:
calculating, based on depth values and pixel coordinates of the depth image, a normal vector on a three-dimensional space object corresponding to each pixel to obtain the normal vector map (Pg. 307, Section 3.1-3.2; After denoising the depth image, we can obtain a vertex map corresponding to a camera coordinate since we assume that the camera's intrinsic parameters are known…Normal vector N(u) at a point u = (u, v)…(3) N(u) = (V(u+1), v) - V(u, v)) X (V(u, v+1) - V(u, v)), V(u, v) is a vertex map corresponding to a camera coordinate…we assume that the light source is distant and objects in the scene have Lambertian surfaces…n(x)is normal vector at a point x)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Ogura’s normal vector map calculation into Zhao’s corresponding depth images. Surface normal are a conventional and standard geometric featured derived from depth data, and Ogura expressly uses those normal to estimate illumination. The combination would yield predictable results in improved lighting accuracy and estimation of observed radiance for scene geometry and incoming light.
Regarding claim 16, claim 16 has similar limitations as of claim 3, except it is a device claim (Section 1-2; mobile device. Section 5; GPU memory), therefore it is rejected under the same rationale as claim 3.
Regarding claim 17, claim 17 has similar limitations as of claim 4, except it is a device claim (Section 1-2; mobile device. Section 5; GPU memory), therefore it is rejected under the same rationale as claim 4.
Regarding claim 19, claim 19 has similar limitations as of claim 6, except it is a device claim (Section 1-2; mobile device. Section 5; GPU memory), therefore it is rejected under the same rationale as claim 6.
Regarding claim 22, claim 22 has similar limitations as of claim 3, except it is a CRM claim (Section 1-2; mobile device. Section 5; GPU memory), therefore it is rejected under the same rationale as claim 3.
Regarding claim 23, claim 23 has similar limitations as of claim 4, except it is a CRM claim (Section 1-2; mobile device. Section 5; GPU memory), therefore it is rejected under the same rationale as claim 4.
Regarding claim 25, claim 25 has similar limitations as of claim 6, except it is a CRM claim (Section 1-2; mobile device. Section 5; GPU memory), therefore it is rejected under the same rationale as claim 6.
Claim(s) 5, 18, 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al. "Pointar: Efficient lighting estimation for mobile augmented reality." In European Conference on Computer Vision, pp. 678-693. Cham: Springer International Publishing, 2020, hereinafter referred to as “Zhao”, in view of Ogura et al. "Illumination Estimation and Relighting using an RGB-D Camera." In VISAPP (2), pp. 305-312. 2015, hereinafter referred to as “Ogura”, in further view of Debevec et al. "Recovering High Dynamic Range Radiance Maps from Photographs." (1997), hereinafter referred to as “Debevec”.
Regarding claim 5, Zhao in view of Ogura discloses the image processing method according to claim 3, and Zhao further discloses wherein the processing the environmental keyframe image to obtain the radiance map comprises:
obtaining one or more color channels corresponding to the environmental keyframe image (Pg. 681, Section 3; (i) an RGB image. Pg. 684, Section 4.2; (1) …where c is the color channel (RGB), l and m are the degree and order of SH coefficients).
Zhao in view of Ogura does not disclose and processing each of the one or more color channels based on a respective calibration mapping table to obtain the radiance map.
In the same art of radiance maps, Debevec discloses and processing each of the one or more color channels based on a respective calibration mapping table to obtain the radiance map (Fig. 7a-7c and Section 2.6; color images, consisting of red, green, and blue channels…the radiance values of the three channels should be scaled so that the pixel value (Zmid, Zmid, Zmid) maps to a radiance with the same color ratios as C. Section 2.1-2.2; recovering g only requires recovering the finite number of values that g(z) can take since the domain of Z, pixel brightness values, is finite…Once the response curve g is recovered, it can be used to quickly convert pixel values to relative radiance values…the recovered radiance map is computed as an array of single-precision floating point values…color radiance map, since it stores just one exponent value for all three color values at each pixel. Fig. 8a-8c).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Debevec’s color channel response-curve calibration when generating radiance/irradiance information into the combined system of Zhao and Ogura. Doing so improves the accuracy of illumination models used for realistic AR rendering, by correcting each captured color channel using its respective response mapping to improve accuracy of the recovered radiance map (Debevec Section 1; construct an accurate map of the radiance in the scene, up to a factor of scale…the recovery of high dynamic range images will allow these methods to obtain accurate radiance values from surface specularities and from incident light sources).
Regarding claim 18, Zhao in view of Ogura discloses the image processing method according to claim 3, and Zhao further discloses wherein the processing the environmental keyframe image to obtain the radiance map comprises:
obtaining one or more color channels corresponding to the environmental keyframe image (Pg. 681, Section 3; (i) an RGB image. Pg. 684, Section 4.2; (1) …where c is the color channel (RGB), l and m are the degree and order of SH coefficients).
Zhao in view of Ogura does not disclose and processing each of the one or more color channels based on a respective calibration mapping table to obtain the radiance map.
In the same art of radiance maps, Debevec discloses and processing each of the one or more color channels based on a respective calibration mapping table to obtain the radiance map (Fig. 7a-7c and Section 2.6; color images, consisting of red, green, and blue channels…the radiance values of the three channels should be scaled so that the pixel value (Zmid, Zmid, Zmid) maps to a radiance with the same color ratios as C. Section 2.1-2.2; recovering g only requires recovering the finite number of values that g(z) can take since the domain of Z, pixel brightness values, is finite…Once the response curve g is recovered, it can be used to quickly convert pixel values to relative radiance values…the recovered radiance map is computed as an array of single-precision floating point values…color radiance map, since it stores just one exponent value for all three color values at each pixel. Fig. 8a-8c).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Debevec’s color channel response-curve calibration when generating radiance/irradiance information into the combined system of Zhao and Ogura. Doing so improves the accuracy of illumination models used for realistic AR rendering, by correcting each captured color channel using its respective response mapping to improve accuracy of the recovered radiance map (Debevec Section 1; construct an accurate map of the radiance in the scene, up to a factor of scale…the recovery of high dynamic range images will allow these methods to obtain accurate radiance values from surface specularities and from incident light sources).
Regarding claim 24, claim 24 has similar limitations as of claim 5, except it is a CRM claim (Section 1-2; mobile device. Section 5; GPU memory), therefore it is rejected under the same rationale as claim 5.
Claim(s) 7 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al. "Pointar: Efficient lighting estimation for mobile augmented reality." In European Conference on Computer Vision, pp. 678-693. Cham: Springer International Publishing, 2020, hereinafter referred to as “Zhao”, in view of Ogura et al. "Illumination Estimation and Relighting using an RGB-D Camera." In VISAPP (2), pp. 305-312. 2015, hereinafter referred to as “Ogura”, in further view of Boom et al. "Interactive light source position estimation for augmented reality with an RGB‐D camera." Computer Animation and Virtual Worlds 28, no. 1 (2017): e1686, hereinafter referred to as “Boom”.
Regarding claim 7, Zhao in view of Ogura discloses the image processing method according to claim 3. Zhao does not disclose wherein the calculating based on the pixel radiance value, the direction vector, and the normal vector to obtain the lighting intensity and the lighting color of each azimuth coordinate of the texture model comprises:
establishing a to-be-solved lighting function based on the pixel radiance value, the direction vector, the normal vector, the lighting intensity, and the lighting color; converting the to-be-solved lighting function to obtain a target lighting function.
In the same art of RGB-D illumination estimation and relighting, Ogura discloses wherein the calculating based on the pixel radiance value, the direction vector, and the normal vector to obtain the lighting intensity and the lighting color of each azimuth coordinate of the texture model comprises:
establishing a to-be-solved lighting function based on the pixel radiance value, the direction vector, the normal vector (Pg. 307, Section 3.2; Irradiance E(x) observed at a point x is given by an integral on the distant sphere 𝛺 … (4)… L(w) is incoming light intensity along the direction vector w = (𝛳, 𝛗) and n(x) is a normal vector at a point x. max((w *n(x)), 0) shows a dot product of a normal vector and incoming light direction), the lighting intensity, and the lighting color (Pg. 308, Section 3.2; we consider equation (5) in RGB color space to apply it to color illumination estimation. The color pixel intensity I(x) is written as … (6));
converting the to-be-solved lighting function to obtain a target lighting function (Pg. 307, Section 3.2; The illumination is approximated with Spherical Harmonics (SH) to reduce the calculating cost. The illumination is shown with SH basis function y and the coefficients c. The equation 4 will be represented in the following equation… (5) …);
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Zhao’s RGB-D mobile AR lighting estimation system to incorporate Ogura’s inverse-illumination calculations. Doing so allows using the relationship between pixel radiance, surface normal, light direction, and RGB illumination to derive a more accurate lighting intensity and color, yielding predictable results in improved realism and visual consistency of rendered AR objects that better matches the directional and color characteristics of the real environment.
Zhao in view of Ogura does not explicitly disclose obtaining a target energy function based on the target lighting function and discretized azimuth degrees; and calculating the target energy function to obtain the lighting intensity and the lighting color of each azimuth coordinate of the texture model.
In the same art of RGB-D light estimation for augmented reality, Boom discloses obtaining a target energy function based on the target lighting function and discretized azimuth degrees (Pg. 10; The light source position is determined by minimizing (Equation 5) the difference between the original intensity Image Io and the reconstructed image Ir (Equation 4). Pg. 11; The error function 3…is minimized…to find the light position. The error function is the L2-norm between the original image intensity Io and the reconstructed image intensity Ir…In order to minimize the error function, the light source parameters (s(p), i) are the search parameters); and
calculating the target energy function to obtain the lighting intensity and the lighting color of each azimuth coordinate of the texture model (Pg. 11-12; The minimization method searches the (s(p), i) space to minimize the Equation).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Boom’s error-function minimization into Zhao and Ogura’s combined system. Minimizing a reconstruction error between the observed image and an image generated from candidate lighting parameters provides a known and predictable way to solve inverse- lighting problem, and Boom’s optimization such modifications provides a known way to select lighting parameters that best reproduce the captured image. Such modifications would yield predictable results in improved robustness and accuracy of calculated lighting intensity/color, and thereby improving the visual realism of the lighting model used for AR rendering.
Regarding claim 20, claim 20 has similar limitations as of claim 7, except it is a device claim (Section 1-2; mobile device. Section 5; GPU memory), therefore it is rejected under the same rationale as claim 7.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JENNY NGAN TRAN whose telephone number is (571)272-6888. The examiner can normally be reached Mon-Thurs 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JENNY N TRAN/Examiner, Art Unit 2615 /ALICIA M HARRINGTON/Supervisory Patent Examiner, Art Unit 2615
Prosecution Insights