DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted is considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-8, 12-16 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Moltaji et al. (US Publication Number 2020/0020149 A1) in view of Du et al. (US Publication Number 2019/0213778 A1, hereinafter “Du”).
(1) regarding claim 1:
As shown in fig. 29, Moltaji disclosed an electronic device (a computing device 2900 is disclosed, see fig. 29), comprising:
circuitry configured to (para. [0051], note that the image processing workflow 200 and/or each of its individual functions, routines, subroutines, or operations may be performed by one or more processors of a computer system (e.g., the computer system 2900 of FIG. 29)):
capture, by a set of cameras in an imaging setup, an image of an object that is illuminated by a set of lighting patterns (para. [0052], note that as schematically illustrated by FIG. 2, the processing workflow 200 may start, at block 210, by acquiring i.e. using multiple cameras one or more series of images of a model. In an illustrative example, a series of images of a model may be acquired using the lighting assembly 100 of FIG. 1);
determine a brightness distribution of the captured image based on a location of at least one camera of the set of cameras (para. [0061], note that the reflectance computation operation which generates the reflectance map 260 illustrating distribution of specular reflectance on the surface of the three-dimensional object (e.g., model's head));
compute distance of each pixel associated with the captured image and the set of cameras (para. [0102], note that for each texture element (“texel”), the distance from the camera to the ray hit point is stored in the depth map and the position of the ray hit point is stored in the position map);
generate a light map of the captured image of the object based on the determined brightness distribution and the computed distance (see fig. 21, para. [0059], note that the mask generation produces, for each camera and illumination pattern, a mask that defines a region of the UV map that is best covered by the field of view of that camera. Also see, para. [0102], note that producing the distortion map involves computing depth and position maps using the ray tracing technique, as schematically illustrated by FIG. 25. For each texture element (“texel”), the distance from the camera to the ray hit point is stored in the depth map and the position of the ray hit point is stored in the position map);
Moltaji disclosed most of the subject matter as described as above except for specifically teaching to update a brightness intensity of a texture map associated with the captured image, based on the generated light map; and render the captured image based on the updated brightness intensity of the texture map.
However, Du teaches update a brightness intensity of a texture map associated with the captured image, based on the generated light map (para. [0203], note that [0203] For example, the rendering component can quantize the brightness values, and then convert the quantized brightness values and color-difference values back to the first color space. FIG. 18b shows an image (1802) in which color values of the image (1801) of FIG. 18a are modified for a “poster” effect); and render the captured image based on the updated brightness intensity of the texture map (para. [0204], note that the rendering component adjusts the brightness of brightness (Y) values that are higher than a threshold amount (e.g., a midpoint gray value), increasing the brightness of such values to a brightest value. The rendering component zeros out the color-difference (U, V) values then adds blue to at least some of the color-difference (U, V) values. The rendering component detects edges using a convolution filter or Sobel filter, then highlights the edges by decreasing the brightness values along the edges).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to update a brightness intensity of a texture map associated with the captured image, based on the generated light map; and render the captured image based on the updated brightness intensity of the texture map. The suggestion/motivation for doing so would have been improve the quality of results when fusing depth maps to generate dynamic three-dimensional (“3D”) models, applying texture details to dynamic 3D models, or rendering views of textured, dynamic 3D models (abs.). Therefore, it would have been obvious to combine Moltaji with Du to obtain the invention as specified in claim 1.
(2) regarding claim 2:
Moltaji further disclosed the electronic device according to claim 1, wherein the circuitry is further configured to: generate a three dimensional (3D) mesh of the object based on the capture of image by use of the set of cameras in the imaging setup (para. [0037], note that the calibrated images are then un-distorted based on the lens and camera sensor configurations. The undistorted images are projected onto a reconstructed polygonal mesh representing the imaged object and mapped into the UV space); and determine a center of the imaging setup based on the 3D mesh of the object (para. [0101], note that a mesh face which is directly facing the camera (i.e., if the mesh surface normal of the center point of the mesh matches the optical axis of the camera lens)).
(3) regarding claim 3:
Moltaji further disclosed the electronic device according to claim 1, wherein the object is at least one of a whiteboard or a body portion of a subject placed in a center of the imaging setup (para. [0101], note that a mesh face which is directly facing the camera (i.e., if the mesh surface normal of the center point of the mesh matches the optical axis of the camera lens), the level of distortion would be close to zero; that level would increase as the angle between the mesh surface normal and the optical axis of the camera lens increases).
(4) regarding claim 4:
Moltaji further disclosed the electronic device according to claim 3, wherein the subject corresponds to a human, the body portion corresponds to a head, and the imaging setup is configured as a polarization-based light cage (para. [0042], note that 0042], note that the height of the mounting frame may be calculated in such a manner that the head of the model 140 i.e. human head positioned within the lighting assembly would be near the center of the vertical axis of symmetry of the lighting assembly. Also see, para. [0045], note that each lighting fixture 115 may include a light source (e.g., a light emitting diode (LED) panel, a discharge lamp, a halogen lamp, etc.), a reflector, and one or more polarization filters).
(5) regarding claim 5:
Moltaji further disclosed the electronic device according to claim 1, wherein the set of lighting patterns includes at least one of: a cross-polarized omni-directional lighting pattern and gradient lighting patterns, or polarized lighting patterns including a cross-polarized lighting pattern and a parallel-polarized lighting pattern (para. [0061], note that the images that have been acquired using L.sub.p (full-on illumination with parallel filters) and L.sub.c (full-on illumination with cross-polarized filters) patterns, are fed to the reflectance computation operation (block 255) which generates the reflectance map 260 illustrating distribution of specular reflectance on the surface of the three-dimensional object (e.g., model's head), as schematically illustrated by FIG. 20).
(6) regarding claim 6:
Moltaji further disclosed the electronic device according to claim 5, wherein the circuitry is further configured to obtain the cross-polarization lighting pattern and the parallel-polarization lighting pattern, based on a polarizer installed on a polarization-based light cage associated with the imaging setup (para. [0061], note that the specular reflection is visible in the images produces using parallel polarization filters, and is filtered by cross-polarized filters. Hence, the reflectance map may be produced by computing the difference between the images produced using L.sub.p (parallel polarization) and L.sub.c (cross-polarization) patterns. The reflectance coefficient for each pixel may be represented by the maximum color value (e.g., in the blue channel) of the pixel).
(7) regarding claim 7:
Moltaji further disclosed the electronic device according to claim 5, wherein the circuitry is further configured to: obtain a set of specular-separated gradient images based on a removal of a diffuse component from the captured image, wherein the captured image is associated with the gradient lighting patterns (para. [0054], note that the second illumination and polarization pattern (denoted as L.sub.p) may provide full-on illumination with parallel-polarized filters, such that all polarization filters are vertically positioned. Using this illumination and polarization pattern allows capturing specular reflection and diffuse lights, and hence, the images produced using this pattern may be utilized for geometry reconstruction (i.e., generating a polygonal mesh representing the geometry of the imaged object) and generating the reflectance map. FIG. 4 shows an example image acquired using L.sub.p illumination and polarization pattern).
(8) regarding claim 8:
Moltaji further disclosed the electronic device according to claim 1, wherein the circuitry is further configured to: capture a test image of a whiteboard that is placed at a center of the imaging setup, wherein the whiteboard is configured to face at least one camera of the set of cameras (para. [0061], note that the images that have been acquired using L.sub.p (full-on illumination with parallel filters) and L.sub.c (full-on illumination with cross-polarized filters) patterns, are fed to the reflectance computation operation (block 255) which generates the reflectance map 260 illustrating distribution of specular reflectance on the surface of the three-dimensional object (e.g., model's head), as schematically illustrated by FIG. 20. Some regions of a human face may be more reflective than others (e.g., because of the type of skin cells and the characteristics of the underlying tissues).).
(9) regarding claim 12:
Moltaji further disclosed the electronic device according to claim 1, wherein the texture map corresponds to at least one of a diffuse texture map, a specular texture map, or a normal texture map (para. [0062], note that the normal map comprises a plurality of elements, such that each element represents a surface normal at a corresponding image point. Normal map generation is based on the theory of spherical harmonic. The lighting patterns utilized for acquiring the images are designed in such a manner that the difference between images acquired using L and L{circumflex over} patterns provides information about the surface normal direction).
The proposed rejection of claims 1, 3-4 and 8, renders obvious the steps of the method of claims 13-16 and the non-transitory computer-readable medium claim 20 because these steps occur in the operation of the proposed rejection as discussed above. Thus, the arguments similar to that presented above for claims 1, 3-4 and 8 are equally applicable to claims 13-16 and 20.
Allowable Subject Matter
Claims 9-11 and 17-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter: the prior arts made of record do not teach “wherein the circuitry is further configured to: determine camera parameters associated with each camera of the set of cameras; obtain a 3D location of each pixel associated with the captured image based on the determined camera parameters; and estimate a center of the imaging setup based on the location of at least one camera of the set of cameras, wherein the determination of the brightness distribution is based on the obtained 3D location of each pixel and the estimated center of the imaging setup”, as claimed in claims 9 and 17. Claims 10-11 and 18-19 depend on claims 9 and 17 respectively.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Coleman (US Patent Number 10,816,939 B1) disclosed a method of illuminating an environment comprises angular cycling angular light flux output from light sources in angular bins of an angularly varying light emitting device, capturing using an imager a plurality of images of the environment synchronized with the light sources, determining a light property of one or more spatial zones associated with the angular bins, and adjusting the light flux output from one or more light sources based on the light properties.
Any inquiry concerning this communication or earlier communication from the examiner should be directed to Hilina K Demeter whose telephone number is (571) 270-1676.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Y. Poon could be reached at (571) 270- 0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about PAIR system, see http://pari-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HILINA K DEMETER/Primary Examiner, Art Unit 2617
Prosecution Insights