Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 20, 21, 28, 38, 39 are amended.
Claims 22-27, 29-37 have been previously presented.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 20, 21, 22, 27-29, 31-35 is/are rejected under 35 U.S.C. 103 as being unpatentable over Suzuki (Patent No. JP 4670700 B2) in view of NPL (Zhou, Ring-Light Photometric Stereo, 2010).
Regarding claim 20 Suzuki teaches A method for 3D reconstruction of a target, the method
comprising: (Suzuki, Pg. 8, “Sub-step 2: The three-dimensional image processing device 42 substitutes
the extracted at least three three-dimensional shape data into the plane equation shown in the
equation 1, respectively, and calculates the unknowns a, b, and c by the least square method. To define
a provisional plane P ′.”)
providing a picture recording arrangement comprising an image sensor and a light source, the
light source being configured to illuminate the target along different emission directions; (Suzuki, Pg. 3,
“In order to achieve the above object, the present invention is characterized by irradiating a
measurement object with a laser beam emitted from a laser light source while changing its direction ,
scanning the measurement object with the laser beam, and in the optical path of the emission laser
beam.”)
taking a plurality of measurement pictures along the emission directions, wherein per
measurement picture only a subset of the emission directions is served by the light source; (Suzuki, Pg,
3, “ Distance detection comprising an imaging lens that condenses the reflected laser beam reflected
and rotated by the object to be measured , and a sensor in which a plurality of light receiving elements
for receiving the collected reflected laser beam are arranged. detecting the distance to the
measurement object in the vessel, three-dimensional shape and outputs the three-dimensional shape
data indicating a three-dimensional shape of the surface of the measurement object using the distance
to be oriented to the detection of the emitted laser beam”)
and reconstructing a three-dimensional shape of the target from the measurement pictures,
(Suzuki, Pg. 3, “The laser beam emitted from the imaging device and the three-dimensional shape
imaging device is reflected and guided to the measurement object from a different direction from the
laser beam emitted from the three-dimensional shape imaging device and irradiated on the
measurement target.”)
wherein taking the plurality of measurement pictures comprises illuminating the target in an
indirect manner so that at least some of the emission directions point next to the target and not onto
the target (Suzuki, Pg. 20, “If the position and orientation can be specified, it is not always necessary to use various objects as described above. For example, sensors for detecting the positions and orientations of the reflection mirrors 31 and 32 in the camera coordinate system C are provided in the reflection mirrors 31 and 32, respectively, and the positions and orientations of the reflection mirrors 31 and 32 detected by the sensors are used.”)
However, Suzuki is silent about and wherein orientations of the emission directions relative to the image sensor are fixed wherein the light source comprises one light-emitting unit for each one of the emission directions, and wherein positions of the light-emitting units relative to one another are fixed.
Zhou teaches and wherein orientations of the emission directions relative to the image sensor are fixed wherein the light source comprises one light-emitting unit for each one of the emission directions, and wherein positions of the light-emitting units relative to one another are fixed (Zhou, Pg. 270, “Fig.3. Geometric explanations of the components of the ‘ring-light ambiguity’. The first row shows the transformations induced to lighting directions in the projective plane. The second row illustrates the corresponding transformations to a 3D shape.”, Examiner interpret the invention’s light source to be constrained in a fixed view-centered cone)
Therefore it would have been obvious for an ordinary skilled person in the art before the
effective filing date of claimed invention to have modified Suzuki art by including and wherein orientations of the emission directions relative to the image sensor are fixed wherein the light source comprises one light-emitting unit for each one of the emission directions, and wherein positions of the light-emitting units relative to one another are fixed as taught by Zhou and use that with Suzuki’s 3D reconstruction method and picture recording arrangement.
The motivation for the combination is to introduce geometric constrain to the light emit direction.
Regarding claim 21 Suzuki teaches The method according to claim 20 according to wherein
taking the plurality of measurement pictures comprises illuminating the target by the light source exclusively in the indirect manner, and wherein a diameter of the light source is at most 0.3 m, seen in
top view of the image sensor. (Suzuki, Pg. 7, “The method according to The method according to
wherein taking the plurality of measurement pictures comprises illuminating the target by the light
source exclusively in an indirect manner, and wherein a diameter of the light source is at most 0.3 m,
seen in top view of the image sensor.”)
Regarding claim 22, Suzuki is silent about The method according to claim 20 according to
wherein for each one of the emission directions exactly one measurement picture is taken, and per
measurement picture exactly one of the emission directions is served by the light source, and wherein a
distance between the picture recording arrangement and the target is between 0.2 m and 6 m, inclusive.
Zhou teaches The method according to claim 20 according to wherein for each one of the
emission directions exactly one measurement picture is taken, and per measurement picture exactly
one of the emission directions is served by the light source, and wherein a distance between the picture
recording arrangement and the target is between 0.2 m and 6 m, inclusive. (Zhou, Pg. 276, “This weak
perspective effects is illustrated in (b). To ensure the opening angle of the cone is larger than 10
degrees, the distance between the camera and captured objects should be within 1.7 meters.”)
Therefore it would have been obvious for an ordinary skilled person in the art before the
effective filing date of claimed invention to have modified Suzuki art by including The method according
to claim 20 according to wherein for each one of the emission directions exactly one measurement
picture is taken, and per measurement picture exactly one of the emission directions is served by the
light source, and wherein a distance between the picture recording arrangement and the target is between 0.2 m and 6 m, inclusive as taught by Zhou and use that with Suzuki’s 3D reconstruction
method and picture recording arrangement.
Regarding claim 27, Suzuki is silent about The method according to claim 20 according to
wherein there are at least six and at most 60 of the emission directions.
Zhou teaches The method according to claim 20 according to wherein there are at least six and
at most 60 of the emission directions. (Zhou, Pg. 268, “Fig.2. Ring-light photometric stereo. (a) Lighting
directions lie on a view centered cone. The term ω denotes the cone opening angle. (b) In the projective
plane, these lights lie on a ring centered at origin (i.e. viewing direction). (c) When there is a linear
ambiguity, these lights lie on a general planar conic. Our algorithm resolves this linear ambiguity by
mapping lights back to their canonic positions.”)
Therefore it would have been obvious for an ordinary skilled person in the art before the
effective filing date of claimed invention to have modified Suzuki art by including The method according
to claim 20 according to wherein there are at least six and at most 60 of the emission directions as
taught by Zhou and use that with Suzuki’s 3D reconstruction method and picture recording
arrangement.
Regarding claim 28, Suzuki is silent about The method according to The method according to
wherein the light-emitting units are arranged in a circular manner, seen in top view of the image sensor.
Zhou teaches The method according to The method according to wherein the light-emitting units are arranged in a circular manner, seen in top view of the image sensor. (Zhou, Pg. 268, “Fig.2. Ring-light photometric stereo. (a) Lighting directions lie on a view centered cone. The term ω denotes the cone opening angle. (b) In the projective plane, these lights lie on a ring centered at origin (i.e. viewing direction). (c) When there is a linear ambiguity, these lights lie on a general planar conic. Our algorithm resolves this linear ambiguity by mapping lights back to their canonic positions.”)
Therefore it would have been obvious for an ordinary skilled person in the art before the
effective filing date of claimed invention to have modified Suzuki art by including The method according to wherein the light-emitting units are arranged in a circular manner, seen in top view of the image sensor as taught by Zhou and use that with Suzuki’s 3D reconstruction method and picture recording arrangement.
Regarding claim 29, Suzuki is silent about The method according to The method according to
wherein the light source comprises one or less light-emitting units than emission directions, the one or
less light-emitting units than the emission directions moving or rotating relative to the image sensor.
Zhou teaches The method according to The method according to wherein the light source
comprises one or less light-emitting units than emission directions, the one or less light-emitting units
than the emission directions moving or rotating relative to the image sensor. (Zhou, Pg. 269, “Rθ rotates
the lighting and normal directions around the origin.”)
Therefore it would have been obvious for an ordinary skilled person in the art before the
effective filing date of claimed invention to have modified Suzuki art by including The method according
to The method according to wherein the light source comprises one or less light-emitting units than
emission directions, the one or less light-emitting units than the emission directions moving or rotating relative to the image sensor as taught by Zhou and use that with Suzuki’s 3D reconstruction method and
picture recording arrangement.
Regarding claim 31, Suzuki is silent about The method according claim 20 according to wherein
the light source independently emits a plurality of beams having different colors along at least some of
the emission directions.
Zhou teaches The method according to claim 20 according to claim 20 wherein the light source
independently emits a plurality of beams having different colors along at least some of the emission
directions. (Zhou, Pg. 270, “Fig.3. Geometric explanations of the components of the ‘ring-light
ambiguity’. The first row shows the transformations induced to lighting directions in the projective
plane. The second row illustrates the corresponding transformations to a 3D shape.”)
Therefore it would have been obvious for an ordinary skilled person in the art before the
effective filing date of claimed invention to have modified Suzuki art by including The method according
claim 20 according to wherein the light source independently emits a plurality of beams having different
colors along at least some of the emission directions as taught by Zhou and use that with Suzuki’s 3D
reconstruction method and picture recording arrangement.
Regarding claim 32 Suzuki teaches The method according to claim 20 according to wherein the
light source emits only a single beam of light along at least some of the emission directions (Suzuki, Pg.
6, “In order to achieve the above object, the present invention is characterized by irradiating a
measurement object with a laser beam emitted from a laser light source while changing its direction ,
scanning the measurement object with the laser beam, and in the optical path of the emission laser
beam.”)
Regarding claim 33 Suzuki teaches The method according to claim 20 according to wherein the
light source comprises an emitter for non-visible radiation. (Suzuki, Pg. 6, “In order to achieve the above
object, the present invention is characterized by irradiating a measurement object with a laser beam
emitted from a laser light source while changing its direction , scanning the measurement object with
the laser beam, and in the optical path of the emission laser beam.”)
Regarding claim 34 Suzuki teaches The method according to claim 20 according to wherein, for
all or for some of the emission directions, there is one emitter for near- infrared radiation per emission
direction of the emission directions. (Suzuki, Pg. 6, “In order to achieve the above object, the present
invention is characterized by irradiating a measurement object with a laser beam emitted from a laser light source while changing its direction , scanning the measurement object with the laser beam, and in
the optical path of the emission laser beam.”)
Regarding claim 35 Suzuki teaches The method according to claim 20 according to wherein the
picture recording arrangement comprises a 3D-sensor. (Suzuki, Pg. 3, “a sensor in which a plurality of
light receiving elements for receiving the collected reflected laser beam are arranged. detecting the
distance to the measurement object in the vessel, three-dimensional shape and outputs the three-
dimensional shape data indicating a three-dimensional shape of the surface of the measurement object
using the distance to be oriented to the detection of the emitted laser beam”)
Claim(s) 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Suzuki (Patent No. JP
4670700 B2) in view of NPL (Zhou, Ring-Light Photometric Stereo, 2010), in further view of NPL (Logothetis, Near-Field Photometric Stereo in Ambient Light, 01/16/2017)
Regarding claim 23 Suzuki is silent about The method according to claim 20 according to
wherein taking the plurality of measurement pictures comprises taking the pictures under low-light
conditions so that there is no illumination source to illuminate the target despite the light source of the
picture recording arrangement.
Logothetis teaches The method according to claim 20 according to wherein taking the plurality
of measurement pictures comprises taking the pictures under low-light conditions so that there is no
illumination source to illuminate the target despite the light source of the picture recording
arrangement. (Logothetis, Pg. 1, “In addition, the environment where the images are acquired is
assumed dark, considering negligible ambient light.”)
Therefore it would have been obvious for an ordinary skilled person in the art before the
effective filing date of claimed invention to have modified Suzuki art by including The method according
to claim 20 according to wherein taking the plurality of measurement pictures comprises taking the
pictures under low-light conditions so that there is no illumination source to illuminate the target
despite the light source of the picture recording arrangement as taught by Logothetis and use that with
Suzuki’s 3D reconstruction method and picture recording arrangement.
The motivation for the combination is to improve measurement environment by reducing
ambient light.
Claim(s) 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Suzuki (Patent No. JP
4670700 B2) in view of NPL (Zhou, Ring-Light Photometric Stereo, 2010), in further view of Colagrande (Patent No. US 20210006681 A1) and in further view of Forutanpour (Patent No. US 20170301078 A1), and in further view Klimovski (Patent No. US 20170046824 A1)
Regarding claim 24 Suzuki teaches and wherein reconstructing the three-dimensional shape of
the target (Suzuki, Pg. 3, “From measurement object A reflection mirror that reflects reflected light and
guides it to the three-dimensional shape imaging device, and a measurement object represented by
three-dimensional shape data based on the reflected light that is guided from the measurement object
to the three-dimensional shape imaging device via the reflection mirror 3D shape data for synthesizing
the 3D shape of the surface of the object to the 3D shape of the surface of the measurement object represented by the 3D shape data based on the reflected light directly guided from the measurement
object to the 3D shape imaging device And synthesizing means.”)
However, Suzuki is silent about The method according to claim 20 according to wherein taking
the plurality of measurement pictures comprises taking the pictures while the target is illuminated by an
exterior illumination source, wherein taking the plurality of measurement pictures includes, prior to
taking the measurement pictures, analyzing illumination conditions of the target with the light source
being turned off by taking a reference image, includes subtracting the illumination conditions present in
the reference image from the measurement pictures.
Colagrande teaches The method according to claim 20 according to wherein taking the plurality
of measurement pictures comprises taking the pictures while the target is illuminated by an exterior
illumination source (Colagrande, “[0008] The second category consists of non-integrated systems,
generally of medium-small format, being based on a matrix camera and a traditional lighting system
arranged and managed in a manual way;”)
Forutanpour teaches wherein taking the plurality of measurement pictures includes, prior to
taking the measurement pictures, analyzing illumination conditions of the target with the light source
being turned off by taking a reference image (Forutanpour, “[0016] Furthermore, the routine 200 can
obtain various images of the electronic device 103 under different conditions, for example under
different lighting conditions (e.g., no lights on, all lights on, only infrared illumination, etc.), under
different operating conditions (e.g., with a display of the electronic device 103 in an “off” mode, with
the display in an “on” mode, with the display presenting a predetermined image (e.g. a QR code or other
known image), etc.). In other embodiments, the routine 200 can obtain a single image of the electronic
device 103 and/or images the electronic device 103 under a single operating condition. With reference
again to the routine 200, the image or images obtained by the imaging device 101 can be transmitted to
the computing device 105, such as via the communications link 109, for analysis and evaluation. ”)
Klimovski teaches includes subtracting the illumination conditions present in the reference
image from the measurement pictures. (Klimovski, “[0013] A signal processor is configured to remove
lighting effects from the at least one reference image to create at least one processed image”)
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Suzuki art by including The method according to claim 20according to wherein taking the plurality of measurement pictures comprises taking the pictures while the target is illuminated by an exterior illumination source, wherein taking the plurality of measurement pictures includes, prior to taking the measurement pictures, analyzing illumination conditions of the target with the light source being turned off by taking a reference image, includes subtracting the illumination conditions present in the reference image from the measurement pictures as taught by Colagrande, Forutanpour, Klimovski and use that with Suzuki’s 3D reconstruction method and picture recording arrangement.
The motivation for the combination is to improve the accuracy of the measurement pictures.
Claim(s) 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Suzuki (Patent No. JP
4670700 B2) in view of NPL (Zhou, Ring-Light Photometric Stereo, 2010), in further view of Choi (Patent No. US 20200066036 A1) and in further view of Kumagai (Patent No. US 20160216109 A1)
Regarding claim 25, Suzuki is silent about The method according to claim 20 according to
wherein taking the plurality of measurement pictures includes estimating a three-dimensional
representation of a scene in which the target is located within and/or out of a field of view of the image
sensor, and wherein reconstructing the three-dimensional shape of the target includes estimating an
influence of reflective surfaces next to the target.
Choi teaches The method according to claim 20 according to wherein taking the plurality of
measurement pictures includes estimating a three-dimensional representation of a scene in which the
target is located within and/or out of a field of view of the image sensor (Choi, “[0158] The RPN is a
network that proposes a region in which an object is expected to exist in a space corresponding to a
field of view (FOV) of a sensor”)
Kumagai teaches reconstructing the three-dimensional shape of the target includes estimating
an influence of reflective surfaces next to the target. (Kumagai, “[0035] In accordance with the
embodiment of claim 2, after obtaining image information, the first searching unit is set up to estimate
the existence estimation region where the reflection target exists based on the image information, and
to detect the reflection target in the existence estimation region, so that the existence estimation region
of the reflection target can be located at a stroke for directly accessing to the existence estimation
region and for effectively detecting the reflection target.”)
Therefore it would have been obvious for an ordinary skilled person in the art before the
effective filing date of claimed invention to have modified Suzuki art by including The method according
to claim 20 according to wherein taking the plurality of measurement pictures includes estimating a
three-dimensional representation of a scene in which the target is located within and/or out of a field of
view of the image sensor, and wherein reconstructing the three-dimensional shape of the target
includes estimating an influence of reflective surfaces next to the target as taught by Choi, Kumagai and
use that with Suzuki’s 3D reconstruction method and picture recording arrangement.
The motivation for the combination is to improve the accuracy of the reconstruction of the 3D
Target with image sensor.
Claim(s) 26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Suzuki (Patent No. JP
4670700 B2) in view of NPL (Zhou, Ring-Light Photometric Stereo, 2010), in further view of Harrington (Patent No. US 20200066036 A1).
Regarding claim 26, Suzuki is silent about The method according to claim 20 according to
wherein an emission angle between an optical axis of the image sensor and at least some of the
emission directions is between 30~ and 750 inclusive, wherein, for at least some of the emission
directions, an emission angle width per emission direction is between 15 and inclusive, and wherein
radiation emitted into the emission directions is emitted out of a field of view of the image sensor.
Harrington teaches The method according to claim 20 according to wherein an emission angle
between an optical axis of the image sensor and at least some of the emission directions is between 30~
and 750 inclusive, wherein, for at least some of the emission directions, an emission angle width
per emission direction is between 15 and inclusive, and wherein radiation emitted into the emission
directions is emitted out of a field of view of the image sensor. (Harrington, “[0013] Preferably, to
strengthen this effect, at least one of the single-direction stereo pairs of images can be generated when
the object is illuminated from a direction which is close to the tangential direction of the object. For
example, if the surface of the object exhibits very small raised or depressed formations, at least one of
the directions may intercept a region of the object at an angle to the surface which is less than 30
degrees, or less than 20 degrees. However, if the surface of the object includes larger raised or
depressed formations, the angle may be chosen to be higher, such as up to 80 degrees.”)
Therefore it would have been obvious for an ordinary skilled person in the art before the
effective filing date of claimed invention to have modified Suzuki art by including The method according
to claim 20 according to wherein an emission angle between an optical axis of the image sensor and at
least some of the emission directions is between 30~ and 750 inclusive, wherein, for at least some of the
emission directions, an emission angle width per emission direction is between 15 and inclusive, and
wherein radiation emitted into the emission directions is emitted out of a field of view of the image sensor as taught by Harrington and use that with Suzuki’s 3D reconstruction method and picture
recording arrangement.
The motivation for the combination is to clarify the light emission direction.
Claim(s) 30 is/are rejected under 35 U.S.C. 103 as being unpatentable over Suzuki (Patent No. JP
4670700 B2) in view of NPL (Zhou, Ring-Light Photometric Stereo, 2010), in further view of Colagrande (Patent No. US 20210006681 A1).
Regarding claim 30 Suzuki is silent about The method according to claim 20 according to
wherein the light source comprises an additional light-emitting unit configured for direct lighting of the
target.
Colagrande teaches The method according to claim 20 according to wherein the light source
comprises an additional light-emitting unit configured for direct lighting of the target. (Colagrande,
“[0008] The second category consists of non-integrated systems, generally of medium-small format,
being based on a matrix camera and a traditional lighting system arranged and managed in a manual
way;”)
Therefore it would have been obvious for an ordinary skilled person in the art before the
effective filing date of claimed invention to have modified Suzuki art by including The method according
to claim 20 according to wherein the light source comprises an additional light-emitting unit configured
for direct lighting of the target as taught by Colagrande and use that with Suzuki’s 3D reconstruction
method and picture recording arrangement.
The motivation for the combination is to improve light emit geometry.
Claim(s) 36, 37 is/are rejected under 35 U.S.C. 103 as being unpatentable over Suzuki (Patent
No. JP 4670700 B2) in view of NPL (Zhou, Ring-Light Photometric Stereo, 2010), in further view of NPL (Yeh, Shape-from-Shifting: Uncalibrated Photometric Stereo with a Mobile Device, 11/16/2017)
Regarding claim 36, Suzuki is silent about The method according to claim 20 according to
wherein the picture recording arrangement is a single mobile device including the image sensor and the
light source.
Yeh teaches The method according to claim 20 according to wherein the picture recording
arrangement is a single mobile device including the image sensor and the light source. (Yeh, Pg. 553,
“Hardware Setup: Two polarizers with opposite polarization directions (marked with blue boxes in
Figure 2(a)) were attached to the camera and flash light on an iPhone 6 through a custom 3D printed
add-on component as shown in Figure 2(b). Specifically, these two polarizers were cut from the same polarization film. The polarizer on the flash light was first glued on the add-on component, and the
orientation of the polarizer on the camera was carefully tuned and fixed at the position of extinction.
The total cost for the polarizers and 3D printed component is less than 5 USD.”)
Therefore it would have been obvious for an ordinary skilled person in the art before the
effective filing date of claimed invention to have modified Suzuki art by including The method according
to claim 20 according to wherein the picture recording arrangement is a single mobile device including
the image sensor and the light source as taught by Yeh and use that with Suzuki’s 3D reconstruction
method and picture recording arrangement.
The motivation for the combination is to improve photometric stereo with mobile device/smart
phone.
Regarding claim 37, Suzuki is silent about The method according to claim 20 according to
wherein the picture recording arrangement is a smart phone.
Yeh teaches The method according to claim 20 according to wherein the picture recording
arrangement is a smart phone. (Yeh, Pg. 552, “Portability and Accessibility: Since Mobile SfS only
requires a mobile device, such as an iPhone, and a small widget, it is very portable and user friendly. We
believe Mobile SfS will be a powerful tool for conservators because it drastically simplifies the 3D
surface acquisition process by allowing objects to be scanned in their natural environment and without
the need of calibration hardware such as a mirrored ball.”)
Therefore it would have been obvious for an ordinary skilled person in the art before the
effective filing date of claimed invention to have modified Suzuki art by including The method according
to claim 20 according to wherein the picture recording arrangement is a smart phone as taught by Yeh
and use that with Suzuki’s 3D reconstruction method and picture recording arrangement.
Claim(s) 38, 39 is/are rejected under 35 U.S.C. 103 as being unpatentable over NPL (Yeh, Shape-
from-Shifting: Uncalibrated Photometric Stereo with a Mobile Device, 11/16/2017) in view of Suzuki
(Patent No. JP 4670700 B2) in further view of NPL (Zhou, Ring-Light Photometric Stereo, 2010).
Regarding claim 38, Yeh teaches A mobile device comprising: an image sensor; a light source;
and a processor (Yeh, Pg. 552, “Portability and Accessibility: Since Mobile SfS only requires a mobile
device, such as an iPhone, and a small widget, it is very portable and user friendly. We believe Mobile
SfS will be a powerful tool for conservators because it drastically simplifies the 3D surface acquisition
process by allowing objects to be scanned in their natural environment and without the need of
calibration hardware such as a mirrored ball.”)
wherein the light source is configured to illuminate a target along different emission directions
(Yeh, Pg. 553, “Hardware Setup: Two polarizers with opposite polarization directions (marked with blue
boxes in Figure 2(a)) were attached to the camera and flash light on an iPhone 6 through a custom 3D
printed add-on component as shown in Figure 2(b). Specifically, these two polarizers were cut from the
same polarization film. The polarizer on the flash light was first glued on the add-on component, and the
orientation of the polarizer on the camera was carefully tuned and fixed at the position of extinction.
The total cost for the polarizers and 3D printed component is less than 5 USD. ”)
wherein the image sensor is configured to take a plurality of measurement pictures along the
emission directions (Yeh, Pg. 554, “Fig. 5. Image pre-processing: Step 1: nine images are captured; Step
2: images are registered with the SIFT function; Step 3: images are cropped to display the same region. ”)
wherein, per measurement picture, only a subset of the emission directions is served by the
light source (Yeh, Pg. 554, “Due to the limited power of the iPhone flash, the examples shown were
captured in a dark room or in the evening to minimize the effect of ambient light.”)
wherein the processor is configured to reconstruct a three-dimensional shape of the target
from the measurement pictures, (Yeh, Pg. 554, “Fig. 4. Acquisition procedure with Mobile SfS: Hand-held
Mobile SfS faces to the object. Slightly shift the phone and take one image at each position. Nine images
are taken in this paper for the surface normal reconstruction.”)
However, Yeh is silent about wherein the picture recording arrangement comprising the image sensor and the light source is configured such that the target is illuminated in an indirect manner so that at least some of the emission directions point next to the target and not onto the target, and out of a field of view of the image sensor wherein the light source comprises one light-emitting unit for each one of the emission directions, wherein positions of the light-emitting units relative to one another are fixed, and wherein the mobile device is a smart phone.
Suzuki teaches wherein the picture recording arrangement comprising the image sensor and the light source is configured such that the target is illuminated in an indirect manner so that at least some of the emission directions point next to the target and not onto the target, and out of a field of view of the image sensor (Suzuki, Pg. 3, “Distance detection comprising an imaging lens that condenses the reflected laser beam reflected and rotated by the object to be measured, and a sensor in which a plurality of light receiving elements for receiving the collected reflected laser beam are arranged. detecting the distance to the measurement object in the vessel, three-dimensional shape and outputs the three-dimensional shape data indicating a three- dimensional shape of the surface of the measurement object using the distance to be oriented to the detection of the emitted laser beam”)
Zhou teaches wherein the light source comprises one light-emitting unit for each one of the emission directions, wherein positions of the light-emitting units relative to one another are fixed, and wherein the mobile device is a smart phone. (Zhou, Pg. 270, “Fig.3. Geometric explanations of the components of the ‘ring-light ambiguity’. The first row shows the transformations induced to lighting directions in the projective plane. The second row illustrates the corresponding transformations to a 3D shape.”, Examiner interpret the invention’s light source to be constrained in a fixed view-centered cone”)
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Yeh art by including wherein the picture recording arrangement is configured such that the target is illuminated in an indirect manner so that at least some of the emission directions point next to the target and not onto the target ,and out of a field of view of the image sensor wherein the light source comprises one light-emitting unit for each one of the emission directions, wherein positions of the light-emitting units relative to one another are fixed, and wherein the mobile device is a smart phone as taught by Suzuki and use that with Yeh’s uncalibrated photometric stereo with a mobile device.
The motivation for the combination is to improve the accuracy of reconstruction using
photometric stereo with a phone.
Regarding claim 39, Yeh teaches A method for 3D reconstruction of a target, the method
comprising: providing a single smart phone comprising an image sensor and a light source, the light
source being configured to illuminate the target along different emission directions; (Yeh, Pg. 552,
“Portability and Accessibility: Since Mobile SfS only requires a mobile device, such as an iPhone, and a
small widget, it is very portable and user friendly. We believe Mobile SfS will be a powerful tool for
conservators because it drastically simplifies the 3D surface acquisition process by allowing objects to be
scanned in their natural environment and without the need of calibration hardware such as a mirrored
ball.”)
taking a plurality of measurement pictures along the emission directions, wherein, per
measurement picture, only a subset of the emission directions is served by the light source; and (Yeh,
Pg. 554, “Due to the limited power of the iPhone flash, the examples shown were captured in a dark
room or in the evening to minimize the effect of ambient light.”)
reconstructing a three-dimensional shape of the target from the measurement pictures, (Yeh,
Pg. 554, “Fig. 4. Acquisition procedure with Mobile SfS: Hand-held Mobile SfS faces to the object. Slightly
shift the phone and take one image at each position. Nine images are taken in this paper for the surface
normal reconstruction.”)
and wherein orientations of the emission directions relative to the image sensor are fixed. (Yeh, Pg. 554, “Fig. 5. Image pre-processing: Step 1: nine images are captured; Step 2: images are registered with the SIFT function; Step 3: images are cropped to display the same region.”)
However, Suzuki is silent about wherein taking the plurality of measurement pictures comprises
illuminating the target in an indirect manner so that at least some of the emission directions point next
to the target and not onto the target, and out of a field of view of the image sensor wherein the light source comprises one light-emitting unit for each one of the emission directions, and wherein positions of the light-emitting units relative to one another are fixed.
Suzuki teaches wherein taking the plurality of measurement pictures comprises illuminating the
target in an indirect manner so that at least some of the emission directions point next to the target and
not onto the target, and out of a field of view of the image sensor (Suzuki, Pg. 3, “Distance detection
comprising an imaging lens that condenses the reflected laser beam reflected and rotated by the object
to be measured , and a sensor in which a plurality of light receiving elements for receiving the collected
reflected laser beam are arranged. detecting the distance to the measurement object in the vessel,
three-dimensional shape and outputs the three-dimensional shape data indicating a three-dimensional
shape of the surface of the measurement object using the distance to be oriented to the detection of
the emitted laser beam”)
Zhou teaches wherein the light source comprises one light-emitting unit for each one of the emission directions, and wherein positions of the light-emitting units relative to one another are fixed. (Zhou, Pg. 270, “Fig.3. Geometric explanations of the components of the ‘ring-light ambiguity’. The first row shows the transformations induced to lighting directions in the projective plane. The second row illustrates the corresponding transformations to a 3D shape.”, Examiner interpret the invention’s light source to be constrained in a fixed view-centered cone)
Therefore it would have been obvious for an ordinary skilled person in the art before the
effective filing date of claimed invention to have modified Yeh art by including wherein taking the plurality of measurement pictures comprises illuminating the target in an indirect manner so that at least some of the emission directions point next to the target and not onto the target, and out of a field of view of the image sensor wherein the light source comprises one light-emitting unit for each one of the emission directions, and wherein positions of the light-emitting units relative to one another are fixed as taught by Suzuki and Zhou, and use that with Yeh’s uncalibrated photometric stereo
with a mobile device.
Response to Arguments
Applicant's arguments filed 07/01/2026 have been fully considered but they are not persuasive.
Applicant argues However, independent claim 20 requires that "orientations of the emission directions relative to the image sensor [are] fixed." This is the exact opposite of what is done in Suzuki as scanning means that the laser beam is swept all over the object to be scanned.
Examiner replies that, Zhou states in Pg. 270, “Fig.3. Geometric explanations of the components of the ‘ring-light ambiguity’. The first row shows the transformations induced to lighting directions in the projective plane. The second row illustrates the corresponding transformations to a 3D shape.”, Examiner interpret the invention’s light source to be constrained in a fixed view-centered cone” Thus, Zhou teaches a sensors configured in a fixed geometric location.
Applicant argues Further, the mirrors 31, 32 in Suzuki are a part of the overall device. This is clearly evident from Suzuki as the mirrors 31, 32 are a core element of the device. Accordingly, Suzuki
does not teach the use of indirect lighting. Rather, all light from the laser, upon leaving the (overall)l device of Suzuki, hits directly the object to be measured.
Examiner replies that, Suzuki states in Pg. 20, “If the position and orientation can be specified, it is not always necessary to use various objects as described above. For example, sensors for detecting the positions and orientations of the reflection mirrors 31 and 32 in the camera coordinate system C are provided in the reflection mirrors 31 and 32, respectively, and the positions and orientations of the reflection mirrors 31 and 32 detected by the sensors are used.” Thus, Suzuki teaches the laser from the sensor reflect from the mirrors before hitting the object, indicating a indirect manner.
Furthermore, Suzuki teaches indirect lighting, Zhou teaches constrained/fixed position lighting, and Choi teaches reconstruction of 3D model using image sensor. Thus, rendering the subject-matter obvious.
Regarding the remaining arguments: Applicant argues with respect to the amended claim language, which is fully addressed in the prior art rejections set forth below.
Regarding the remaining arguments: Applicant argues with respect to the amended claim language, which is fully addressed in the prior art rejections set forth below.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHAK FUNG A LAM whose telephone number is (571)272-9823. The examiner can
normally be reached Monday-Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a
USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use
the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor,
Said Broome can be reached at 5712722931. The fax phone number for the organization where this
application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from
Patent Center. Unpublished application information in Patent Center is available to registered users. To
file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and
https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/C.A.L./Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
Prosecution Insights