DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 and 22-26 are pending.
Claim 21 is canceled.
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claim(s) 1-20 and 22-26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Horesh (US2016/0227113) in view of Skogo et al (US2014/0268055).
Regarding claims 1 and 14, Horesh teaches a method for controlling one or more illumination devices in an eye tracker such that a measured pupil/iris contrast exceeds a predefined minimum pupil/iris contrast
(Horesh, “to perform proper gaze tracking, the iris-pupil contrast of the user's eye captured in the images may need to be above a certain threshold value”, [0014]; “maintain the iris-pupil contrast at a contrast value not less than a predefined contrast value”, [0059]; controlling light sources in an eye tracker system that measures iris-pupil contrast and optimizes power based on contrast constraints)
Horesh does not expressly disclose but Skogo teaches:
... under bright pupil conditions,
(Skogo, “If the light source is located close to the camera, in the resulting image the pupil will typically be represented by image elements associated with a relatively bright signal level compared to the Surrounding iris ... This is referred to as bright pupil (BP) imaging”, [0004]; “For some people, the contrast between iris and pupil, and hence the stability of the eye tracking algorithms, may be optimal in BP mode”, [0005]; Fig. 4a, “the light L1 emitted from the first light source 121 is expected to cause a brightpupil effect in the image data D-Img representing the eye E”, [0056]; some of light sources 124 of Horesh (Fig. 1) may be the BP light source 121 of Skogo to create a BP environment)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of Skogo into the system or method of Horesh in order to create a BP environment for achieving better iris-pupil contrast in some situations. The combination of Horesh and Skogo also teaches other enhanced capabilities.
The combination of Horesh and Skogo further teaches:
capturing images of a subject, including one or both of the subject's eyes, during predefined image capture periods with a camera;
(Horesh, “The image capturing module 202 controls the camera 122 to capture images within the field of view of the camera 122 (e.g., for eye and gaze tracking). It should be appreciated that the imaging device 100 may track one or both of the Subject's eyes and, therefore, the captured images analyzed by the imaging device 100 include at least one of the Subject’s eyes”, [0023]; capturing images of one or both eyes using the camera in a controlled (predefined) manner suitable for tracking)
illuminating, from one or more bright pupil illumination devices, one or both of the subject's eyes during the predefined image capture periods,
(Horesh, “The light source(s) 124 may be embodied as any type of light Source capable of illuminating an object being tracked by the imaging device 100 (e.g., the Subject's eyes). For example, in the illustrative embodiment, the light sources 124 are embodied as infrared light sources configured to project infrared light into the Subject's eyes”, [0020]; the light source 124 may the BP light source 121 of Skogo for creating a BP environment)
wherein each of the bright pupil devices is located sufficiently close to a lens of the camera to generate bright pupil effects; and
(Skogo, “If the light source is located close to the camera, in the resulting image the pupil will typically be represented by image elements associated with a relatively bright signal level compared to the Surrounding iris ... This is referred to as bright pupil (BP) imaging”, [0004], “FIG. 4 a illustrates a design of the eye/gaze tracker, where the first light source 121 is arranged at a distance to the image registering means 120 and a stipulated operating distance between the eye/gaze tracker and the eye E, such that the light L1 emitted from the first light source 121 is expected to cause a brightpupil effect in the image data D-Img representing the eye E. In practice, this is equivalent to locating the first light source 121 as close as possible to the image registering means 120”, [0056]; placing a light source close to the camera generates a bright pupil effect)
selectively varying the output power of at least one of the bright pupil illumination devices with a controller
to generate a bright pupil reflection intensity and
to vary a ratio between ambient and controlled light such that a measured pupil/iris contrast in a captured image exceeds a predefined minimum pupil/iris contrast.
(Horesh, Figs. 2 and 5; “The power optimization module 220 optimizes (e.g., reduces or minimizes) the total cumulative power of the light sources 124 based on the image parameters and one or more power constraints”, [0033]; “The light source power control module 222 regulates the amount of power supplied to the light sources 124 ... based on the optimized total power of the light sources 124”, [0034]; “In block 506, the imaging device 100 ensures the iris-pupil contrast (e.g., of each eye) is not less than a predefined contrast value”, [0044]; “Example 9 ... maintain the iris-pupil contrast at a contrast value not less than a predefined contrast value”, [0059]; “Example 10 ... maintain an inequality,
PNG
media_image1.png
69
213
media_image1.png
Greyscale
, ... IPC0 is the predefined contrast value”, [0060]; systematically varies the output power to the sources based on feedback, with an explicit goal of keeping pupil/iris contrast above a threshold, which necessarily adjusts the ratio of ambient and controlled light; again, the light source 124 may the BP light source 121 of Skogo for creating a BP environment)
Regarding claims 2 and 19, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method according to claim 1 wherein the output power of at least one of the bright pupil illumination devices is selectively varied based on a direct measure of pupil/iris contrast determined by pixel intensity of a pupil region relative to an iris region of one or both of the subject's eyes.
(Horesh, “The iris-pupil contrast is determined by comparing the intensity of the pixels in the iris region with the intensity of the pixels in the pupil region”, [0027])
Regarding claim 3, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method according to claim 1 wherein the output power of at least one of the bright pupil illumination devices is selectively varied based on one or more of:
i. a measure of ambient light;
ii. a measure of pupil diameter of the subject; and/or
iii. a current or recent gaze direction of the subject.
(Horesh, “the power constraint may also consider the effects of ambient”, [0044])
Regarding claim 4, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method according to claim 3 wherein the measure of ambient light is obtained from an exposure setting of the camera and/or illumination settings of the one or more bright pupil illumination devices.
(Horesh, inequality
PNG
media_image2.png
56
219
media_image2.png
Greyscale
, “Ti is an activation time of the corresponding light source 124”, [0044]; Ti is the illumination settings of the light sources; when Ei and Pi are constant, to satisfy the above inequality, Ti is determined by ambient light, or vice versus; the light source 124 may the BP light source 121 of Skogo for creating a BP environment)
Regarding claim 5, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method method according to claim 1 wherein the output power of at least one of the bright pupil illumination devices is selectively varied based on physiological parameters of the subject.
(Horesh, optimize the light source power based on various image parameters such as the iris-pupil contrast which is determined based on the determined regions of subject's iris and subject's pupil; while the size of pupil for objects are similar, the size of iris for different objects can be noticeably different; e.g., an adult has a larger measurable iris size and a child has a smaller measurable iris size; both iris size and pupil size are physiological parameters of the subject based on which the illumination power of light sources are determined; “The power optimization module 220 receives the illumination efficiency determined for each of the light sources 124 and various image parameters (e.g., the iris-pupil contrast, glint intensity, and/or the glint-eye contrast). .... The power optimization module 220 optimizes (e.g., reduces or minimizes) the total cumulative power of the light sources 124 based on the image parameters”, [0033]; “the imaging device 100 determines the iris-pupil contrast of each of the subject's eyes … In doing so, the imaging device 100 identifies various regions of the subject's eye in the captured image ... Specifically, the imaging device 100 determines the region corresponding with the subject's iris, the region corresponding with the subject's pupil”, [0038])
Regarding claim 6, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method according to claim 1 wherein the output power of at least one of the bright pupil illumination devices is selectively varied between at least two different power levels within an image capture period.
(Skogo, “an infrared illuminator is modulated in synchronicity with a camera, such that for example, a first frame contains both the illuminator signal and the ambient radiation information, and a second frame contains only the ambient radiation information. Thus, given that the variation in the ambient radiation is slow, it is possible to subtract the ambient radiation information from the image data of the first frame, and thereby the eye/gaze tracking can be improved”, [0007]; On and Off illuminator signals for 1st frame and 2nd frame, respectively; time period for the two frames may be considered as one capture period)
Regarding claim 7, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method according to claim 6
wherein the camera captures at least two images within an image capture period, and
wherein the two images are captured with different illumination or image capture settings.
(Skogo, see comments on claim 6)
Regarding claim 8, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method according to claim 7 including the step of performing image subtraction on the two images to generate a resultant image of increased pupil contrast.
(Skogo, see comments on claim 6)
Regarding claims 9 and 15, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method according to claim 1 wherein the eye tracker includes a single bright pupil illumination device configured and positioned to generate only bright pupil effects.
(Horesh, Fig. 6, single light source 124; Skogo, Fig. 4a, BP light source 121)
Regarding claim 10, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method according to claim 1 wherein the controller is configured to modulate the illumination power of the one or more bright pupil illumination devices during an image capture period.
(Skogo, see comments on claim 6)
Regarding claim 11, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method according to claim 1 wherein the eye tracker includes two bright pupil illumination devices disposed at different distances from the camera, each configured and positioned to generate only bright pupil effects.
(Horesh, Fig. 1; light sources 124 (1 to N), “one or more light sources 124”, [0014]; Skogo, Fig. 4a, BP light source 121; some of light sources 124 of Horesh (Fig. 1) may be the BP light source 121 of Skogo to create a BP environment)
Regarding claims 12 and 18, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method according to claim 11 wherein selectively varying the output power of at least one of the illumination devices includes deactivating one of the two bright pupil illumination devices during an image capture period.
(Horesh, only one light source 124 is used in Fig. 6; this can be the result of turning off all light sources except light source 1 in Fig. 1)
Regarding claims 13 and 16, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method according to claim 11 wherein each bright pupil illumination device is located sufficiently close to a lens of the camera to generate bright pupil effects but at different distances from the lens to generate different bright pupil reflection characteristics.
(Horesh, Fig. 1, all light sources 1-N are located within the imaging device 100)
Regarding claim 17, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the system according to claim 16 wherein a first bright pupil illumination device is disposed a distance of 3 mm to 15 mm from the camera while a second bright pupil illumination device is disposed a distance of 7 mm to 50 mm from the camera.
(Skogo, Fig. 4a; “the first light source 121 is arranged at a distance to the image registering means 120 and …, such that the light L1 emitted from the first light source 121 is expected to cause a brightpupil effect in the image data D-Img representing the eye E. In practice, this is equivalent to locating the first light source 121 as close as possible to the image registering means 120”, [0056]; “The second light source 122 is here arranged at a distance to the image registering means 120 and …, such that the light L2 emitted from the second light source 122 is expected to cause a dark-pupil effect in the image data D-Img representing the eye E. This typically means that the distance between the image registering means 120 and the second light source 122 is around 3 cm to 60 cm”, [0057]; “The image registering means 120 is configured to record image data D-Img representing the eye E. .... The image registering means 120 contains an image sensor with a plurality of light sensitive elements, or pixels”, [0041]; refer to Fig. 4a, when the distance between the center of the second light source 122 and the center of the image registering means 120 (a camera) is 3 cm (30 mm), the distance between the center of the first light source 121 and the center of the image registering means 120 is approximately around 1-1.5 cm (10-15 mm) measured based on the scale of Fig. 4a)
Regarding claim 20, the combination of Horesh and Skogo teaches an eye tracking system comprising:
a camera configured to capture images of a subject, including one or both of the subject's eyes, during predefined image capture periods;
one or more bright pupil illumination devices configured to illuminate one or both of the subject's eyes during the predefined image capture periods,
wherein each of the bright pupil illumination device is located sufficiently close to a lens of the camera to generate bright pupil effects; and
a controller configured to:
control an output power of the one or more bright pupil illumination devices based on the one or more control parameters
to generate a bright pupil reflection intensity and
to vary a ratio between ambient and controlled light such that a measured pupil/iris contrast in a captured image exceeds a predefined minimum pupil/iris contrast.
(Horesh, Skogo, see comments on claim 1)
Horesh further teaches:
process the captured images to perform an eye gaze tracking routine to track the eyes of the subject, the eye gaze tracking routine including determining one or more control parameters;
(Horesh, “the imaging device 100 balances the power consumed by the light sources 124 and the image quality of the images captured by the camera 122. That is, the imaging device 100 endeavors to calculate a precise and robust gaze point for gaze tracking with minimal power consumption by the light sources 124. As discussed below, the imaging device 100 optimizes the power based on various image quality constraints. For example, to perform proper gaze tracking, the iris-pupil contrast of the user's eye captured in the images may need to be above a certain threshold value”, [0014]; control parameters: iris-pupil contrast threshold and optimum power levels of the light sources)
Regarding claim 22, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method according to claim 1 wherein the output power of at least one of the bright pupil illumination devices is selectively varied based on a measure of pupil contrast of the subject's eyes from a previous image capture period.
(Horesh, Fig. 2, “the imaging device 100 establishes an environment 200 to optimize light source power. As discussed below, the imaging device 100 computes the illumination efficiency of one or more of the light sources 124 (e.g., each light source 124) given the relative location of the subject's eyes being tracked and dynamically adjusts the amount of power driving the light sources 124 in order to reduce or minimize the total power output of the light sources 124 while increasing or maximizing the pupil-iris contrast”, [0021])
Regarding claim 23, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method according to claim 1 wherein the one or more bright pupil illumination devices are driveable at one of a number of predefined non-zero voltage or current levels.
(Horesh, “the light source power control module 222 supplies an amount of power to each of the light sources 124 based on the optimized total power of the light sources 124 ... the light source power control module 222 may employ analog mechanisms (e.g., power limiting) and/or digital mechanisms (e.g., duty-cycle modification) to regulate the power supplied to each of the light sources 124”, [0034]; the light source 124 may the BP light source 121 of Skogo for creating a BP environment; supplying power at various quantized non-zero analog or digitally-limited steps inherently implies selection among a set of predefined voltage or current levels)
Regarding claim 24, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the method according to claim 1, wherein the illuminating step comprising using only one or more bright pupil illumination devices to illuminate one or both of the subject's eyes during the predefined capture periods.
(Skogo, “the eye E is illuminated by light L1 from the first light source 121 during at least one first interval”, [0044]; “the light L1 emitted from the first light source 121 is expected to cause a brightpupil effect in the image data D-Img representing the eye E”, [0056])
Regarding claim 25, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the system according to claim 14, wherein the controller is configured to selectively vary the output power of only the one or more bright pupil illumination devices such that a measured pupil/iris contrast in a captured image exceeds a predefined minimum pupil/iris contrast.
(Horesh, “The power optimization module 220 optimizes (e.g., reduces or minimizes) the total cumulative power of the light sources 124 based on the image parameters and one or more power constraints”, [0033]; “the light source power control module 222 supplies an amount of power to each of the light sources 124 based on the optimized total power of the light sources 124”, [0034]; “Example 9 includes the subject matter of any of Examples 1-8, and wherein to reduce the total cumulative power Supplied to the light sources based on one or more power constraints comprises to maintain the iris-pupil contrast at a contrast value not less than a predefined contrast value”, [0059]; the light source 124 may the BP light source 121 of Skogo for creating a BP environment; a (system) controller that individually and selectively varies the output power to achieve/satisfy a minimum iris-pupil contrast, and the claims can be interpreted as applying this to BP sources in BP mode)
Regarding claim 26, the combination of Horesh and Skogo teaches its/their respective base claim(s).
The combination further teaches the system according to claim 20, wherein the controller is configured to control an output power of only the one or more bright pupil illumination devices such that a measured pupil/iris contrast in a captured image exceeds the predefined minimum pupil/iris contrast.
(Horesh, “the light source power control module 222 supplies an amount of power to each of the light sources 124 based on the optimized total power of the light sources 124”, [0034]; “wherein to reduce the total cumulative power Supplied to the light sources based on one or more power constraints comprises to maintain the iris-pupil contrast at a contrast value not less than a predefined contrast value”, [0059]; the light source 124 may the BP light source 121 of Skogo for creating a BP environment; control of output power for the purpose of maintaining thresholded iris-pupil contrast, which, when the device is operating with only BP sources, is "only" controlling BP sources)
Response to Arguments
Applicant's arguments filed on 9/8/2025 with respect to one or more of the pending claims have been fully considered but they are not persuasive.
Regarding claim(s) 1, Applicant, in the remarks, argues that the combination of the cited reference(s) fails to teach the newly amended limitations in the claims.
The Examiner respectfully disagreed. The office action has been updated to address applicant’s argument. See the updated review comments for details.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANXUN YANG whose telephone number is (571)272-9874. The examiner can normally be reached on MON-FRI: 8AM-5PM Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JIANXUN YANG/
Primary Examiner, Art Unit 2662 10/11/2025