DETAIL OFFICE ACTIONS
The United States Patent & Trademark Office appreciates the response filed for the current application that is submitted on 11/19/2025. The United States Patent & Trademark Office reviewed the following documents submitted and has made the following comments below.
Amendment
Applicant submitted amendments on 11/19/2025. The Examiner acknowledges the amendment and has reviewed the claims accordingly.
Applicant Arguments:
Applicant/s state/s that the cited prior arts do not teach the amended claims 1 and 11, specially, the limitation “selectively obtain a feature point of the eye from the coded image or reconstruct an original image of the eye from the coded image, obtain gaze information of a user based on the feature point, and obtain authentication information of the user if the original image is reconstructed.”; therefore, the rejection under 35 U.S.C. 102 should be withdrawn.
Examiner’s Responses:
Applicant’s arguments and amendments, see Remarks, filed 11/19/2025, with respect to the rejection(s) of claim(s) 1 and 11 under 35 U.S.C. 102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration of amendments, a new ground(s) of rejection is made in view of Tesdahl et al. (US-20240028117-A1) in view of Henrik et al. (US-20190278987-A1), and further in view of Lee et al. (US-20190102595-A1).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5-6, 10-13, 15-16 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tesdahl et al. (US-20240028117-A1, claimed priority of U.S. application 17/832,424 filed 06/03/2022, hereinafter Tesdahl) in view of Henrik et al. (US-20190278987-A1, hereinafter Henrik), and further in view of Lee et al. (US-20190102595-A1, hereinafter Lee).
CLAIM 1
Regarding Claim 1, Tesdahl teaches an electronic device (Tesdahl, Abstract: “Eye and hand tracking systems in head-mounted display (HMD) devices”) comprising:
a light source configured to output light (Tesdahl, ¶ [0050]: “An illumination source 720 is configured to provide non-visible illumination to the eye 115, for example, using infrared (IR) or near-IR wavelengths”, see FIG. 7 with annotations below);
a pattern mask configured to change a path of light transmitted through a pattern of the pattern mask (Tesdahl, ¶ [0050]: “Reflected light corresponding to eye features (e.g., glints) impinge on the optical mask which controls transmittance of light in a spatially-coded pattern based on the mask configuration”, ¶ [0056]: “the optical mask can include diffractive optical elements (DOEs) 805 and refractive optical elements (ROEs) 810.” The Examiner notes in optics, refractive lenses bend light by changing its speed as it passes through different materials, while diffractive lenses bend light by causing it to interfere constructively or destructively, creating patterns of light and dark; see FIG. 7 with annotations below);
an image sensor configured to receive light that is output from the light source, reflected by an eye, and transmitted through the pattern mask (Tesdahl, ¶ [0050]: “an adjacent inward-facing sensor 715 such as a CMOS (complementary metal oxide semiconductor) image sensor, CCD (charge-coupled device) image sensor, or other suitable passive- or active-pixel image sensor”; see FIG. 7 with annotations below); and
at least one processor (Tesdahl, ¶ [0101]: “one or more processors having a logic subsystem and a data storage subsystem in communication with the sensors”; see FIG. 7 with annotations below) configured to:
obtain a coded image that is phase-modulated (Tesdahl, ¶ [0004 and 0057]: “the optical mask is described by a point spread function (PSF) and may be implemented using diffractive optical elements such as coded apertures, amplitude masks, phase masks, diffusers, …”, ¶ [0084]: “diffractive optical elements (DOEs)…Each DOE is an optical element comprising a periodic structure that can modulate various properties of light in a periodic pattern such as the direction of optical axis, optical path length, and the like.” Tesdahl suggests the reflected light may be phase-modulated by a phase mask) based on light transmitted through the pattern mask (Tesdahl, ¶ [0050-0053]: “The reflected eye features are captured by the inward-facing sensor as a coded eye feature map”; see FIG. 7 with
PNG
media_image1.png
490
1113
media_image1.png
Greyscale
annotations below),
obtain a feature point of the eye from the coded image (Tesdahl, ¶ [0050-0055]: “eye features”; ¶ [0060-0061]: “The machine learning system extracts eye features from the reflected eye features as encoded by the optical mask into the coded eye feature map”), and
obtain gaze information of a user based on the feature point (Tesdahl, ¶ [0061]: “uses the extracted features to provide an output including estimated gaze directions”, ¶ [0043-0044 and 0052]).
Tesdahl does not explicitly disclose selectively perform a gaze tracking operation or perform an user authentication operation.
Henrik is in the same field of art of eye tracking device. Further, Henrik teaches selectively perform a gaze tracking operation or perform an iris authentication operation. (Henrik, ¶ [0016]: “For example, logging into the headset device may be a trigger that causes iris authentication. Once authenticated, the headset may switch to the eye-tracking mode. In some examples, the eye-tracking mode is the default mode.” Henrik teaches a head mounted device selects authentication mode when a user is logging, and selects eye-tracking mode after the user logged in successfully)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Tesdahl by incorporating eye-tracking mode and authentication mode that is taught by Henrik, to make a system that can perform both eye tracking and authentication in an efficient manner; thus, one of ordinary skilled in the art would be motivated to combine the references since among its several aspects, the present invention recognizes there is a need to improve efficiency and user’s experience (Henrik, ¶ [0021]: “the process performs more efficiently than other conventional systems. In some examples, use of the system described herein may also provide for improved user experience as the user is unlikely to notice distinctions between when iris identification is being performed and eye tracking”).
The combination of Tesdahl and Henrik does not explicitly disclose reconstruct an original image of the eye from the coded image, and obtain authentication information of the user if the original image is reconstructed.
Lee is in the same field of art of user authentication system using iris data. Further, Lee teaches reconstruct an original image of the eye (Lee, ¶ [0041, 0061, 0086 and 0101], see iris scanning, iris sensor and iris image) from the coded image (Lee, ¶ [0078]: “the processor may produce and restore images of different areas of the object, each corresponding to a respective one of the sensors, from the image (or image information) detected by the sensor layer”, ¶ [0062]: “a processor … may produce second image information from information regarding the coded pattern including an array of openings and the first image information detected by the sensors S1, S2, S3, . . . , S8, and the processor may perform user authentication based on the first image information or the second image information”), and obtain authentication information of the user if the original image is reconstructed. (Lee, ¶ [0078]: “may produce image information necessary for user authentication or the actual image (or an image close thereto) of the object from the detected image (e.g., the first image information) based on the restored pattern corresponding to the coded pattern …”, ¶ [0101]: “the processor may only obtain and restore image information about some portion(s) of the detected image and then compare the stored user information with the restored image information, performing user authentication”) (Lee teaches a system which obtains coded/encrypted biometric data using a filter layer laid on sensors, the system can restore the original biometric data for authentication process)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Tesdahl and Henrik by incorporating the encrypting and decrypting iris data method that is taught by Lee, to make an authentication system that protect user’s data using encrypting/decrypting; thus, one of ordinary skilled in the art would be motivated to combine the references since among its several aspects, the present invention recognizes there is a need to securing user’s biometric information (Lee, ¶ [0142]: “the coded pattern of the filter layer and the restored pattern stored in the electronic device enable securing sufficient information required for, at least, user authentication, while permitting the superposition of object information-containing light beams”).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
CLAIM 2
Regarding claim 2, the combination of Tesdahl, Henrik and Lee teaches the device of claim 1. In addition, the combination of Tesdahl, Henrik and Lee teaches the feature point comprises information about at least one of position coordinates of the eye (Tesdahl, ¶ [0005]: “the eye features may include pupil ellipse coordinates, pupil center, illumination glint locations, and the like.”) and a shape of the eye (Tesdahl, ¶ [0006 and 0053-0054]: “the neural network extracts depth estimates for the eye features… depth estimation is performed on single (i.e., monocular) eye feature images” The Examiner notes eye’s depth reads on shape of an eye).
CLAIM 3
Regarding claim 3, the combination of Tesdahl, Henrik and Lee teaches the device of claim 1. In addition, the combination of Tesdahl, Henrik and Lee teaches the feature point comprises at least one of a pupil feature point and a glint feature point. (Tesdahl, ¶ [0005]: “the eye features may include pupil ellipse coordinates, pupil center, illumination glint locations, and the like.”, ¶ [0052]: “Changes in reflected eye features, such as glints from the user's eyeballs and/or a location of a user's pupil, as determined from coded eye features gathered using the inward-facing sensor, may be used to estimate a direction of gaze with some degree of probability.”)
CLAIM 5
Regarding claim 5, the combination of Tesdahl, Henrik and Lee teaches the device of claim 1. In addition, the combination of Tesdahl, Henrik and Lee teaches identify iris information of the user (Lee, ¶ [0041, 0061, 0086 and 0101], see iris scanning, iris sensor and iris image) based on the original image (Lee, ¶ [0078]: “the processor may produce and restore images of different areas of the object, each corresponding to a respective one of the sensors, from the image (or image information) detected by the sensor layer”, ¶ [0062]: “a processor … may produce second image information from information regarding the coded pattern including an array of openings and the first image information detected by the sensors S1, S2, S3, . . . , S8, and the processor may perform user authentication based on the first image information or the second image information”), and obtain the authentication information based on the identified iris information. (Lee, ¶ [0078]: “may produce image information necessary for user authentication or the actual image (or an image close thereto) of the object from the detected image (e.g., the first image information) based on the restored pattern corresponding to the coded pattern …”, ¶ [0101]: “the processor may only obtain and restore image information about some portion(s) of the detected image and then compare the stored user information with the restored image information, performing user authentication”)
CLAIM 6
Regarding claim 6, the combination of Tesdahl, Henrik and Lee teaches the device of claim 1. In addition, the combination of Tesdahl, Henrik and Lee teaches obtaining process determination information based on a situation of the user (Henrik, ¶ [0016]: “the headset is configured to operate in an iris recognition mode and an eye-tracking mode. Predefined triggers may cause the headset to switch modes. Such triggers may include user inputs, computer application events, and other similar triggers. For example, logging into the headset device may be a trigger that causes iris authentication”. Henrik teaches predefined trigger events that could affect how the device operate, a user’s login is an example), and based on the process determination information, selectively obtaining the gaze information of the user or the authentication information of the user. (Henrik, ¶ [0016]: “For example, logging into the headset device may be a trigger that causes iris authentication. Once authenticated, the headset may switch to the eye-tracking mode. In some examples, the eye-tracking mode is the default mode.” Henrik teaches the device selects authentication mode when a user is logging, and selects eye-tracking mode after the user logged in successfully)
CLAIM 10
Regarding claim 10, the combination of Tesdahl, Henrik and Lee teaches the device of claim 1. In addition, the combination of Tesdahl, Henrik and Lee teaches an artificial intelligence (AI) model configured to extract a feature point (Tesdahl, ¶ [0005]: “A neural network, such as a convolutional neural network (CNN), trained with sets of coded eye features, extracts eye features for the HMD device user directly from the coded eye feature maps”, [0058-0060]) from an image that is phase-modulated by the pattern of the pattern mask (Tesdahl, ¶ [0004 and 0057]: “the optical mask is described by a point spread function (PSF) and may be implemented using diffractive optical elements such as coded apertures, amplitude masks, phase masks, diffusers, …”, ¶ [0084]: “diffractive optical elements (DOEs)…Each DOE is an optical element comprising a periodic structure that can modulate various properties of light in a periodic pattern such as the direction of optical axis, optical path length, and the like.” Tesdahl suggests the reflected light may be phase-modulated by a phase mask), and
wherein the at least one processor is further configured to obtain the feature point from the coded image, using the Al model. (Tesdahl, ¶ [0105-0106]: “a computing system that may be configured for eye and/or hand tracking in accordance with the present principles…Computing system includes a logic processor”)
CLAIM 11
Regarding Claim 11, Tesdahl teaches a method (Tesdahl, Abstract: “Eye and hand tracking systems in head-mounted display (HMD) devices”) comprising:
obtaining a coded image (Tesdahl, ¶ [0050-0053]: “The reflected eye features are captured by the inward-facing sensor as a coded eye feature map”; see FIG. 7 with annotations below) that is phase-modulated (Tesdahl, ¶ [0004 and 0057]: “the optical mask is described by a point spread function (PSF) and may be implemented using diffractive optical elements such as coded apertures, amplitude masks, phase masks, diffusers, …”, ¶ [0084]: “diffractive optical elements (DOEs)…Each DOE is an optical element comprising a periodic structure that can modulate various properties of light in a periodic pattern such as the direction of optical axis, optical path length, and the like.” Tesdahl suggests the reflected light may be phase-modulated by a phase mask), based on light reflected by an eye and transmitted through a pattern mask (Tesdahl, ¶ [0050]: “Reflected light corresponding to eye features (e.g., glints) impinge on the optical mask which controls transmittance of light in a spatially-coded pattern based on the mask configuration”, ¶ [0056]: “the optical mask can include diffractive optical elements (DOEs) 805 and refractive optical elements (ROEs) 810.” The Examiner notes in optics, refractive lenses bend light by changing its speed as it passes through different materials, while diffractive lenses bend light by causing it to interfere constructively or destructively, creating patterns of light and dark; see FIG. 7 with annotations below) and received through an image sensor (Tesdahl, ¶ [0050]: “an adjacent inward-facing sensor 715 such as a CMOS (complementary metal oxide semiconductor) image sensor, CCD (charge-coupled device) image sensor, or other suitable passive- or active-pixel image sensor”; see FIG. 7 with annotations below);
PNG
media_image1.png
490
1113
media_image1.png
Greyscale
obtain a feature point of the eye from the coded image (Tesdahl, ¶ [0050-0055]: “eye features”; ¶ [0060-0061]: “The machine learning system extracts eye features from the reflected eye features as encoded by the optical mask into the coded eye feature map”), and
obtain gaze information of a user based on the feature point (Tesdahl, ¶ [0061]: “uses the extracted features to provide an output including estimated gaze directions”, ¶ [0043-0044 and 0052]).
Tesdahl does not explicitly disclose selectively perform a gaze tracking operation or perform an user authentication operation.
Henrik is in the same field of art of eye tracking device. Further, Henrik teaches selectively perform a gaze tracking operation or perform an iris authentication operation. (Henrik, ¶ [0016]: “For example, logging into the headset device may be a trigger that causes iris authentication. Once authenticated, the headset may switch to the eye-tracking mode. In some examples, the eye-tracking mode is the default mode.” Henrik teaches a head mounted device selects authentication mode when a user is logging, and selects eye-tracking mode after the user logged in successfully)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Tesdahl by incorporating eye-tracking mode and authentication mode that is taught by Henrik, to make a system that can perform both eye tracking and authentication in an efficient manner; thus, one of ordinary skilled in the art would be motivated to combine the references since among its several aspects, the present invention recognizes there is a need to improve efficiency and user’s experience (Henrik, ¶ [0021]: “the process performs more efficiently than other conventional systems. In some examples, use of the system described herein may also provide for improved user experience as the user is unlikely to notice distinctions between when iris identification is being performed and eye tracking”).
The combination of Tesdahl and Henrik does not explicitly disclose reconstruct an original image of the eye from the coded image, and obtain authentication information of the user if the original image is reconstructed.
Lee is in the same field of art of user authentication system using iris data. Further, Lee teaches reconstruct an original image of the eye (Lee, ¶ [0041, 0061, 0086 and 0101], see iris scanning, iris sensor and iris image) from the coded image (Lee, ¶ [0078]: “the processor may produce and restore images of different areas of the object, each corresponding to a respective one of the sensors, from the image (or image information) detected by the sensor layer”, ¶ [0062]: “a processor … may produce second image information from information regarding the coded pattern including an array of openings and the first image information detected by the sensors S1, S2, S3, . . . , S8, and the processor may perform user authentication based on the first image information or the second image information”), and obtain authentication information of the user if the original image is reconstructed. (Lee, ¶ [0078]: “may produce image information necessary for user authentication or the actual image (or an image close thereto) of the object from the detected image (e.g., the first image information) based on the restored pattern corresponding to the coded pattern …”, ¶ [0101]: “the processor may only obtain and restore image information about some portion(s) of the detected image and then compare the stored user information with the restored image information, performing user authentication”) (Lee teaches a system which obtains coded/encrypted biometric data using a filter layer laid on sensors, the system can restore the original biometric data for authentication process)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Tesdahl and Henrik by incorporating the encrypting and decrypting iris data method that is taught by Lee, to make an authentication system that protect user’s data using encrypting/decrypting; thus, one of ordinary skilled in the art would be motivated to combine the references since among its several aspects, the present invention recognizes there is a need to securing user’s biometric information (Lee, ¶ [0142]: “the coded pattern of the filter layer and the restored pattern stored in the electronic device enable securing sufficient information required for, at least, user authentication, while permitting the superposition of object information-containing light beams”).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
CLAIM 12
Regarding claim 12, the combination of Tesdahl, Henrik and Lee teaches the method of claim 11. In addition, the combination of Tesdahl, Henrik and Lee teaches the feature point comprises information about at least one of position coordinates of the eye (Tesdahl, ¶ [0005]: “the eye features may include pupil ellipse coordinates, pupil center, illumination glint locations, and the like.”) and a shape of the eye (Tesdahl, ¶ [0006 and 0053-0054]: “the neural network extracts depth estimates for the eye features… depth estimation is performed on single (i.e., monocular) eye feature images” The Examiner notes eye’s depth reads on shape of an eye).
CLAIM 13
Regarding claim 13, the combination of Tesdahl, Henrik and Lee teaches the method of claim 11. In addition, the combination of Tesdahl, Henrik and Lee teaches the feature point comprises at least one of a pupil feature point and a glint feature point. (Tesdahl, ¶ [0005]: “the eye features may include pupil ellipse coordinates, pupil center, illumination glint locations, and the like.”, ¶ [0052]: “Changes in reflected eye features, such as glints from the user's eyeballs and/or a location of a user's pupil, as determined from coded eye features gathered using the inward-facing sensor, may be used to estimate a direction of gaze with some degree of probability.”)
CLAIM 15
Regarding claim 15, the combination of Tesdahl, Henrik and Lee teaches the method of claim 11. In addition, the combination of Tesdahl, Henrik and Lee teaches identify iris information of the user (Lee, ¶ [0041, 0061, 0086 and 0101], see iris scanning, iris sensor and iris image) based on the original image (Lee, ¶ [0078]: “the processor may produce and restore images of different areas of the object, each corresponding to a respective one of the sensors, from the image (or image information) detected by the sensor layer”, ¶ [0062]: “a processor … may produce second image information from information regarding the coded pattern including an array of openings and the first image information detected by the sensors S1, S2, S3, . . . , S8, and the processor may perform user authentication based on the first image information or the second image information”), and obtain the authentication information based on the identified iris information. (Lee, ¶ [0078]: “may produce image information necessary for user authentication or the actual image (or an image close thereto) of the object from the detected image (e.g., the first image information) based on the restored pattern corresponding to the coded pattern …”, ¶ [0101]: “the processor may only obtain and restore image information about some portion(s) of the detected image and then compare the stored user information with the restored image information, performing user authentication”)
CLAIM 16
Regarding claim 16, the combination of Tesdahl, Henrik and Lee teaches the method of claim 11. In addition, the combination of Tesdahl, Henrik and Lee teaches obtaining process determination information based on a situation of the user (Henrik, ¶ [0016]: “the headset is configured to operate in an iris recognition mode and an eye-tracking mode. Predefined triggers may cause the headset to switch modes. Such triggers may include user inputs, computer application events, and other similar triggers. For example, logging into the headset device may be a trigger that causes iris authentication”. Henrik teaches predefined trigger events that could affect how the device operate, a user’s login is an example), and based on the process determination information, selectively obtaining the gaze information of the user or the authentication information of the user. (Henrik, ¶ [0016]: “For example, logging into the headset device may be a trigger that causes iris authentication. Once authenticated, the headset may switch to the eye-tracking mode. In some examples, the eye-tracking mode is the default mode.” Henrik teaches the device selects authentication mode when a user is logging, and selects eye-tracking mode after the user logged in successfully)
CLAIM 20
Regarding Claim 20, the combination of Tesdahl, Henrik and Lee teaches a non-transitory computer-readable recording medium having recorded thereon a program (Tesdahl, ¶ [0109]: “Non-volatile storage device 2206 includes one or more physical devices configured to hold instructions executable by the logic processors to implement the methods and processes described herein”) for implementing the method according to claim 11. (see the rejection of Claim 11)
CLAIM 21
Regarding Claim 21, the combination of Tesdahl, Henrik and Lee teaches the device of claim 1. In addition, the combination of Tesdahl, Henrik and Lee teaches the at least one processor comprises a first processor and a second processor. (Tesdahl, ¶ [0101]: “The HMD device can further include a controller such as one or more processors”)
CLAIM 22
Regarding Claim 22, the combination of Tesdahl, Henrik and Lee teaches the device of claim 21. In addition, the combination of Tesdahl, Henrik and Lee teaches the first processor is configured to obtain the gaze information of the user based on the feature point (Tesdahl, ¶ [0101 and 0106-0109]: “Non-volatile storage device includes one or more physical devices configured to hold instructions executable by the logic processors to implement the methods and processes described herein”, Tesdahl teaches a processor configured to execute an eye tracking program), and wherein the second processor is configured to obtain the authentication information of the user based on the reconstructed image. (Lee, ¶ [0100-0101]: “For example, when user authentication is carried out with the image detected by the sensor layer 641, the processor may produce the second image information by obtaining and restoring, e.g., image information about some preset area(s) of the object” Lee teaches a processor configured to execute authentication program)
Claim(s) 7-9 and 17-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tesdahl in view of Henrik in view of Lee, and further in view of Nieser et al. (WO-2016142489-A1, hereinafter Nieser).
CLAIM 7
Regarding claim 7, the combination of Tesdahl, Henrik and Lee teaches the device of claim 1.
The combination of Tesdahl, Henrik and Lee does not explicitly disclose the image sensor comprises a time of flight (TOF) sensor configured to obtain depth information based on received light.
Nieser is in the same field of art of eye tracking device. Further, Nieser teaches the image sensor comprises a time of flight (TOF) sensor (Nieser, page 3-4, last paragraph of page 3: “So the 3D capturing device can comprise e.g. a stereo camera, a laser scanner, a time-of-flight camera, a light coding system, a sonar/ultrasound sensor or any other depth sensor.”) configured to obtain depth information based on received light. (Nieser, Page 6, second paragraph: “The captured depth data, which can be seen as a depth image, and the 2D image do not necessarily have to be separate image but can also form one single image, e.g. like an image of the eye, wherein each pixel of the image contains a depth value and a color or brightness value. Such images can be captured e.g. by a time of flight camera or a stereo camera system”)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Tesdahl, Henrik and Lee by incorporating time of flight camera that is taught by Nieser, to make an eye-tracking device that use multiple image sensor to obtain eye’s features; thus, one of ordinary skilled in the art would be motivated to combine the references since among its several aspects, the present invention recognizes there is a need to improve the accuracy of the task identifying eye features (Nieser, Page 6, second paragraph: “Thereby a much higher accuracy can be achieved in identifying certain features or regions of the eye when using the 2D brightness or color image in combination with the captured depth data.”).
The combination of Tesdahl, Henrik, Lee and Nieser then teaches the at least one processor (Tesdahl, ¶ [0105-0106]: “a computing system that may be configured for eye and/or hand tracking in accordance with the present principles…Computing system includes a logic processor”) is further configured to obtain the coded image , based on light transmitted through the pattern mask (Tesdahl, ¶ [0050-0053]: “The reflected eye features are captured by the inward-facing sensor as a coded eye feature map”) and received through the TOF sensor. (Nieser, Page 6, second paragraph: “The captured depth data, which can be seen as a depth image, and the 2D image do not necessarily have to be separate image but can also form one single image, e.g. like an image of the eye, wherein each pixel of the image contains a depth value and a color or brightness value. Such images can be captured e.g. by a time of flight camera or a stereo camera system”)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
CLAIM 8
Regarding Claim 8, the combination of Tesdahl, Henrik, Lee and Nieser teaches the device of Claim 7. In addition, the combination of Tesdahl, Henrik, Lee and Nieser teaches the coded image obtained based on the light received through the TOF sensor comprises depth information corresponding to the eye (Nieser, Page 6, second paragraph: “the iris or pupil can first be identified in the 2D image of the eye. Then the corresponding depth information, which corresponds to the 2D coordinates of the identified iris and pupil, can be derived from the depth data…Such images can be captured e.g. by a time of flight camera”), and wherein the coded image comprises an image modulated by the pattern mask. (Tesdahl, ¶ [0004 and 0057]: “the optical mask is described by a point spread function (PSF) and may be implemented using diffractive optical elements such as coded apertures, amplitude masks, phase masks, diffusers, …”, ¶ [0084]: “diffractive optical elements (DOEs)…Each DOE is an optical element comprising a periodic structure that can modulate various properties of light in a periodic pattern such as the direction of optical axis, optical path length, and the like.” Tesdahl suggests the reflected light may be phase-modulated by a phase mask)
CLAIM 9
Regarding Claim 9, the combination Tesdahl, Henrik, and Lee teaches the device of claim 1.
The combination Tesdahl, Henrik, and Lee does not explicitly disclose the image sensor comprises a time of flight (TOF) sensor configured to obtain depth information.
Nieser is in the same field of art of eye tracking device. Further, Nieser teaches the image sensor comprises a time of flight (TOF) sensor (Nieser, page 3-4, last paragraph of page 3: “So the 3D capturing device can comprise e.g. a stereo camera, a laser scanner, a time-of-flight camera, a light coding system, a sonar/ultrasound sensor or any other depth sensor.”) configured to obtain depth information. (Nieser, Page 6, second paragraph: “The captured depth data, which can be seen as a depth image, and the 2D image do not necessarily have to be separate image but can also form one single image, e.g. like an image of the eye, wherein each pixel of the image contains a depth value and a color or brightness value. Such images can be captured e.g. by a time of flight camera or a stereo camera system”)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Tesdahl, Henrik, and Lee by incorporating time of flight camera that is taught by Nieser, to make an eye-tracking device that use multiple image sensor to obtain eye’s features; thus, one of ordinary skilled in the art would be motivated to combine the references since among its several aspects, the present invention recognizes there is a need to improve the accuracy of the task identifying eye features. (Nieser, Page 6, second paragraph: “Thereby a much higher accuracy can be achieved in identifying certain features or regions of the eye when using the 2D brightness or color image in combination with the captured depth data.”)
The combination of Tesdahl, Henrik, Lee and Nieser then teaches a light sensor configured to obtain an image of a target object (Tesdahl, ¶ [0050-0053]: “The reflected eye features are captured by the inward-facing sensor as a coded eye feature map”), and
wherein the at least one processor (Tesdahl, ¶ [0105-0106]: “a computing system that may be configured for eye and/or hand tracking in accordance with the present principles…Computing system includes a logic processor”) is further configured to:
obtain a first coded image based on light transmitted through the pattern mask (Tesdahl, ¶ [0050-0053]: “The reflected eye features are captured by the inward-facing sensor as a coded eye feature map”) and received through the TOF sensor (Nieser, Page 6, second paragraph: “an image of the eye, wherein each pixel of the image contains a depth value and a color or brightness value. Such images can be captured e.g. by a time of flight camera”),
obtain a second coded image, based on light transmitted through the pattern mask and received by the light sensor (Tesdahl, ¶ [0050-0053]: “The reflected eye features are captured by the inward-facing sensor as a coded eye feature map”), and
obtain the feature point, based on at least one of the first coded image and the second coded image. (“Thereby a much higher accuracy can be achieved in identifying certain features or regions of the eye when using the 2D brightness or color image in combination with the captured depth data. The captured depth data, which can be seen as a depth image, and the 2D image do not necessarily have to be separate image but can also form one single image, e.g. like an image of the eye, wherein each pixel of the image contains a depth value and a color or brightness value. Such images can be captured e.g. by a time of flight camera or a stereo camera system. Such images also can be captured separately, e.g. the depth data can be captured by a laser scanner or a sonar/ultrasound sensor, and the 2D image can be captured by a normal camera or image sensor.”)
CLAIM 17
Regarding Claim 17, the combination of Tesdahl, Henrik and Lee teaches the method of claim 11.
The combination of Tesdahl, Henrik and Lee does not explicitly disclose the image sensor comprises a time of flight (TOF) sensor configured to obtain depth information based on received light.
Nieser is in the same field of art of eye tracking device. Further, Nieser teaches the image sensor comprises a time of flight (TOF) sensor (Nieser, page 3-4, last paragraph of page 3: “So the 3D capturing device can comprise e.g. a stereo camera, a laser scanner, a time-of-flight camera, a light coding system, a sonar/ultrasound sensor or any other depth sensor.”) configured to obtain depth information based on received light. (Nieser, Page 6, second paragraph: “The captured depth data, which can be seen as a depth image, and the 2D image do not necessarily have to be separate image but can also form one single image, e.g. like an image of the eye, wherein each pixel of the image contains a depth value and a color or brightness value. Such images can be captured e.g. by a time of flight camera or a stereo camera system”)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Tesdahl, Henrik and Lee by incorporating time of flight camera that is taught by Nieser, to make an eye-tracking device that use multiple image sensor to obtain eye’s features; thus, one of ordinary skilled in the art would be motivated to combine the references since among its several aspects, the present invention recognizes there is a need to improve the accuracy of the task identifying eye features (Nieser, Page 6, second paragraph: “Thereby a much higher accuracy can be achieved in identifying certain features or regions of the eye when using the 2D brightness or color image in combination with the captured depth data.”).
The combination of Tesdahl, Henrik, Lee and Nieser then teaches obtaining of the coded image comprises obtaining the coded image, based on light transmitted through the pattern mask (Tesdahl, ¶ [0050-0053]: “The reflected eye features are captured by the inward-facing sensor as a coded eye feature map”) and received through the TOF sensor. (Nieser, Page 6, second paragraph: “The captured depth data, which can be seen as a depth image, and the 2D image do not necessarily have to be separate image but can also form one single image, e.g. like an image of the eye, wherein each pixel of the image contains a depth value and a color or brightness value. Such images can be captured e.g. by a time of flight camera or a stereo camera system”)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
CLAIM 18
Regarding Claim 18, the combination of Tesdahl, Henrik, Lee and Nieser teaches the method of Claim 17. In addition, the combination of Tesdahl, Henrik, Lee and Nieser teaches the coded image obtained based on the light received through the TOF sensor comprises depth information corresponding to the eye (Nieser, Page 6, second paragraph: “the iris or pupil can first be identified in the 2D image of the eye. Then the corresponding depth information, which corresponds to the 2D coordinates of the identified iris and pupil, can be derived from the depth data…Such images can be captured e.g. by a time of flight camera”), and wherein the coded image comprises an image modulated by the pattern mask. (Tesdahl, ¶ [0004 and 0057]: “the optical mask is described by a point spread function (PSF) and may be implemented using diffractive optical elements such as coded apertures, amplitude masks, phase masks, diffusers, …”, ¶ [0084]: “diffractive optical elements (DOEs)…Each DOE is an optical element comprising a periodic structure that can modulate various properties of light in a periodic pattern such as the direction of optical axis, optical path length, and the like.” Tesdahl suggests the reflected light may be phase-modulated by a phase mask)
CLAIM 19
Regarding Claim 19, the combination Tesdahl, Henrik, and Lee teaches the method of claim 11.
The combination Tesdahl, Henrik, and Lee does not explicitly disclose the image sensor comprises a time of flight (TOF) sensor configured to obtain depth information.
Nieser is in the same field of art of eye tracking device. Further, Nieser teaches the image sensor comprises a time of flight (TOF) sensor (Nieser, page 3-4, last paragraph of page 3: “So the 3D capturing device can comprise e.g. a stereo camera, a laser scanner, a time-of-flight camera, a light coding system, a sonar/ultrasound sensor or any other depth sensor.”) configured to obtain depth information. (Nieser, Page 6, second paragraph: “The captured depth data, which can be seen as a depth image, and the 2D image do not necessarily have to be separate image but can also form one single image, e.g. like an image of the eye, wherein each pixel of the image contains a depth value and a color or brightness value. Such images can be captured e.g. by a time of flight camera or a stereo camera system”)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Tesdahl, Henrik, and Lee by incorporating time of flight camera that is taught by Nieser, to make an eye-tracking device that use multiple image sensor to obtain eye’s features; thus, one of ordinary skilled in the art would be motivated to combine the references since among its several aspects, the present invention recognizes there is a need to improve the accuracy of the task identifying eye features. (Nieser, Page 6, second paragraph: “Thereby a much higher accuracy can be achieved in identifying certain features or regions of the eye when using the 2D brightness or color image in combination with the captured depth data.”)
The combination Tesdahl, Henrik, Lee and Nieser then teaches a light sensor configured to obtain an image of a target object (Tesdahl, ¶ [0050-0053]: “The reflected eye features are captured by the inward-facing sensor as a coded eye feature map”), and
wherein the at least one processor (Tesdahl, ¶ [0105-0106]: “a computing system that may be configured for eye and/or hand tracking in accordance with the present principles…Computing system includes a logic processor”) is further configured to:
obtain a first coded image based on light transmitted through the pattern mask (Tesdahl, ¶ [0050-0053]: “The reflected eye features are captured by the inward-facing sensor as a coded eye feature map”) and received through the TOF sensor (Nieser, Page 6, second paragraph: “an image of the eye, wherein each pixel of the image contains a depth value and a color or brightness value. Such images can be captured e.g. by a time of flight camera”),
obtain a second coded image, based on light transmitted through the pattern mask and received by the light sensor (Tesdahl, ¶ [0050-0053]: “The reflected eye features are captured by the inward-facing sensor as a coded eye feature map”), and
obtain the feature point, based on at least one of the first coded image and the second coded image. (“Thereby a much higher accuracy can be achieved in identifying certain features or regions of the eye when using the 2D brightness or color image in combination with the captured depth data. The captured depth data, which can be seen as a depth image, and the 2D image do not necessarily have to be separate image but can also form one single image, e.g. like an image of the eye, wherein each pixel of the image contains a depth value and a color or brightness value. Such images can be captured e.g. by a time of flight camera or a stereo camera system. Such images also can be captured separately, e.g. the depth data can be captured by a laser scanner or a sonar/ultrasound sensor, and the 2D image can be captured by a normal camera or image sensor.”)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NHUT HUY (JEREMY) PHAM whose telephone number is (703)756-5797. The examiner can normally be reached Mo - Fr. 8:30am - 6pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O'Neal Mistry can be reached on (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
NHUT HUY (JEREMY) PHAMExaminerArt Unit 2674
/Ross Varndell/Primary Examiner, Art Unit 2674