Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Preliminary Amendment
1. Applicant's preliminary amendment, filed November 04, 2025, is respectfully acknowledged and has been fully considered.
Claims 1-20 are cancelled. Claims 21-40 are newly added.
Claims 21-40 remain pending.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 21, 25-26, 32-34, and 36-37 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Iseringhausen et al. (U.S. Patent 20190101978 A1, hereinafter “Iseringhausen”).
Regarding Claim 21 (New), Iseringhausen teaches a system (paras 0016,0019 Figs 1,2 system 100), comprising:
one or more cameras configured to capture images of an eye (par 0033 Fig 2 camera 230 captures images of the dense structured light pattern 245 reflected from portions of the eye 220 in the target area (e.g., cornea, iris, and/or sclera));
a controller comprising one or more processors (par 0033 Fig 2 controller 235, par 0047 comprising one or more processors) configured to:
calculate eye poses for the eye based on eye features in the captured eye images (par 0047 Fig 3 camera assembly 320 captures images of the [eye and] dense structured light pattern reflected from the eye as it looks at [multiple] known locations and at multiple known orientations);
update a current eye model based on cornea and pupil features determined based on the calculated eye poses (par 0047 Fig 3 captured images of the eye [features] at multiple known orientations, overlaid with the dense structured light pattern, can be interpolated into [calculated and integrated into] a model; par 0041 Fig 3 model, M stored in the data store 340 can be a 3D model of a portion of the eye including e.g., a cornea, an iris, a pupil, a sclera, an anterior chamber, etc.; par 0054 Fig 5 eye features are calculated); and
perform one or more additional iterations of said capturing eye images, said calculating the eye poses, and said updating the current eye model until the current eye model reaches a threshold level of performance (par 0074 Fig 7 the system may receive information (comprising a number of lost locators 735) that tracking of the HMD 705 is lost (e.g., imaging device 710 loses line of sight of at least a threshold number of locators 735), the tracking module 750 re-calibrates some or all of the system 700 components).
Regarding Claim 25 (New), Iseringhausen teaches the system of claim 21, further comprising
one or more light sources (par 0003 The structured light emitter may generate the structured light pattern (e.g., in an infrared band) using, e.g., one or more light sources), wherein
said calculating the eye pose of the eye is based on reflections of light emitted from the one or more light sources included in the captured eye images (par 0033 Fig 2 camera 230 captures images of the dense structured light pattern 245 reflected from portions of the eye 220 in the target area (e.g., cornea, iris, and/or sclera); par 0047 Fig 3 captured images of the eye [features] at multiple known orientations, overlaid with the dense structured light pattern, are interpolated into [calculated and integrated into] a model; par 0041 Fig 3 model, M stored in the data store 340 can be a 3D model of a portion of the eye including e.g., a cornea, an iris, a pupil, a sclera, an anterior chamber, etc.; par 0054 Fig 5 eye features are calculated).
Regarding Claim 26 (New), Iseringhausen teaches a method, comprising:
performing an eye enrollment process (par 0002 modeling at least one eye of a user, par 0047 through a calibration process) while a user is using a head mounted device (HMD) (par 0047 while wearing an HMD), wherein
performing the eye enrollment process comprises:
capturing eye images of an eye of the user while the user is using the HMD (par 0033 Fig 2 camera 230 captures images of the dense structured light pattern 245 reflected from portions of the eye 220 in the target area (e.g., cornea, iris, and/or sclera); par 0047 while wearing an HMD);
calculating eye poses for the eye of the user based on eye features in the captured eye images (par 0047 Fig 3 camera assembly 320 captures images of the [eye and] dense structured light pattern reflected from the eye as it looks at [multiple] known locations and at multiple known orientations);
updating a current eye model based on cornea and pupil features determined based on the calculated eye poses (par 0047 Fig 3 captured images of the eye [features] at multiple known orientations, overlaid with the dense structured light pattern, can be interpolated into [calculated and integrated into] a model; par 0041 Fig 3 model, M stored in the data store 340 can be a 3D model of a portion of the eye including e.g., a cornea, an iris, a pupil, a sclera, an anterior chamber, etc.; par 0054 Fig 5 eye features are calculated); and
performing one or more additional iterations of said capturing eye images, said calculating the eye poses, and said updating the current eye model until the current eye model reaches a threshold level of performance (par 0074 Fig 7 the system may receive information (comprising a number of lost locators 735) that tracking of the HMD 705 is lost (e.g., imaging device 710 loses line of sight of at least a threshold number of locators 735), the tracking module 750 re-calibrates some or all of the system 700 components).
Regarding Claim 32 (New), Iseringhausen teaches the method of claim 26, wherein
said calculating the eye poses for the eye comprises calculating positions and orientations of the eye (par 0047 Fig 3 camera assembly 320 captures images of the [eye and] dense structured light pattern reflected from the eye as it looks at [multiple] known locations and at multiple known orientations).
Claim 33 presents the limitations of Claim 25 in a different claim category, and therefore Claim 33 is rejected with a rationale similar to Claim 25, mutatis mutandis.
Regarding Claim 34 (New), Iseringhausen teaches the method of claim 26, wherein
the threshold level of performance is an error rate of a use process (par 0074 Fig 7 the system may receive information (comprising a number of lost locators 735) that tracking of the HMD 705 is lost (e.g., imaging device 710 loses line of sight of at least a threshold number of locators 735), the tracking module 750 re-calibrates some or all of the system 700 components).
Regarding Claim 36 (New), Iseringhausen teaches the method of claim 26, further comprising
using the current eye model in a gaze tracking process (par 0047 Fig 6 step 630 once the calibration module 370 has produced M eye tracking may begin; par 0058 eye tracking system 300 estimates 630 a position and gaze angle of the eye based on the one or more images and a model of the eye).
Regarding Claim 37 (New), Iseringhausen teaches a controller, comprising one or more processors (par 0033 Fig 2 controller 235, par 0047 comprising one or more processors), configured to:
receive eye images of an eye from one or more cameras (par 0033 Fig 2 camera 230 captures images of the dense structured light pattern 245 reflected from portions of the eye 220 in the target area (e.g., cornea, iris, and/or sclera); par 0047 while wearing an HMD);
calculate eye poses for the eye based on eye features in the received eye images (par 0047 Fig 3 camera assembly 320 captures images of the [eye and] dense structured light pattern reflected from the eye as it looks at [multiple] known locations and at multiple known orientations);
update a current eye model based on cornea and pupil features determined based on the calculated eye poses (par 0047 Fig 3 captured images of the eye [features] at multiple known orientations, overlaid with the dense structured light pattern, can be interpolated into [calculated and integrated into] a model; par 0041 Fig 3 model, M stored in the data store 340 can be a 3D model of a portion of the eye including e.g., a cornea, an iris, a pupil, a sclera, an anterior chamber, etc.; par 0054 Fig 5 eye features are calculated); and
perform one or more additional iterations of said capturing eye images, said calculating the eye poses, and said updating the current eye model until the current eye model reaches a threshold level of performance (par 0074 Fig 7 the system may receive information (comprising a number of lost locators 735) that tracking of the HMD 705 is lost (e.g., imaging device 710 loses line of sight of at least a threshold number of locators 735), the tracking module 750 re-calibrates some or all of the system 700 components).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 22 and 38 are rejected under 35 U.S.C. 103 as being unpatentable over Iseringhausen in view of Klingström (U.S. Patent Application 20200257359 A1).
Regarding Claim 22 (New), Iseringhausen teaches the system of claim 21, further comprising
a display (par 0019 Fig 2 display 205).
However, Iseringhausen appears not to expressly teach wherein the controller is further configured to:
cause the display to show a target at one or more target positions while the one or more cameras capture the images of the eye; and
perform said calculating the eye poses based at least in part on the one or more target positions.
Klingström teaches wherein
cause the display to show a target at one or more target positions while the one or more cameras capture the images of the eye (par 0034 Fig 3 processing circuitry 304 may also be communicatively connected to the display 303, for example for controlling (or triggering) the display 303 to show test stimulus [target] points 305 for calibration of the eye tracking system 300; par 0074 Fig 6 processing circuitry 304 of the eye tracking system 300 controls the display 303 to show the reference stimulus [target] 305 at a known position of the display 303 (for example as a clearly visible dot or X). The processing circuitry 304 instructs the user (for example via text at the display 303 or via an audio message) to look at the reference stimulus [target] 305. The camera(s) 302 then captures an image of the eye 100 when the eye 100 is looking at the reference stimulus 305 (or when the eye 100 is assumed/believed to be looking at the reference stimulus 305)); and
perform said calculating the eye poses based at least in part on the one or more target positions (par 0074 the processing circuitry 304 then calculates estimates 604 of values for the plurality of parameters [paras 0075-0078 including eye position and orientation information] based on at least the obtained image and the known position of the reference stimulus [target] 305. The calibration 401 typically includes use of multiple reference stimulus [target] points, which the user is prompted to look at one after the other).
Iseringhausen and Klingström are analogous art as they each pertain to systems employing eye tracking. It would have been obvious to a person of ordinary skill in the art to modify the system of Iseringhausen with the inclusion of the prompt display target positions of Klingström. The motivation would have been in order to allow performing eye tracking for the eye 100 using the values estimated at the calibration 401 (Klingström par 0082).
Claim 38 presents the limitations of Claim 22 in a different claim category, and therefore Claim 38 is rejected with a rationale similar to Claim 22, mutatis mutandis.
Claims 24, 30, and 40 are rejected under 35 U.S.C. 103 as being unpatentable over Iseringhausen in view of Stent (U.S. Patent Application 20200089315 A1).
Regarding Claim 24 (New), Iseringhausen teaches the system of claim 21, wherein
to perform said updating the current eye model, the controller is further configured to:
construct a graph system using the calculated eye poses; and
apply an optimizer to the cornea and pupil features of the current model in view of the graph system (par 0053 Fig 1 the computing device 130 collects and compiles a new [model] training dataset using images from the eye tracker 125, which is, par 0025, is an array of light detectors positioned on or adjacent to the display 120 at different orientations, and from images collected from the plurality of cameras 141-148 from respective viewpoints 241-248, par 0036; par 0053 Fig 1 the [model] update is carried out by error backpropagation using one of a suite of gradient-based optimization methods such as, but not limited to, stochastic gradient descent or Adam).
Iseringhausen and Stent are analogous art as they each pertain to systems employing eye tracking. It would have been obvious to a person of ordinary skill in the art to modify the system of Iseringhausen with the inclusion of the optimization process of Stent. The motivation would have been in order to provide more robust training data for gaze estimation models (Stent par 0001).
Claim 30 presents the limitations of Claim 24 in a different claim category, and therefore Claim 30 is rejected with a rationale similar to Claim 24, mutatis mutandis.
Claim 40 presents the limitations of Claim 24 in a different claim category, and therefore Claim 40 is rejected with a rationale similar to Claim 24, mutatis mutandis.
Claim 27 is rejected under 35 U.S.C. 103 as being unpatentable over Iseringhausen in view of Hoanca et al. (U.S. Patent 7986816 B1, hereinafter “Hoanca”).
Regarding Claim 27 (New), Iseringhausen teaches the method of claim 26, wherein
the captured eye images comprise eye images of the user’s eyes in two or more different orientations (par 0047 Fig 3 camera assembly 320 captures images of the [eye and] dense structured light pattern reflected from the eye as it looks at [multiple] known locations and at multiple known orientations).
However, Iseringhausen appears not to expressly teach wherein
the captured eye images comprise eye images of the user’s eyes at two or more different levels of brightness.
Hoanca teaches wherein
the captured eye images comprise eye images of the user’s eyes at two or more different levels of brightness (col 14 lines 34-41 sample images to collect data for eye tracking include a moving object that the user must track (the computer can record the response time and other statistics of the eye movement), an object that appears in a random area on the screen (the computer records the response time to focus on the object), and an image with a rapidly changing brightness (the computer records speed and magnitude of change in pupil size), wherein modulating the brightness is performed to stimulate changes in pupil diameter (col 14 lines 18-21 Appropriately constructed images that change in brightness and provide motion for the subject to track can be used to evoke responses in the eye relating to both velocity and pupil diameter).
Iseringhausen and Hoanca are analogous art as they each pertain to systems employing eye tracking. It would have been obvious to a person of ordinary skill in the art to modify the method/system of Iseringhausen with the inclusion of the modulated display brightness of Hoanca. The motivation would have been in order to provide the ability to obtain eye feature data at various pupil dilations for more robust eye and gaze tracking models.
Claim 28 is rejected under 35 U.S.C. 103 as being unpatentable over Iseringhausen in view of Hoanca et al. (U.S. Patent 7986816 B1, hereinafter “Hoanca”) and further in view of Fogelström et al. (U.S. Patent Application 20210350565 A1, hereinafter “Fogelström”).
Regarding Claim 28 (New), Iseringhausen as modified teaches the method of claim 27, wherein
an initial generic eye model is used as a starting point for the current eye model (par 0041-0042 suggest an initial “ideal” eye model comprising average eye radii, average schlera radii, etc. from which deviations reflecting real eye parameters are accounted for with additional measured parameters).
However, Iseringhausen as modified appears not to expressly teach wherein
ground truth information about target gaze positions corresponding to the two or more different orientations of the user’s eye is used in calculating the eye poses.
Fogelström teaches wherein
ground truth information about target gaze positions corresponding to the two or more different orientations of the user’s eye is used in calculating the eye poses (par 0044 Fig 5 eye position information obtained by the PCCR based eye tracker 610 is used as ground truth to calibrate the non-PCCR based eye tracker 630 in order to get more accurate gaze estimations using the non-PCCR based eye tracker 630; par 0054 the tracked movement of the head pose of the subject 310 is used to calculate the gaze of the subject 310; par 0056 the gaze is calculated using the PCCR based eye tracker 610).
Iseringhausen Hoanca and Fogelström are analogous art as they each pertain to methods employing eye tracking. It would have been obvious to a person of ordinary skill in the art to modify the method of Iseringhausen/Hoanca with the inclusion of the gaze positions ground truth information of Fogelström. The motivation would have been in order to provide gaze tracking using the non-PCCR based eye tracker 630 when the PCCR based eye tracker 610 is unable to track movement of the at least one first eye position 104 (Fogelström par 0063).
Claim 31 is rejected under 35 U.S.C. 103 as being unpatentable over Iseringhausen in view of Rougeaux (U.S. Patent Application 20160363995 A1).
Regarding Claim 31 (New), Iseringhausen teaches the method of claim 26. However, Iseringhausen appears not to expressly teach wherein
said updating the current eye model comprises calculating an optimal eyeball center.
Rougeaux teaches wherein
said updating the current eye model comprises calculating an optimal eyeball center (par 0067-0069 Fig 7 the optimal eyeball center location is calculated using several frames of captured images where the user gazes at different locations within the camera's field of view, and the result is fed forward for use in updating the eye model).
Iseringhausen and Rougeaux are analogous art as they each pertain to systems employing eye tracking. It would have been obvious to a person of ordinary skill in the art to modify the method of Iseringhausen with the inclusion of the optimal eyeball center calculation of Rougeaux. The motivation would have been in order to provide an eye-gaze estimation technique which relies on the measurement of the eyeball center of rotation (Rougeaux par 0066).
Claim 35 is rejected under 35 U.S.C. 103 as being unpatentable over Iseringhausen in view of Zschau (U.S. Patent Application 20100303294 A1).
Regarding Claim 35 (New), Iseringhausen teaches the method of claim 26. However, Iseringhausen appears not to expressly teach wherein
the threshold level of performance is based on an evaluation of the current eye model.
Zschau teaches wherein
the threshold level of performance is based on an evaluation of the current eye model (par 0047 Fig 1 the segmentation threshold, the algorithms to compute the threshold, or the parameters of the algorithms to compute the threshold are found and optimized [to a desired, threshold level of performance] by measuring the detection performance in a large number of test patterns and test pattern sequences).
Iseringhausen and Zschau are analogous art as they each pertain to systems employing eye tracking. It would have been obvious to a person of ordinary skill in the art to modify the method of Iseringhausen with the inclusion of the performance level threshold of Zschau. The motivation would have been in order to provide a method that allows to find and track reliably, precisely and efficiently the eye positions of one or multiple faces in all three spatial directions in a sufficiently large detection and tracking range in real time, while only causing a low computational load (Zschau par 0011).
Claims 23 and 39 are rejected under 35 U.S.C. 103 as being unpatentable over Iseringhausen in view of Drozdov et al. (U.S. Patent Application 20220206571 A1, hereinafter “Drozdov”).
Regarding Claim 23 (New), Iseringhausen teaches the system of claim 21. However, Iseringhausen appears not to expressly teach wherein
said calculating the eye poses does not include using ground truth information about target gaze positions.
Drozdov teaches wherein
said calculating the eye poses does not include using ground truth information about target gaze positions (par 0027 Fig 6 teach a process that determines in real-time, for each user (when multiple users are present) using any camera or eye-tracking method, a PoR prediction, and a mapping/calculation of the user's gaze region-of-interest (ROI). It does not require any special devices, nor active user interaction. Given the screen layout, the SAP is trained in an unsupervised manner (no ground-truth labels are required during the process), to select the optimal regional boundary on the screen that calculates/maps the head-eye combination generating the raw gaze-prediction that optimizes the accuracy and precision of PoR mapping on the digital display).
Iseringhausen and Drozdov are analogous art as they each pertain to methods employing eye tracking. It would have been obvious to a person of ordinary skill in the art to modify the method of Iseringhausen with the inclusion of the non-use of ground truth gaze information of Drozdov. The motivation would have been in order to provide methods that perform a weighted selection of refined screen grid boundaries that best represent the user's PoR on the screen based on the head-eye information and the provided original screen layout (Drozdov par 0028).
Claim 39 presents the limitations of Claim 23 in a different claim category, and therefore Claim 39 is rejected with a rationale similar to Claim 23, mutatis mutandis.
Claim 29 is rejected under 35 U.S.C. 103 as being unpatentable over Iseringhausen in view of Hoanca et al. (U.S. Patent 7986816 B1, hereinafter “Hoanca”) and further in view of Drozdov et al. (U.S. Patent Application 20220206571 A1, hereinafter “Drozdov”).
Regarding Claim 29 (New), Iseringhausen as modified teaches the method of claim 27. However, Iseringhausen as modified appears not to expressly teach wherein
said calculating the eye poses does not include using ground truth information about target gaze positions.
Drozdov teaches wherein
said calculating the eye poses does not include using ground truth information about target gaze positions (par 0027 Fig 6 teach a process that determines in real-time, for each user (when multiple users are present) using any camera or eye-tracking method, a PoR prediction, and a mapping/calculation of the user's gaze region-of-interest (ROI). It does not require any special devices, nor active user interaction. Given the screen layout, the SAP is trained in an unsupervised manner (no ground-truth labels are required during the process), to select the optimal regional boundary on the screen that calculates/maps the head-eye combination generating the raw gaze-prediction that optimizes the accuracy and precision of PoR mapping on the digital display).
Iseringhausen Hoanca and Drozdov are analogous art as they each pertain to methods employing eye tracking. It would have been obvious to a person of ordinary skill in the art to modify the method of Iseringhausen/Hoanca with the inclusion of the non-use of ground truth gaze information of Drozdov. The motivation would have been in order to provide methods that perform a weighted selection of refined screen grid boundaries that best represent the user's PoR on the screen based on the head-eye information and the provided original screen layout (Drozdov par 0028).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARK EDWARDS whose telephone number is 571-270-7731. The examiner can normally be reached on M-F 9a-5p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Eason can be reached on 571-270-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARK EDWARDS/
Primary Examiner, Art Unit 2624
Prosecution Insights