DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the response to this Office Action, the Examiner respectfully requests that support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line numbers in the specification and/or drawing figure(s). This will assist the Examiner in prosecuting this application.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112 (b):
(b) CONCLUSION. —The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 20-23 are rejected under 35 U.S.C. 112 (b), as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim 20 recites “The method according to claim 18… the extracted pupil position, iris eccentricity, and glint position”.
The claim elements above lack antecedent basis and are not defined in the claim.
Claims 21 and 22 recite “The method according to claim 18… the pupil, the iris eccentricity, and the glint”.
The claim elements above lack antecedent basis and are not defined in the claims.
Claim 23 recites “The method according to claim 18… the eyeball, the pupil, and the iris”.
The claim elements above lack antecedent basis and are not defined in the claim.
As best understood by the Examiner, the claim language was meant to be "The method according to claim 19", in order to adhere to proper antecedent basis for the afore-mentioned limitations in claims 20-23.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2021/0173474 A1 to Sztuk et al. (hereinafter "Sztuk") in view of U.S. Patent Application Publication 2022/0083134 A1 to Kassner et al. (hereinafter "Kassner").
Regarding Claim 18, Sztuk teaches a method comprising: obtaining eye image from a camera; image processing the eye image to obtain positions of eye parts (Figs. 3-5, 11; Para. 56-75, 132-145 of Sztuk; eye tracking unit 215 estimates the foveal axis by directly detecting a location of the fovea 362 and/or of other features of the eye's retina… eye tracking data is obtained for both eyes of a user, such as a user wearing head-mountable display device 102, using at least one camera mounted to a housing of the device… eye tracking data may be processed to identify one or more features of the user's eyes, to identify a current gaze location for each eye); estimating eyeball model parameters based on the positions of the eye parts (Figs. 3-5, 11; Para. 56-75, 132-145 of Sztuk; eye tracking unit 215 estimates the foveal axis by directly detecting a location of the fovea 362 and/or of other features of the eye's retina… eye tracking data may be processed to identify one or more features of the user's eyes, to identify a current gaze location for each eye… a first predicted gaze location and a first gaze location confidence level may be determined (e.g., by saccade filter 806) based on the eye tracking data and using a saccade model (e.g., by attempting to fit the features of the eye tracking data to a model of the same features of an eye movement undergoing a saccade, or by providing the eye tracking data as input to a machine-learning saccade model that has been trained to output a predicted gaze location); computing a 3D gaze direction based on the eyeball model parameters (Figs. 3-5, 11; Para. 56-75, 132-145 of Sztuk; eye tracking data may be processed to identify one or more features of the user's eyes, to identify a current gaze location for each eye (e.g., a location corresponding to the location at which the user's foveal axis intersects display panel 118), identify an eye movement, identify a change in a gaze location, and/or to determine a current direction and/or speed of motion of the gaze location and/or foveal axis for each eye); creating a 3D eyeball model from the eyeball model parameters (Figs. 3-5, 11; Para. 56-75, 132-145 of Sztuk; predicted vergence plane and a vergence plane confidence level may be determined (e.g., by vergence filter 812) based on the eye tracking data and assuming that the current eye movement of the user is a vergence movement (e.g., by attempting to fit the features of the eye tracking data to a model of the same features of an eye movement undergoing a vergence movement… eye tracking data, the predicted gaze location, the predicted vergence plane, the actual gaze location, and/or the actual vergence plane resulting form that eye movement can be fed back to the models (e.g., as additional training data) for further fine-tuning of the models for each user).
Sztuk does not explicitly disclose estimating next eyeball model parameters; and feeding back the estimated next eyeball model parameters to the image processing.
However, Kassner teaches estimating next eyeball model parameters; and feeding back the estimated next eyeball model parameters to image processing (Claims 46-47; Figs. 5-6; Para. 107-114, 409-449 of Kassner; core data corpus can thus dynamically grow in volume, and the universal network can at intervals be refined respectively re-trained to provide an even more accurate universal neural network. This can be done from scratch using the increased training data volume, but with all model parameters re-initialized, or a re-training of the universal NN can use all of or part of the previous version of the parameters as initialization… image or images recorded by the cameras facing the eye(s) of the user are used to predict the user's gaze direction (gaze point) in block 7251. The offset of the predicted gaze direction (gaze point) gpr and the expected (ground truth) gaze direction (gaze point) ge defined by the object or marker position can then be calculated and used to generate a correction mapping or function Fcorr in a block 7254 to be applied henceforth (block 7252) to the prediction of the universal NN to arrive at a calibrated gaze-value gcpr).
Therefore, at the time when the invention was filed, it would have been obvious to a person of ordinary skill in the art to include estimating next eyeball model parameters; and feeding back the estimated next eyeball model parameters to the image processing using the teachings of Kassner in order to modify the method taught by Sztuk. The motivation to combine these analogous arts would have been to further improve generating data suitable for calibrating a head-wearable device such as a head-wearable spectacles device that may be used to detect one or more gaze-related parameters of a user, and an eye tracking system including a head-wearable device (Para. 1, 7 of Kassner).
Potentially Allowable Subject Matter
Claim 19 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
None of the references, either singularly or in combination, teach or fairly suggest the method according to claim 18, wherein: the step of image processing the eye image includes obtaining positions of a pupil, iris eccentricity, and a glint from the eye image; the step of estimating the eyeball model parameters includes estimating positions and orientation radiuses of an eyeball, a pupil, and an iris based on the positions of the pupil, the iris eccentricity, and the glint; the step of estimating the next eyeball model parameters includes estimating next positions of the pupil, the iris eccentricity, and the glint using the 3D eyeball model; and the step of feeding back includes feeding back the estimated next positions of the pupil, the iris eccentricity, and the glint to the image processing.
Claims 20-23 would be allowable if rewritten to overcome applicable 112 (b) rejections indicated above. As allowable subject matter has been indicated, applicant's reply must either comply with all formal requirements or specifically traverse each requirement not complied with. See 37 CFR 1.111(b) and MPEP § 707.07(a).
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Response to Arguments
Applicant's arguments filed 12/23/2025 with respect to claim 18 have been fully considered but they are not persuasive.
Examiner respectfully disagrees with applicant representative’s arguments that “However, while Sztuk describes using 3D eye-related data, it does not disclose creating a 3D eyeball model. When using 3D eye data, such data may simply be stored or sampled and therefore does not necessarily need to be generated as a model. Neither of the cited references describes generating a 3D eyeball model. Further, Sztuk does not disclose any feedback component as claimed in the present application. Although the Examiner considers that feedback is disclosed in Kassner, the feedback in the present invention is used to generate a next 3D model. Accordingly, even if the feedback concepts of Kassner were applied to the configuration of Sztuk, which does not disclose generation of a 3D eyeball model, the claimed configuration of the present invention would still not be obtained”.
Examiner points out that the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). The claim merely recites teaches creating a 3D eyeball model from the eyeball model parameters without providing any structural or constituent details about the afore-mentioned 3D model, which is taught by Sztuk (Figs. 3-5, 11; Para. 56-75, 132-145 of Sztuk; predicted vergence plane and a vergence plane confidence level may be determined (e.g., by vergence filter 812) based on the eye tracking data and assuming that the current eye movement of the user is a vergence movement (e.g., by attempting to fit the features of the eye tracking data to a model of the same features of an eye movement undergoing a vergence movement… eye tracking data, the predicted gaze location, the predicted vergence plane, the actual gaze location, and/or the actual vergence plane resulting form that eye movement can be fed back to the models (e.g., as additional training data) for further fine-tuning of the models for each user). Since Sztuk clearly and explicitly teaches using eye tracking data from the eye tracking units 215 to track at least a subset of the 3D position, roll, pitch, and yaw of each eye 350 and use eye tracking data including or based on these quantities to estimate the gaze location 317, the vergence plane, and/or a 3D gaze point of each eye (Para. 72); and that a first predicted gaze location and a first gaze location confidence level may be determined (e.g., by saccade filter 806) based on the eye tracking data and using a saccade model (e.g., by attempting to fit the features of the eye tracking data to a model of the same features of an eye movement undergoing a saccade, or by providing the eye tracking data as input to a machine-learning saccade model that has been trained to output a predicted gaze location (Para. 135), a 3D eyeball model is created from the eyeball model parameters.
Kassner further teaches estimating next eyeball model parameters; and feeding back the estimated next eyeball model parameters to image processing (Claims 46-47; Figs. 5-6; Para. 107-114, 409-449 of Kassner; core data corpus can thus dynamically grow in volume, and the universal network can at intervals be refined respectively re-trained to provide an even more accurate universal neural network. This can be done from scratch using the increased training data volume, but with all model parameters re-initialized, or a re-training of the universal NN can use all of or part of the previous version of the parameters as initialization… image or images recorded by the cameras facing the eye(s) of the user are used to predict the user's gaze direction (gaze point) in block 7251. The offset of the predicted gaze direction (gaze point) gpr and the expected (ground truth) gaze direction (gaze point) ge defined by the object or marker position can then be calculated and used to generate a correction mapping or function Fcorr in a block 7254 to be applied henceforth (block 7252) to the prediction of the universal NN to arrive at a calibrated gaze-value gcpr).
In addition, according to MPEP §2144, it is proper to take into account not only specific teachings of the reference but also the inferences which one skilled in the art would reasonably be expected to draw therefrom. In re Preda, 401 F.2d 825, 826, 159 USPQ 342, 344 (CCPA 1968).
Therefore, the combination of Sztuk and Kassner teaches all the limitations of claim 18.
Examiner, thus maintains the rejections for independent claim 18.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABHISHEK SARMA whose telephone number is (571)272-9887. The examiner can normally be reached on Mon - Fri 8:00-5:00.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amr Awad can be reached on 571-272-7764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABHISHEK SARMA/
Primary Examiner, Art Unit 2621
Prosecution Insights