Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s amendment filed on 12/1/2025:
Amends claims 1, 8, and 14.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Agaoglu et al (US 20230418372 A1) in view of Ben-ami et al (US 20220354363 A1).
Regarding claim 1, Agaoglu et al teaches a computer apparatus comprising: at least one eye tracking camera (Para 40, for example, when wearing a head-mounted device (HMD), one sensor (e.g., a camera inside the HMD) may acquire the pupillary data for eye tracking, and one sensor on a separate device (e.g., one camera, such as a wide range view) may be able to capture all of the facial feature data of the user. i.e. eye tracking camera); and at least one processor in data communication with a memory storing processor executable code (Para 12, electronic device having a processor and one or more sensors, obtaining eye data associated with a gaze during a first period of time, obtaining head data associated with the gaze during the first period of time, and determining, based on the eye data and the head data, a first gaze behavior state during the first period of time to identify gaze shifting events, gaze holding events, or loss events i.e. processor);
and wherein the processor executable code configures the at least one processor to: receive an image stream from the at least one eye tracking camera (Para 4, various implementations disclosed herein include devices, systems, and methods that provides a real-time gaze classification algorithm to classify eye movement and gaze behavior types based on eye tracking data (e.g., gaze direction, gaze angle, pupil diameter, pupil location, etc. obtained via video-based eye tracker. i.e. video is an image stream from camera);
include the eye tracking data into a training set or eye tracking data (Para 53 and Fig 3, the process flow for example environment 300 is acquiring physiological data over a period of time at the eye movement and gaze behavior instruction set 340 and determining gaze behavior event labels 360, in real-time, for a user's eye movements based on the physiological data. The gaze behavior event labels 360 may include different classifications of gaze behavior events, such as Class-1 371 (e.g., a fast behavior state—saccades), Class-2 372 (e.g., a stabilizing behavior state—smooth pursuit, VOR, or fixation events), or Class-3 373 (e.g., a loss behavior state—blink, wink, or other data losses). The gaze behavior event labels 360 can then be sent to one or more applications 370 that can quickly utilize that information as an interaction event according to techniques described herein (e.g., predicting whether the user intends an interaction with a portion of displayed content). i.e. the gaze behavior over a period of eye movement time is determined and then labeled by the system 360 - this therefore is training data set/tracking data that has a label. This gaze behavior that is labeled then ca be sent in further applications 370. Further applications can include predicting user eye movement/interaction with content);
Agaoglu et al does not teach, produce a physiology-based bio-kinematic model of eye movement and pupil dynamics defining physical limitations of eye movement as determined by anatomical properties of bone, muscles, and tendons, via a machine learning algorithm trained on the training set. However, Agaoglu et al teaches producing a bio-kinematic model via a machine learning algorithm trained on the training set (See Para 5-7 and 70).
In a similar field of endeavor, Ben-ami et al teaches, wherein the bio-kinematic model includes a physiology-based bio-kinematic model of eye movement and pupil dynamics defining physical limitations of eye movement as determined by anatomical properties of bone, muscles, and tendons (Para 41, in some embodiments, the OPS 114 may further process 193 the iris data 153 to extract iris related parameters such as iris rotation 158, iris translation 160 and iris radius 162. In some embodiments, the above iris related parameters are obtained using an ML technique. For example, the OPS 114 may implement a maximum likelihood estimation (MLE) based curve fitting method to estimate the iris related parameters. The iris data is provided as input to the MLE-based curve fitting method (e.g., oval fit—based on an assumption that iris is circular and so when projected onto a 2D plane it is oval/ellipse), which generates the above iris related parameters. Furthermore, see Para 42, the score for each iteration may reflect a log-likelihood of the examined parameter set (e.g., coordinates of the iris shape, radius, and center), based on the known physical constraints and assumptions. In some embodiments, the physical constraints and assumptions may improve the efficiency of the MLE process by introducing better estimated initial conditions before iterating towards convergence. […] in some embodiments, examples of physical constraints may include: pupil dilation dependence in overall brightness (light conditions), limited range of face orientation as users look at the display, blinking may initiate uncertainty due to readjustment time, or physical possible range of motion (both face and eyeball). i.e. produce a physiology-based bio-kinematic model of eye movement (can be considered estimating iris related parameters using machine learning), parameter set is based on known physical constraints and assumptions (pupil dynamics defining physical limitations of eye movement), the physical constraints include physical range of motion (both face and eyeball) (anatomical properties of bone, muscles, and tendons)).
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date to incorporate the teachings of Agaoglu et al (US 20230418372 A1) in view of Ben-ami et al (US 20220354363 A1) so that the bio-kinematic mode is physiology-based and includes eye movement and pupil dynamics defining physical limitations of eye movement as determined by anatomical properties of bone, muscles, and tendons. Doing so would allow the physical constraints and assumptions to be improved, as well as the efficiency of the MLE process by introducing better estimated initial conditions before iterating towards convergence (Para 42, Ben-ami et al).
Regarding claim 2, Agaoglu et al teaches the computer apparatus of claim 1, wherein the processor executable code further configures the at least one processor to characterize a user's eye movement to identify gaze and scan pattern with respect to the bio-kinematic model (Para 5 and Para 6, in some implementations, the techniques described herein can classify gaze behavior states in real-time. […] improving the accuracy for determining a gaze behavior states to identify gaze shifting events, gaze holding events, and loss events based on eye tracking and head pose data in real-time and determining sample-wise event levels for eye movement types. Eye movements occur all the time, even when a user thinks his or her gaze is fixed on an object, there are miniature eye movements occurring. i.e. predict and identify scan pattern of user/subject in real time).
Regarding claim 3, Agaoglu et al teaches the computer apparatus of Claim 1, wherein: the processor executable code further configures the at least one processor to receive a training scenario or task; and producing the bio-kinematic model is further based on the training scenario or task (Fig 3 and Para 55, the object representation data 332 may be utilized by the eye movement and gaze behavior instruction set 340 to refine the eye behavior event classifications based on (e.g., learning from) object specific actions. For example, an object displayed in the scene, such as a dog, may be identified, and the dog's location, speed, direction of motion, etc. may be tracked such that if the user gazes towards the dog (e.g., an interruption) the classification analysis techniques described herein may utilize the information to further refine the eye behavior event classification. i.e. training scenario is gaze behavior instruction set 340, which refines the classifications (training labels) performed on the video stream of user data that produces the bio-kinematic model 360 to be used in further applications 370).
Regarding claim 4, Agaoglu et al teaches the computer apparatus of Claim 3, wherein the training scenario or task includes predicted eye movement (Fig 3 and Para 55, object representation data 332 may be utilized by the eye movement and gaze behavior instruction set 340 to refine the eye behavior event classifications based on (e.g., learning from) object specific actions. For example, an object displayed in the scene, such as a dog, may be identified, and the dog's location, speed, direction of motion, etc. may be tracked such that if the user gazes towards the dog (e.g., an interruption) the classification analysis techniques described herein may utilize the information to further refine the eye behavior event classification. i.e. the training scenario includes predicted eye movement, or the user's gaze/looking/movement following the dog).
Regarding claim 5, Agaoglu et al teaches the computer apparatus of Claim 3, wherein the processor executable code further configures the at least one processor to annotate the training set according to the training scenario or task (Fig 3 and Para 54, the physiological data that the eye movement and gaze behavior instruction set 340 utilizes to determine the gaze behavior event labels 360 in real-time for a user's eye movements is based on eye tracking data 310 and head tracking data 320. The eye tracking data 310 may include video-based data, pupil and glint-based data, retinal imaging-based data, scleral coil-based data, EOG-based data, and the like, or any other eye movement data discussed herein. The eye tracking data may then be obtained and analyzed by the eye movement and gaze behavior instruction set 340 or another eye tracking analysis algorithm to determine left eye representation 312, a right eye representation 314, and binocular stats 316. i.e. annotate (label) the training set according to the gaze behavior instruction (340) which can be considered the training "scenario or task.").
Regarding claim 6, Agaoglu et al teaches the computer apparatus of Claim 1, wherein the bio-kinematic model defines user specific physical limitations (Para 9, physiological data, such as EEG amplitude/frequency, sensor data corresponding to pupil modulation, sensor data corresponding to eye gaze saccades, etc., can depend on the individual, characteristics of the scene in front of him or her (e.g., video content), and attributes of the physical environment surrounding the user including the activity/movement of the user. Physiological data can be obtained while using a device with eye tracking technology (and other physiologic sensors) while users perform tasks. In some implementations, physiological data can be obtained using other sensors, such as electroencephalography (EEG) sensors or electrodermal activity (EDA) sensors. Observing repeated measures of physiological data to an experience can give insights about the intent of the user based on his or her eye movement and gaze behavior. i.e. bio-kinematic model defines specific user physiological limitations after repeated measures, including physical responses that are not under direct user control. See Para 44 and 45 regarding more physiological limitations)).
Regarding claim 7, Agaoglu et al does not teach, the computer apparatus of Claim 1, wherein the processor executable code further configures the at least one processor as a machine learning neural network embodying the bio-kinematic model. Agaoglu et al does not specifically mention a neural network. Agaoglu et al does, however, teach a machine learning model embodying a bio-kinematic model.
In a similar field of endeavor, Ben-Ami et al teaches, a neural network (Fig 2 and Para 75-76).
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date to incorporate the teachings of Agaoglu et al (US 20230418372 A1) in view of Ben-ami et al (US 20220354363 A1) so that the processor executable code further configures the at least one processor as a machine learning neural network embodying the bio-kinematic model. Doing so would allow the physical constraints and assumptions to be improved, as well as the efficiency of the MLE process by introducing better estimated initial conditions before iterating towards convergence (Para 42, Ben-ami et al).
Regarding claims 8-13, claims 8-13 rejected for the same reasons as claims 1-6 in the combination above, respectively.
Regarding claims 14-19, claims 14-19 rejected for the same reasons as claims 1-6 in the combination above, respectively.
Regarding claim 20, claim 20 rejected for the same reasons as claim 7 in the combination above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 20220245812 A1
US 20220192485 A1
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACK PETER KRAYNAK whose telephone number is (703)756-1713. The examiner can normally be reached Monday - Friday 7:30 AM - 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACK PETER KRAYNAK/Examiner, Art Unit 2668
/UTPAL D SHAH/Primary Examiner, Art Unit 2668
Prosecution Insights