DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 2026 February 17 has been entered. This Office action is responsive to the Reply to Office Action filed 17 February 2026. The Examiner acknowledges the amendments to claims 1, 4, 6, 10 and 12, and the cancellation of claim 5. Claims 1-4, 6, 8-12, 15, and 17-24 are currently pending.
Oath/Declaration
The declaration under 37 CFR 1.132 filed 2026 February 17 is insufficient to overcome the rejection of claim claims 1-6, 8-12, 15 and 17-24 based upon lack of utility under 35 USC 101 as set forth in the last Office action because: the showing is not commensurate in scope with the claims. It refer(s) only to the system described in the above referenced application and not to the individual claims of the application. Thus, there is no showing that the objective evidence of utility under 35 USC 101 is commensurate in scope with the claims. See MPEP § 716. In the declaration, Applicants argue that it is a technical challenge to implement a real-time eye-tracking system that achieves both high accuracy and computational efficiency under unforgiving vehicle environment conditions, and further that a person having ordinary skill in the art would recognize the claimed approach improves the technology itself because it represents a fundamentally different processing architecture that achieves superior performance characteristics compared to conventional approaches because early-stage validation controls which data segments proceed to later processing stages. This is not persuasive as this does not demonstrate how the claims specifically recite the alleged technical improvements to the alleged technical problems related to the claimed invention.
Response to Arguments
Applicant’s arguments filed 2026 February 17 have been fully considered but they are not persuasive. Applicant argues that the rejection of the claims under 35 USC 101 should be withdrawn because the claims do not merely apply a known mental process on a computer and instead provides a specific technical solution, specifically a multi-stage selective processing architecture with validation-based classification modification using physiologically-based criteria that addresses technical problems in eye-tracking systems for driver monitoring. The Examiner respectfully disagrees with these arguments. Claim 1 involves a multi-step classification that first generates preliminary classifications, then modifies those classifications and uses the modified classification to detect driver drowsiness. The detection of driver drowsiness is merely an abstract idea that one could do in their mind (i.e., make a determination that a driver is drowsy). Furthermore, though the method has been amended to add the detection of driver drowsiness, the detection of the driver drowsiness is not output, used, or applied in any way, and outputting the detection of driver drowsiness would merely be insignificant extra-solution activity.
Moreover, though Applicant alleges in the subject matter eligibility declaration that there are technological improvements, the declaration is not persuasive to convince the examiner that the claims set forth technological improvements. As stated above with respect to the declaration, the showing is not commensurate in scope with the claims because it only relates to the system described in the instant application and not to the individual claims of the application. Thus, there is no showing that the objective evidence of utility under 35 USC 101 is commensurate in scope with the claims. See MPEP § 716. The rejection of the claims under 35 USC 101 still stands. See 35 USC 101 section below.
Regarding the rejections under 35 USC 103, Applicant argues that with respect to claim 24, the cited references in combination fail to teach or suggest “deriving a blink velocity from the first signal, wherein the blink velocity is derived from a change of the distance between the eyelids of the eye over time”, and “deriving an eye movement velocity from the second signal, wherein the eye movement velocity is derived from a change of the second signal over time”, and as such fails to adequately address the requirement of deriving a blink velocity, and further the same portions are recited for both deriving steps although different parameters are derived from different signals. The Examiner agrees and as such, the rejection has been adapted using the previously cited references to accordingly address the above-mentioned limitations. See 35 USC 103 rejections below.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 6 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 6 recites “wherein deriving the one or more of a saccade amplitude, a saccade velocity, a saccade duration, and a fixation duration comprises deriving the saccade velocity”. It is unclear how a saccade velocity is used to derive a saccade velocity. The Examiner respectfully requests clarification of this claim limitation. For examination purposes, it will be interpreted that gaze position data is used to derive a saccade velocity and the saccade velocity is used to determine whether gaze position data represents a fixation or a saccade.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-4, 6, 8-12, 15, and 17-24 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) as a whole, considering all claim elements both individually and in combination, do not amount to significantly more than an abstract idea. A streamlined analysis of claim 1 follows.
Regarding claim 1, the claim recites a series of steps or acts, including determining one or more first periods of time in which the first signal exceeds a first threshold, determining one or more second periods of time in which the first signal does not exceed the first threshold, determining an eye openness classification of an eye gaze/eye movement of a person, determining that a blink velocity exceeds a predetermined maximum blink velocity for a segment of the first signal, determining that an image of a series of images indicates that an eye is in a closed state and subsequent and preceding images indicate that the eye is in the open state, detecting driver drowsiness based on a modified eye openness classification, determining eye gaze from only a selected portion of the first signal, determining an eye movement velocity by calculating a derivative of a second signal, and determining whether an eye movement velocity indicates a saccade, a fixation, or is inconclusive as to whether it indicates a saccade or a fixation based on whether the eye movement velocity exceeds a predetermined threshold. Thus, the claim is directed to a process, which is one of the statutory categories of invention.
The claim is then analyzed to determine whether it is directed to any judicial exception. The steps of determining one or more first periods of time in which the first signal exceeds a first threshold, determining one or more second periods of time in which the first signal does not exceed the first threshold, determining an eye openness classification of an eye gaze/eye movement of a person, determining that a blink velocity exceeds a predetermined maximum blink velocity for a segment of the first signal, determining that an image of a series of images indicates that an eye is in a closed state and subsequent and preceding images indicate that the eye is in the open state, detecting driver drowsiness based on a modified eye openness classification, determining eye gaze from only a selected portion of the first signal, determining an eye movement velocity by calculating a derivative of a second signal, and determining whether an eye movement velocity indicates a saccade, a fixation, or is inconclusive as to whether it indicates a saccade or a fixation based on whether the eye movement velocity exceeds a predetermined threshold set forth a judicial exception. These steps describe concepts performed in the human mind (including an observation, evaluation, judgment, opinion). Thus, the claim is drawn to a Mental Process, which is an Abstract Idea.
Next, the claim as a whole is analyzed to determine whether the claim recites additional elements that integrate the judicial exception into a practical application. The claim fails to recite an additional
element or a combination of additional elements to apply, rely on, or use the judicial exception in a
manner that imposes a meaningful limitation on the judicial exception. Therefore, the claim does not
provide any additional element or a combination of additional elements such that the judicial exception
can provide an improvement to the technological field, the method does not effect a particular treatment or effect a particular change, nor does the method use a particular machine to perform the Abstract Idea.
Next, the claim as a whole is analyzed to determine whether any element, or combination of elements, is sufficient to ensure that the claim amounts to significantly more than the exception. Besides the Abstract Idea, the claim recites additional steps of recording a series of images of an eye with a camera, outputting the images to an eye openness analysis unit of a computing device, generating a first signal indicating a distance between eyelids of the eye, deriving a blink velocity from the first signal representative of movement associated with the eye, modifying the classification of the eye openness, and generating a second signal indicative of eye gaze. The recording, outputting, deriving, generating, and modifying steps are each recited at a high level of generality such that it amounts to insignificant presolution activity, e.g., mere data gathering step necessary to perform the Abstract Idea. When recited at this high level of generality, there is no meaningful limitation, such as a particular or unconventional step that distinguishes it from well-understood, routine, and conventional data gathering and comparing activity engaged in by medical professionals prior to Applicant's invention. Furthermore, it is well established that the mere physical or tangible nature of additional elements such as the obtaining and comparing steps do not automatically confer eligibility on a claim directed to an abstract idea (see, e.g., Alice Corp. v. CLS Bank Int'l, 134 S.Ct. 2347, 2358-59 (2014)).
Consideration of the additional elements as a combination also adds no other meaningful limitations to the exception not already present when the elements are considered separately. Unlike the eligible claim in Diehr in which the elements limiting the exception are individually conventional, but taken together act in concert to improve a technical field, the claim here does not provide an improvement to the technical field. Even when viewed as a combination, the additional elements fail to transform the exception into a patent-eligible application of that exception. Thus, the claim as a whole does not amount to significantly more than the exception itself. The claim is therefore drawn to non-statutory subject matter.
The same rationale applies to claim 24.
Regarding claim 10, the device recited in the claim is a generic device comprising generic components configured to perform the abstract idea. The recited camera is a generic camera configured to perform pre-solutional data gathering activity, and the computing device is configured to perform the Abstract Idea. According to section 2106.05(f) of the MPEP, merely using a computer as a tool to perform an abstract idea does not integrate the Abstract Idea into a practical application.
The dependent claims also fail to add something more to the abstract independent claims as they generally recite method steps pertaining to data gathering, data modification, and data analysis. The recording, outputting, deriving, generating, and modifying steps recited in the independent claims maintain a high level of generality even when considered in combination with the dependent claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-2, 4, 6, 10-11, 15, 19-20, and 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application Publication 20180032816 --as previously cited--, hereinafter referenced as "Trapp" in view of US Patent Application Publication 20200265251 --as previously cited--, hereinafter referenced as "Vandommele", in further view of US Patent Application Publication 20160132726 -- as previously cited--, hereinafter referenced as "Kempinski".
With respect to claim 1, Trapp teaches a computer-implemented method for determining a movement associated with an eye of a person, the method comprising:
recording, via a camera, a series of images of the eye (see Trapp, par 0028);
outputting the recorded series of images of the eye to an eye openness analysis unit of a computing device (i.e., outputting the recorded images to a computing device that determines whether the eye is open) (see Trapp, par 0029, wherein the fixation identification device is a computerized device comprising a processor and memory that receives gaze position data elements to identify user gaze fixation regions, thus determining whether the user’s eyes are open),
based on the recorded series of images, generating, by the eye openness analysis unit (i.e., by a processor) a first signal representative of a movement associated with the eye (i.e., gaze position data elements) (see Trapp, par 0029 & 0034);
deriving one or more movement parameters from the first signal (i.e., an angular velocity of the gaze position data) (see Trapp, par 0039-0040);
determining one or more first periods of time in which the first signal exceeds a first threshold (i.e., determining whether an angular velocity meets or exceeds a velocity threshold in order to categorize a gaze point as a saccade, or whether an angular velocity is below a velocity threshold in order to classify a gaze point as a fixation)(see Trapp, par 0005, 0039-0041);
determining one or more second periods of time in which the first signal does not exceed the first threshold (i.e., determining whether an angular velocity meets or exceeds a velocity threshold in order to categorize a gaze point as a saccade, or whether an angular velocity is below a velocity threshold in order to classify a gaze point as a fixation)(see Trapp, par 0005, 0039-0041);
determining a classification of the eye, the classification including the first signal indicating that the eye is in an open state (i.e., in a state of fixation) during the one or more first periods of time, indicating that the eye is in a closed state during the one or more second periods of time (i.e., whether or not the glint created on the surface of the pupil of a user can be identified by the camera) (see Trapp, par 0028, 0034, 0039);
modifying the classification based on a comparison of the one or more movement parameters and a predetermined maximum parameter value (i.e., modifying whether the eye is in a state of a fixation or saccade based upon the predetermined thresholds) (see Trapp, par 0039-0040);
after the classification is modified, selecting a portion of the first signal corresponding to one or more periods when the eye is classified in the open state (see Trapp, par 0038-0040);
and determining eye gaze from only the selected portion of the first signal (see Trapp, par 0038-0040), wherein determining eye gaze comprises generating a second signal indicative of eye gaze (i.e., generating second gaze position data elements subsequent and consecutive to the first gaze position data elements) (see Trapp, par 0038), determining an eye movement velocity by calculating a derivative of the second signal (i.e., by using the position values of the glint of the pupil of a person created by the eye tracking device to derive at the angular velocity of the second gaze position data) (see Trapp, par 0039), and determining whether the eye movement velocity indicates a saccade, indicates a fixation, or is inconclusive as to whether it indicates a saccade or a fixation, based on whether the eye movement velocity exceeds a predetermined threshold (see Trapp, par 0039-0040).
Trapp fails to teach that the method is for drivers of a vehicle, that the first signal indicates a distance between eyelids of the eye of a driver, deriving a blink velocity derived from a change of the distance between the eyelids of the eye over time, determining an eye openness of the driver, determining that an image of the series of images indicates the eye is in the closed state and subsequent and preceding images indicate that the eye is in the open state, modifying the eye openness classification associated with the image to the open state, that for one or more points in time the preliminary eye movement classification is modified as inconclusive if the second movement parameter exceeds a second threshold, thereby generating an eye movement classification, wherein the second threshold is a physiologically known limit of eye movement, and detecting driver drowsiness based on the modified eye openness classification.
Vandommele teaches a method and device for classifying eye opening data of at least one eye of an occupant of a vehicle wherein a position of the eyelids of the eyes of a driver are recorded, and a distance between the eyelids of at least one of the driver’s eyes is determined (see Vandommele, par 0014, 0042, 0046). Further, a speed and/or acceleration of a movement of the eyelids of at least one of the eyes is recorded, and the eyelid speed is derived from the determined eye opening degree (i.e., the distance between the eyelids) (see Vandommele, par 0042, 0046, 0048). Additionally, images are analyzed to determine that an eye is in a transition state when it goes from closing to opening in an opening phase (i.e., the eye is in the closed state and subsequent and preceding images indicate that the eye is in the open state and the status of the eye is modified accordingly) (see Vandommele, par 0014, 0023-0024, 0053, 0065-0069, fig. 4, 6). The aforementioned techniques are used to detect driver drowsiness based on the eye openness classification (see Vandommele, abstract, 0017, 0025-0028, 0042-0043, 0070, 0083-0084, fig. 1-2, 4).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Trapp such that the method is for drivers of a vehicle, that the first signal indicates a distance between eyelids of the eye of a driver, that the one or more movement parameters include a blink velocity derived from a change of the distance between the eyelids of the eye over time, determining an eye openness of the driver, determining that an image of the series of images indicates the eye is in the closed state and subsequent and preceding images indicate that the eye is in the open state, modifying the eye openness classification associated with the image to the open state, that for one or more points in time the preliminary eye movement classification is modified as inconclusive if the second movement parameter exceeds a second threshold, thereby generating an eye movement classification, wherein the second threshold is a physiologically known limit of eye movement, and detecting driver drowsiness based on the modified eye openness classification because the aforementioned techniques permit determining whether or not a driver of a vehicle is drowsy so as to prevent dangerous accidents that arise from a driver falling asleep behind the wheel (see Vandommele, par 0002, 0007, 0010-0013, 0042-0043, 0078-0085, fig. 9).
Trapp as modified by Vandommele fails to teach modifying the openness classification as inconclusive for the segment of the first signal if the blink velocity exceeds the predetermined maximum blink velocity for a segment of the first signal.
Kempinski teaches a system and method for analysis of eye movements using two dimensional images wherein eye movement data that has been classified based on iris to eye corner movement is filtered to remove signals that are unreliable, and determining that the data is indicative of eye movement or of noise is based upon normal eye physiology (see Kempinski, par 0053-0055).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Trapp as modified by Vandommele such that the method comprises modifying the openness classification as inconclusive for the segment of the first signal if the blink velocity exceeds the predetermined maximum blink velocity for a segment of the first signal because that removes unwanted signals that come from noise, or unwanted signals that are not truly indicative of a user’s eye movement (see Kempinski, par 0053-0055).
With respect to claim 2, Trapp as modified by Vandommele and Kempinski teaches the method of claim 1, and Trapp further teaches deriving an amplitude or a frequency of the movement associated with the eye from the first signal (see Trapp, par 0039-0040, 0045-0046).
With respect to claim 4, Trapp as modified by Vandommele and Kempinski teaches the method of claim 1, and Trapp further teaches deriving one or more of: a saccade amplitude; a saccade velocity (i.e., an angular velocity of the gaze position data element that represents a saccade); a saccade duration; and a fixation duration (i.e., a duration value of the fixation gaze position data elements) from the second signal (see Trapp, par 0039 & 0045-0046).
With respect to claim 6, Trapp as modified by Vandommele and Kempinski teaches the method of claim 4, and Trapp further teaches the method further comprises:
deriving the one or more of a saccade amplitude, a saccade velocity (i.e., an angular velocity of the gaze position data element that represents a saccade), a saccade duration, and a fixation duration (i.e., a duration value of the fixation gaze position data elements) comprises deriving the saccade velocity (i.e., gaze position data is used to derive an angular velocity and the angular velocity is used to determine whether gaze position data represents a fixation or a saccade) (see Trapp, par 0039 & 0045-0046).
With respect to claim 10, Trapp teaches a system for determining an eye movement associated with an eye of a person, the system comprising:
a camera 24 (see Trapp, fig. 1) operable to record a series of images of the eye (see Trapp, par 0028);
and a computing device 28 (i.e., a controller) communicatively coupled to the camera and comprising an eye openness analysis unit and an eye gaze analysis unit (i.e., an eye-tracking device coupled to the controller of an eye fixation identification device), the eye openness analysis unit of the computing device operable to execute the steps of (see Trapp, fig. 1):
based on the recorded series of images, generate a first signal of a movement associated with the eye (i.e., gaze position data elements) (see Trapp, par 0029 & 0034);
derive one or more movement parameters from the first signal (i.e., an angular velocity of the gaze position data) (see Trapp, par 0039-0040);
for one or more points in time, determine whether the first signal indicates that the eye is in an open state (i.e., in a state of fixation), indicates that the eye is in a closed state (i.e., whether or not the glint created on the surface of the pupil of a user can be identified by the camera) (see Trapp, par 0028, 0034, 0039), or is inconclusive as to whether the eye is in the open state or the closed state, wherein determining whether the first signal indicates that the eye is in the open state, indicates that the eye is in the closed state, or is inconclusive as to whether the eye is in the open state of the closed state comprises:
generating a first movement parameter of the one or more movement parameters (i.e., an angular velocity of the gaze position data) (see Trapp, par 0039-0040);
generating a preliminary classification of the first signal as indicative of the open state if the first movement parameter exceeds a first threshold, and as indicative of the closed state if the first movement parameter does not exceed the first threshold (i.e., classifying the eye movement as a fixation or a saccade depending upon whether the angular velocity of data associated with each state meets or exceeds the velocity threshold) (see par 0039-0040);
generating a second movement parameter of the one or more movement parameters (i.e., generating second gaze position data elements subsequent and consecutive to the first gaze position data elements) (see Trapp, par 0038);
select a portion of the first signal based on the determination (i.e., identifying certain data elements from the gaze position data as fixation gaze position data elements) (see Trapp, par 0042-0043), wherein the portion of the first signal corresponds to when the eye is in the open state (i.e., identifying certain data elements from the gaze position data as fixation gaze position data elements) (see Trapp, par 0042-0043);
wherein the eye gaze analysis unit of the computing device is operable to execute the steps of:
based on the recorded series of images corresponding to the selected portion of the first signal, generate a second signal indicative of eye gaze (i.e., gaze position data elements) (see Trapp, par 0029 & 0034);
derive one or more third movement parameters from the second signal, wherein the one or more third movement parameters include an eye movement velocity (i.e., an angular velocity) (see Trapp, par 0039-0040 & 0045-0046);
and for one or more points in time, determine whether the second signal indicates a saccade, indicates a fixation, or is inconclusive as to whether it indicates a saccade or a fixation, based on whether at least one of the one or more third movement parameter exceeds a predetermined threshold (see Trapp, fig. 1 & 3, par 0025, 0029, 0033-0048);
and for the one or more points in time in which the eye gaze analysis unit determines that the second signal indicates a fixation, determine an averaged direction of gaze, and identifying an object at which the eye is looking (see Trapp, fig. 3, 0033-0048);
wherein the second signal indicative of eye gaze is generated only for the selected portion of the first signal (i.e., identifying certain data elements from the gaze position data as fixation gaze position data elements) (see Trapp, par 0042-0043).
Trapp fails to teach that the first signal indicates a distance between eyelids of the eye of a driver, that the one or more movement parameters include a blink velocity derived from a change of the distance between the eyelids of the eye over time, and that for one or more points in time the preliminary eye movement classification is modified as inconclusive if the second movement parameter exceeds a second threshold, thereby generating an eye movement classification.
Vandommele teaches a method and device for classifying eye opening data of at least one eye of an occupant of a vehicle wherein a position of the eyelids of the eyes of a driver are recorded, and a distance between the eyelids of at least one of the driver’s eyes is determined (see Vandommele, par 0014, 0042, 0046). Further, a speed and/or acceleration of a movement of the eyelids of at least one of the eyes is recorded, and the eyelid speed is derived from the determined eye opening degree (i.e., the distance between the eyelids) (see Vandommele, par 0042, 0046, 0048). Additionally, images are analyzed to determine that an eye is in a transition state when it goes from closing to opening in an opening phase (i.e., the eye is in the closed state and subsequent and preceding images indicate that the eye is in the open state and the status of the eye is modified accordingly) (see Vandommele, par 0014, 0023-0024, 0053, 0065-0069, fig. 4, 6). The aforementioned techniques are used to detect driver drowsiness based on the eye openness classification (see Vandommele, abstract, 0017, 0025-0028, 0042-0043, 0070, 0083-0084, fig. 1-2, 4).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Trapp such that the first signal indicates a distance between eyelids of the eye of a driver, and such that the one or more movement parameters include a blink velocity derived from a change of the distance between the eyelids of the eye over time because that permits determining whether or not a driver of a vehicle is drowsy so as to prevent dangerous accidents that arise from a driver falling asleep behind the wheel (see Vandommele, par 0002, 0007, 0010-0013, 0042-0043, 0078-0085, fig. 9).
Trapp as modified by Vandommele fails to teach that for one or more points in time the preliminary eye movement classification is modified as inconclusive if the second movement parameter exceeds a second threshold, thereby generating an eye movement classification.
Kempinski teaches a system and method for analysis of eye movements using two dimensional images wherein eye movement data that has been classified based on iris to eye corner movement is filtered to remove signals that are unreliable, and determining that the data is indicative of eye movement or of noise is based upon normal eye physiology (see Kempinski, par 0053-0055).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Trapp as modified by Vandommele such that for one or more points in time the preliminary eye movement classification is modified as inconclusive if the second movement parameter exceeds a second threshold, thereby generating an eye movement classification because that removes unwanted signals that come from noise, or unwanted signals that are not truly indicative of a user’s eye movement (see Kempinski, par 0053-0055).
With respect to claim 11, Trapp as modified by Vandommele and Kempinski teaches the system of claim 10, and Trapp further teaches the one or more movement parameters indicates an amplitude of the movement, a duration of the movement, a velocity of the movement (i.e., an angular velocity), or a frequency of the movement (see Trapp, par 0039-0040, 0045-0046).
With respect to claim 15, Trapp as modified by Vandommele and Kempinski teaches the system of claim 10, and Trapp further teaches one or more third movement parameters is indicative of one or more of a saccade amplitude, a saccade velocity (i.e., an angular velocity of the gaze position data element that represents a saccade), a saccade duration, and a fixation duration (i.e., a duration value of the fixation gaze position data elements) (see Trapp, par 0039 & 0045-0046).
With respect to claim 19, Trapp as modified by Vandommele and Kempinski teaches the method of claim 1, and further teaches that recording the series of images of the eye begins once the driver is seated in the vehicle (see Vandommele, par 0042).
With respect to claim 20, Trapp as modified by Vandommele and Kempinski teaches the method of claim 4, and Trapp further teaches the eye movement velocity is determined by calculating a derivative (i.e., by using the position values of the glint of the pupil of a person created by the eye tracking device to derive at the angular velocity of the second gaze position data) (see Trapp, par 0039).
With respect to claim 22, Trapp as modified by Vandommele and Kempinski teaches the method of claim 5, and Trapp further teaches that analyzing the selected portion further comprises modifying the determination of the second signal based on a second comparison between the one or more second movement parameters and second predetermined criteria, wherein the second predetermined criteria comprise physiological minimum and maximum values for a saccade amplitude, a saccade velocity, a saccade duration, and a fixation duration (i.e., an angular velocity of the gaze position data element that represents a saccade and a duration value of the fixation gaze position data elements) (see Trapp, par 0039 & 0045-0046).
Claim(s) 3, 12, 21 and 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Trapp as modified by Vandommele and Kempinski, as applied to claims 1 and 10, in further view of US Patent Application Publication 20170188823 --as previously cited--, hereinafter referenced as "Ganesan".
With respect to claim 3, Trapp as modified by Vandommele and Kempinski teaches the method of claim 1 as discussed above. Trapp as modified by Vandommele and Kempinski fails to teach deriving a blink duration, number of blinks per minute, time between blinks, or a maximum blink velocity during a movement of an eyelid when the eyelid is being opened, being closed, or both from the first signal.
Ganesan teaches an eye tracker system and method wherein a signal representative of a movement associated with an eye is generated from a series of images of the eye recorded from a camera (see Ganesan, par 0082 & fig. 7). Ganesan further teaches that the signal is indicative of eye openness (see Ganesan, par 0026), and that a computing device determines a first state associated with an open eye (i.e., the proportion of time where the eyes 0-20% closed) (see Ganesan, par 0026) or a second state associated with a closed eye (i.e., the proportion of time that the eyes are 80-100% closed) (see Ganesan, par 0026) based on whether at least one of one or more movement parameters exceeds a predetermined threshold (see Ganesan, par 0026). Ganesan teaches that the one or more movement parameters comprise: an eye openness (i.e., the proportion of time that the eyes are open or closed) (see Ganesan, par 0026), and one or more of: a blink duration (see Ganesan, par 0026); a number of blinks per minute (i.e., blink frequency) (see Ganesan, par 0026); a time between blinks; and a maximum blink velocity during a movement of an eyelid when the eye is being opened, being closed, or both.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Trapp as modified by Vandommele and Kempinski such that the method determines eye openness and derives movement parameters because determining eye openness and eye movement of a subject can be used to determine a variety of physiological and psychological conditions (see Ganesan, abstract).
With respect to claim 12, Trapp as modified by Vandommele and Kempinski teaches the system of claim 10 as discussed in paragraph 8 above. As discussed above, Trapp as modified by Vandommele and Kempinski teaches that the one or more third movement parameters further comprise one or more of a saccade amplitude, a saccade velocity, a saccade duration, and a fixation duration (i.e., an angular velocity of the gaze position data element that represents a saccade and a duration value of the fixation gaze position data elements) (see Trapp, par 0039 & 0045-0046). Trapp as modified by Vandommele and Kempinski does not teach that the one or more movement parameters comprise one or more of eye openness, a blink duration, number of blinks per minute, time between blinks, or a maximum blink velocity during a movement of an eyelid when the eye is being opened, being closed, or both.
Ganesan teaches an eye tracker system and method wherein a signal representative of a movement associated with an eye is generated from a series of images of the eye recorded from a camera (see Ganesan, par 0082 & fig. 7). Ganesan further teaches that the signal is indicative of eye openness (see Ganesan, par 0026), and that a computing device determines a first state associated with an open eye (i.e., the proportion of time where the eyes 0-20% closed) (see Ganesan, par 0026) or a second state associated with a closed eye (i.e., the proportion of time that the eyes are 80-100% closed) (see Ganesan, par 0026) based on whether at least one of one or more movement parameters exceeds a predetermined threshold (see Ganesan, par 0026). Ganesan teaches that the one or more movement parameters comprise: an eye openness (i.e., the proportion of time that the eyes are open or closed) (see Ganesan, par 0026), and one or more of: a blink duration (see Ganesan, par 0026); a number of blinks per minute (i.e., blink frequency) (see Ganesan, par 0026); a time between blinks; and a maximum blink velocity during a movement of an eyelid when the eye is being opened, being closed, or both.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Trapp as modified by Vandommele and Kempinski such that the method determines eye openness and derives movement parameters because determining eye openness and eye movement of a subject can be used to determine a variety of physiological and psychological conditions (see Ganesan, abstract).
With respect to claim 21, Trapp as modified by Vandommele and Kempinski fails to teach modifying the eye openness classification based on a comparison to physiological minimum and maximum values for a blink duration, a number of blinks per minute, and a time between blinks.
Ganesan teaches an eye tracker system and method wherein a signal representative of a movement associated with an eye is generated from a series of images of the eye recorded from a camera (see Ganesan, par 0082 & fig. 7). Ganesan teaches that predetermined criteria comprise a blink duration (see Ganesan, par 0026), a number of blinks per minute (i.e., blink frequency) (see Ganesan, par 0026), a time between blinks, and a maximum blink velocity during a movement of an eyelid when the eye is being opened, being closed, or both (see Ganesan, par 0026).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Trapp as modified by Vandommele and Kempinski such that is comprises modifying the eye openness classification based on a comparison to physiological minimum and maximum values for a blink duration, a number of blinks per minute, and a time between blinks because determining eye openness and eye movement of a subject can be used to determine a variety of physiological and psychological conditions (see Ganesan, abstract).
With respect to claim 24, Trapp teaches a computer-implemented method for determining a movement associated with an eye of a person, the method comprising:
recording, via a camera, a series of images of the eye (see Trapp, par 0028);
outputting the recorded series of images of the eye to an eye openness analysis unit of a computing device (i.e., outputting the recorded images to a computing device that determines whether the eye is open) (see Trapp, par 0029, wherein the fixation identification device is a computerized device comprising a processor and memory that receives gaze position data elements to identify user gaze fixation regions, thus determining whether the user’s eyes are open),
based on the recorded series of images, generating, by the eye openness analysis unit (i.e., by a processor) a first signal representative of a movement associated with the eye (i.e., gaze position data elements) (see Trapp, par 0029 & 0034);
deriving one or more movement parameters from the first signal (i.e., an angular velocity of the gaze position data) (see Trapp, par 0039-0040);
determining one or more first periods of time in which the first signal exceeds a first threshold (i.e., determining whether an angular velocity meets or exceeds a velocity threshold in order to categorize a gaze point as a saccade, or whether an angular velocity is below a velocity threshold in order to classify a gaze point as a fixation)(see Trapp, par 0005, 0039-0041);
determining one or more second periods of time in which the first signal does not exceed the first threshold (i.e., determining whether an angular velocity meets or exceeds a velocity threshold in order to categorize a gaze point as a saccade, or whether an angular velocity is below a velocity threshold in order to classify a gaze point as a fixation)(see Trapp, par 0005, 0039-0041);
determining a classification of the eye, the classification including the first signal indicating that the eye is in an open state (i.e., in a state of fixation) during the one or more first periods of time, indicating that the eye is in a closed state during the one or more second periods of time (i.e., whether or not the glint created on the surface of the pupil of a user can be identified by the camera) (see Trapp, par 0028, 0034, 0039);
modifying the classification based on a comparison of the one or more movement parameters and a predetermined maximum parameter value (i.e., modifying whether the eye is in a state of a fixation or saccade based upon the predetermined thresholds) (see Trapp, par 0039-0040);
after the classification is modified, selecting a portion of the first signal corresponding to one or more periods when the eye is classified in the open state (i.e., identifying certain data elements from the gaze position data as fixation gaze position data elements) (see Trapp, par 0042-0043);
and generating, by an eye gaze analysis unit of the computing device, a second signal from the recorded series of images indicating an eye gaze only during the one or more periods when the eye is classified in the open state and not when the eye is classified in the closed state, wherein the second signal comprises a pair of Euler angles or a position of a pupil of the eye relative to a center of the eye (i.e., the glint of light in the pupil of a user's eye is used as a reference point for the camera to track movement of the user's eye during different movements) (see Trapp, par 0028, 0038-0040);
deriving an eye movement velocity from the second signal, wherein the eye movement velocity is derived from a change of the second signal over time (see Trapp, par 0038-0040);
for each point in time of the one or more periods when the eye is classified in the open state, determining an eye gaze classification of the eye, wherein determining the eye gaze classification comprises classifying as a saccade when the eye movement velocity exceeds a second threshold and classifying as a fixation when the eye movement velocity is below the second threshold (see Trapp, fig. 1 & 3, par 0025, 0029, 0033-0048);
and during periods where the eye gaze classification of the eye is classified as the fixation, determining an averaged direction of gaze, and identifying an object at which the eye is looking (see Trapp, fig. 3, 0033-0048).
Trapp fails to teach modifying the eye gaze classification of the eye, wherein modifying the eye gaze classification comprises classifying as inconclusive when the eye movement velocity exceeds a predetermined maximum eye movement velocity, wherein the predetermined maximum eye movement velocity is based on a physiologically possible eye movement velocity, deriving a blink velocity derived from a change of the distance between the eyelids of the eye over time, modifying the eye openness classification based on a comparison of the blink velocity and a predetermined maximum blink velocity, wherein the predetermined blink velocity is a physiologically known limit of eye movement, and further Trapp fails to teach detecting driver drowsiness based on the eye openness classification.
Vandommele teaches a method and device for classifying eye opening data of at least one eye of an occupant of a vehicle wherein a position of the eyelids of the eyes of a driver are recorded, and a distance between the eyelids of at least one of the driver’s eyes is determined (see Vandommele, par 0014, 0042, 0046). Further, a speed and/or acceleration of a movement of the eyelids of at least one of the eyes is recorded, and the eyelid speed is derived from the determined eye opening degree (i.e., the distance between the eyelids) (see Vandommele, par 0042, 0046, 0048). Additionally, images are analyzed to determine that an eye is in a transition state when it goes from closing to opening in an opening phase (i.e., the eye is in the closed state and subsequent and preceding images indicate that the eye is in the open state and the status of the eye is modified accordingly) (see Vandommele, par 0014, 0023-0024, 0053, 0065-0069, fig. 4, 6). The aforementioned techniques are used to detect driver drowsiness based on the eye openness classification (see Vandommele, abstract, 0017, 0025-0028, 0042-0043, 0070, 0083-0084, fig. 1-2, 4).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Trapp such that the method is for drivers of a vehicle, and detects driver drowsiness based on the modified eye openness classification because the aforementioned techniques permit determining whether or not a driver of a vehicle is drowsy so as to prevent dangerous accidents that arise from a driver falling asleep behind the wheel (see Vandommele, par 0002, 0007, 0010-0013, 0042-0043, 0078-0085, fig. 9).
Trapp as modified by Vandommele fails to teach modifying the eye gaze classification as inconclusive when the eye movement velocity exceeds a predetermined maximum eye velocity, wherein the predetermined maximum eye movement velocity is based on a physiologically possible eye movement velocity.
Kempinski teaches a system and method for analysis of eye movements using two dimensional images wherein eye movement data that has been classified based on iris to eye corner movement is filtered to remove signals that are unreliable, and determining that the data is indicative of eye movement or of noise is based upon normal eye physiology (see Kempinski, par 0053-0055).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Trapp as modified by Vandommele such that the method comprises modifying the eye gaze classification as inconclusive when the eye movement velocity exceeds a predetermined maximum eye velocity, wherein the predetermined maximum eye movement velocity is based on a physiologically possible eye movement velocity because that removes unwanted signals that come from noise, or unwanted signals that are not truly indicative of a user’s eye movement (see Kempinski, par 0053-0055).
Trapp as modified by Vandommele and Kempinski fails to teach deriving a blink velocity derived from a change of the distance between the eyelids of the eye over time, and modifying the eye openness classification based on a comparison of the blink velocity and a predetermined maximum blink velocity, wherein the predetermined blink velocity is a physiologically known limit of eye movement.
Ganesan teaches an eye tracker system and method wherein a signal representative of a movement associated with an eye is generated from a series of images of the eye recorded from a camera (see Ganesan, par 0082 & fig. 7). Ganesan teaches that predetermined criteria comprise a blink duration (see Ganesan, par 0026), a number of blinks per minute (i.e., blink frequency) (see Ganesan, par 0026), a time between blinks, and a maximum blink velocity during a movement of an eyelid when the eye is being opened, being closed, or both (see Ganesan, par 0026).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Trapp as modified by Vandommele and Kempinski such that it comprises deriving a blink velocity derived from a change of the distance between the eyelids of the eye over time, and modifying the eye openness classification based on a comparison of the blink velocity and a predetermined maximum blink velocity, wherein the predetermined blink velocity is a physiologically known limit of eye movement because determining eye openness and eye movement of a subject based upon blinking can be used to determine a variety of physiological and psychological conditions (see Ganesan, abstract).
Claim(s) 8 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Trapp as modified by Vandommele and Kempinski, as applied to claims 1 and 10, in further view of US Patent Application Publication 20050073136 --as previously recited--, hereinafter referenced as "Larsson".
With respect to claim 8, Trapp as modified by Vandommele and Kempinski teaches the method of claim 4 as discussed above.
Trapp further teaches determining one or more fixation periods (i.e., determining fixation gaze position data durations) (see Trapp, par 0045-0046).
Trapp as modified by Vandommele and Kempinski does not teach that the method further comprises determining for each fixation period, an averaged eye gaze signal.
Larsson teaches determining an averaged eye gaze signal (i.e., determining the gaze direction by calculating the mean of gaze vectors representing data for each eye) (see Larsson, par 0160).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to determine an averaged eye gaze signal, as taught by Larsson, in the method of determining an eye movement, as taught by Trapp as modified by Vandommele and Kempinski, because determining the averaged eye gaze signal enables the determination of the gaze direction of the subject (see Larsson, par 0160).
With respect to claim 17, Trapp as modified by Vandommele and Kempinski teaches the system of claim 10 as discussed in paragraph 8 above.
Trapp teaches that the computing device 28 is further operable to execute the steps of: determine one or more fixation periods (i.e., determining fixation gaze position data durations) (see Trapp, par 0045-0046).
Trapp as modified by Vandommele and Kempinski does not teach that the system further comprises determining for each fixation period, an averaged eye gaze signal.
Larsson teaches determining an averaged eye gaze signal (i.e., determining the gaze direction by calculating the mean of gaze vectors representing data for each eye) (see Larsson, par 0160).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to determine an averaged eye gaze signal, as taught by Larsson, in the system for determining an eye movement, as taught by Trapp as modified by Vandommele and Kempinski, because determining the averaged eye gaze signal enables the determination of the gaze direction of the subject (see Larsson, par 0160).
Claim(s) 9 & 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Trapp as modified by Vandommele and Kempinski, as applied to claims 1 and 10, in further view of US Patent Application Publication 20180338700 --as previously cited--, hereinafter referenced as "Khasnobish".
With respect to claim 9, Trapp as modified by Vandommele and Kempinski teaches the method of claim 1 as discussed in paragraph 8 above. Trapp as modified by Vandommele and Kempinski does not teach that the method further comprises filtering measurement noise from the first signal.
Khasnobish teaches filtering measurement noise from an original signal (see Khasnobish, par 0021, fig. 2).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to filter measurement noise from an original signal, as taught by Khasnobish, in the method as taught by Trapp as modified by Vandommele and Kempinski, because filtering noise from the original signal generates filtered signals that are better representative of a subject’s eye movement (see Khasnobish, par 0003-0006).
With respect to claim 18, Trapp as modified by Vandommele and Kempinski teaches the system of claim 10 as discussed in paragraph 8 above. Trapp as modified by Vandommele and Kempinski does not teach that the computing device is further operable to execute the steps of filtering measurement noise from the first signal.
Khasnobish teaches filtering measurement noise from an original signal (see Khasnobish, par 0021, fig. 2).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to filter measurement noise from an original signal, as taught by Khasnobish, in the system as taught by Trapp as modified by Vandommele and Kempinski, because filtering noise from the original signal generates filtered signals that are better representative of a subject’s eye movement (see Khasnobish, par 0003-0006).
Claim(s) 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Trapp as modified by Vandommele and Kempinski, as applied to claims 1 and 10, in further view of US Patent Application Publication 20190294312 --as previously cited--, hereinafter referenced as "Rohrbacher".
With respect to claim 23, Trapp as modified teaches the system of claim 10, but fails to teach the system comprises an illumination unit. Rohrbacher teaches a system for combined eye and gesture tracking comprising an illumination source, such as an infrared/near-infrared light as the illumination source that as using a light source in an eye tracking system permits bright-pupil tracking for reliable eye-tracking, and further as infrared/near-infrared light is invisible to the human eye and thus does not cause distraction (see Rohrbacher, par 0041).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Trapp as modified with Vandommele and Kempinski such that the system comprises an illumination unit because using a light source in an eye tracking system permits bright-pupil tracking for reliable eye-tracking, and further light sources such as infrared/near-infrared light are invisible to the human eye and thus do not cause distraction to a user (see Rohrbacher, par 0041).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Destiny J Cruickshank whose telephone number is (571)270-0187. The examiner can normally be reached M-F, 9am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Charles Marmor II can be reached at (571) 272-4730. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHARLES A MARMOR II/Supervisory Patent Examiner
Art Unit 3791
/D.J.C./Examiner, Art Unit 3791
Prosecution Insights