DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
2. Claim 1 is objected to because of the following informalities:
in line 27, “the first pupil” is a typographical error, and should be amended to “the [[ ]]first pupil”.
Appropriate correction is required.
Claim Rejections - 35 USC § 102
3. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
5. Claims 1-3, 6-9, 11, 13-16, 18, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Fichtler et al., U.S. Patent Application Publication No. 2023/0233120 A1 (“Fichtler”).
As to Claim 1, Fichtler teaches the following:
A system (“system”) 200 configured to process images (see “The system includes a processor programmed to analyze the eye movements and pupil data size.” in Abstract), the system comprising:
a computer device (“processor”) 210 (see “The system 200 includes a processor 210 and an automated impairment decision engine 205.” in para. [0056]);
non-transitory computer readable memory (“automated impairment decision engine”) 205 having program instructions stored thereon that when executed by the computer device 210 cause the system 200 to perform operations (see “The processor 210 may be integral to the automated impairment decision engine 205 as would be appreciated by one of ordinary skill in the art having the benefit of this disclosure. The processor 210 is configured to process the data from the one or more eye tracking sensors 235, the camera 245, and/or the one or more other sensors 240. The data may be received from the onboard storage 215 and/or the cloud storage 220.” in para. [0056]) comprising:
accessing a first plurality of images of a user captured using a camera (see “An output of the apparatus may be video recording and/or digital imagery which is not evaluated, used to train evaluative algorithms, or altered by the system. The video recording and/or imagery is captured by one or more cameras and stored either on the testing apparatus, on a mobile or desktop computing platform, or in cloud computing environments. This video and/or imagery can then be evaluated by existing human Drug Recognition Experts or other interested parties as required. Either visible or infrared cameras may be utilized to capture video and imagery from the automatically performed tests.” in para. [0050]);
enhancing the first plurality of images by adjusting luminescence, contrast, and/or sharpness (see “Initial data processing may include data normalization and cleaning.” in para. [0040]);
locating a face in the first plurality of images using Haar cascades, a Histogram of Oriented Gradients, a Viola-Jones framework, and/or a first deep learning algorithm (see “Calibration—This test seeks to precisely calibrate the system to the face shape, eye characteristics and eye geometry of the user. The eye movement of the user is measured during a multi-part test that may include measuring interpupillary distance, eye tracking using a moving or stationary stimulus, the measurement of pupil size, blink rate, and other biometric or eye movement characteristics. Utilizing this data, the system may make adjustments to the software or hardware of the testing apparatus to precisely align the eye tracking sensors, eye cameras, and other biometric sensors to capture optimal data. The calibration test may additionally measure reaction time, and capture eye movement data during the calibration test for later evaluation by the machine learning/artificial intelligence algorithm.” in para. [0029]);
locating first and second eyes in the face using a plurality of located facial landmarks and/or a convolutional neural network (see “Examples of data which may be utilized to automatically identify a test subject includes eye measurement data such as interpupillary distance, or any other data captured by the system, such as eye movement characteristics, corneal print, image recognition, pupil size, or other biometric data. The system may utilize this test data to automatically match the test subject against a database of known test subjects. This test subject matching algorithm can then be utilized to group a test subject's test data together, to send test results to specific third parties, or to allow the system to administer certain tests which may be most relevant to that user.” in para. [0025]);
locating respective pupils in the located first and second eyes (see “Calibration—This test seeks to precisely calibrate the system to the face shape, eye characteristics and eye geometry of the user. … Utilizing this data, the system may make adjustments to the software or hardware of the testing apparatus to precisely align the eye tracking sensors, eye cameras, and other biometric sensors to capture optimal data.” in para. [0029]);
using optical flow and/or a second deep learning algorithm to determine movements of at least the first eye (see “While the test apparatus conducts the automatically performed tests, the apparatus collects gaze vector data, pupil size measurement data, and other optional biometric data. This data is then automatically evaluated by a processor, or the like, programmed to use statistical models, and/or machine learning or artificial intelligence algorithms to identify characteristics within the data that are consistent with impairment and/or sobriety.” in para. [0027]);
detecting eye jerking of at least the first eye over two or more images in the first plurality of images based at least in part on the determined movements of the first eye (see “Horizontal Gaze Nystagmus—tests for involuntary jerking movement of the eyes at the left and right periphery of vision. The system conducts this test by moving the stimulus to the left and/or right periphery of the user's vision one or more times. An impaired person's eyes may exhibit nystagmus, or sustained jerking motion, in the eyes during stationary focus.” in para. [0031]);
determining a distance of the detected eye jerking of the first eye and how long the detected eye jerking lasted (see “Onset of Horizontal Gaze Nystagmus Before 45 Degrees—tests for involuntary jerking of the eye using a stimulus held at 45 degrees or less horizontally from center. Similar to the Horizontal Gaze Nystagmus test above, the system moves the stimulus left and/or right one or more times. However, this test stops the stimulus at or before 45 degrees from center. The test may also stop the stimulus as soon as nystagmus is detected and record the angle of onset. An impaired person's eyes may exhibit nystagmus at angles of onset that are not on the periphery of vision.” in para. [0032]);
based at least in part on the determined distance of the detected eye jerking of the first eye and how long the detected eye jerking lasted, generating a first intoxication indicator (see “Onset of Horizontal Gaze Nystagmus Before 45 Degrees—tests for involuntary jerking of the eye using a stimulus held at 45 degrees or less horizontally from center. Similar to the Horizontal Gaze Nystagmus test above, the system moves the stimulus left and/or right one or more times. However, this test stops the stimulus at or before 45 degrees from center. The test may also stop the stimulus as soon as nystagmus is detected and record the angle of onset. An impaired person's eyes may exhibit nystagmus at angles of onset that are not on the periphery of vision.” in para. [0032]; and see “Vertical Gaze Nystagmus—tests for involuntary jerking of the eye at the upper or lower periphery of vision. The system conducts this test by moving the stimulus up and/or down to the upper and/or lower periphery of the user's vision one or more times. The system may also stop the stimulus as soon as nystagmus is detected and record the angle of onset. An impaired person's eyes may exhibit nystagmus on the upper or lower peripheries of vision, or at an angle of onset prior to the periphery.” in para. [0033]);
determining a size of a first pupil in one or more of the first plurality of images using a number of pixels in a line defining a diameter of the first pupil (see “The system measures pupil size throughout this test and may optionally monitor eye movement.” in para. [0035]);
based at least in part on the determined size of the first pupil, generating a second intoxication indicator (see “The rate at which a user's pupil responds to a change in light conditions may indicate impairment. For example, a person's pupils may persist in a dilated or constricted state, depending on the substance a person is impaired on.” in para. [0035]);
detecting, in the first plurality of images, eye blinking of at least the first eye using positions of one or more eye landmarks and/or using changes in pixel intensities over time (see “The eye movement of the user is measured during a multi-part test that may include measuring interpupillary distance, eye tracking using a moving or stationary stimulus, the measurement of pupil size, blink rate, and other biometric or eye movement characteristics.” in para. [0029]);
determining an eye blink rate of at least the first eye based at least in part on the detected eye blinking (see “The eye movement of the user is measured during a multi-part test that may include measuring interpupillary distance, eye tracking using a moving or stationary stimulus, the measurement of pupil size, blink rate, and other biometric or eye movement characteristics.” in para. [0029]);
using the determined eye blink rate of the first eye, generating a third intoxication indicator (see “Additional types of non-compliance can include, but are not limited to, looking straight ahead or at another random point for the duration or part of the testing process, moving the eyes in a random manner for some or all of the testing process, tracking the stimulus only intermittently, ignoring test instructions, rapidly blinking for a sustained period of time, or cessation of stimulus tracking at any point. The detection of non-compliant users may be important step in accurately determining whether or not an individual is impaired.” in para. [0051]);
using the first intoxication indicator, the second intoxication indicator, and the third intoxication indicator, determining whether the user has a capacity to consent to a first act (see “If a subject is found to be impaired or with abnormal eye movement due to one or multiple tests, the test results may be communicated automatically to the test administrator, or other third party as required. This is accomplished through any of the following methods: email, SMS/MMS messages, notification utilizing the companion software application, or any other digital means. The data may be utilized in evaluated form, or in raw form by test administrators, test subjects, or other interested parties as required.” in para. [0048]; and see “If a subject is found to be not impaired, the test data may similarly be made available, or stored for later reference. Test subjects could alternatively request that test data be deleted rather than stored.” in para. [0049]); and
at least partly in response to determining that the user lacks the capacity to consent to the first act, causing one or more messages to be generated and transmitted to one or more respective electronic destinations (see “If a subject is found to be impaired or with abnormal eye movement due to one or multiple tests, the test results may be communicated automatically to the test administrator, or other third party as required. This is accomplished through any of the following methods: email, SMS/MMS messages, notification utilizing the companion software application, or any other digital means. The data may be utilized in evaluated form, or in raw form by test administrators, test subjects, or other interested parties as required.” in para. [0048]).
As to Claim 2, Fichtler teaches the following:
wherein the first intoxication indicator, the second intoxication indicator, and the third intoxication indicator are weighted differently in determining whether the user has the capacity to consent to the first act (see “The weight of each wavelet feature in a recording is how prevalent the pattern of that wavelet is along the recording.” in para. [0042]).
As to Claim 3, Fichtler teaches the following:
wherein determining the distance of the detected eye jerking of the first eye and how long the detected eye jerking lasted, further comprises determining whether the user has nystagmus (see “Vertical Gaze Nystagmus—tests for involuntary jerking of the eye at the upper or lower periphery of vision. The system conducts this test by moving the stimulus up and/or down to the upper and/or lower periphery of the user's vision one or more times. The system may also stop the stimulus as soon as nystagmus is detected and record the angle of onset. An impaired person's eyes may exhibit nystagmus on the upper or lower peripheries of vision, or at an angle of onset prior to the periphery.” in para. [0033]).
As to Claim 6, Fichtler teaches the following:
wherein the system is configured to enable instructions to the user to hold the camera in a left hand facing the user, position the camera at eye level, move the camera from far left of the user's face to directly in front of the user's face, and to hold the camera in a right hand facing the user, position the camera at eye level, move the camera from far right of the user's face to directly in front of the user's face, wherein at least a portion of the first plurality of images are captured during such movements (see “Calibration—This test seeks to precisely calibrate the system to the face shape, eye characteristics and eye geometry of the user. The eye movement of the user is measured during a multi-part test that may include measuring interpupillary distance, eye tracking using a moving or stationary stimulus, the measurement of pupil size, blink rate, and other biometric or eye movement characteristics. Utilizing this data, the system may make adjustments to the software or hardware of the testing apparatus to precisely align the eye tracking sensors, eye cameras, and other biometric sensors to capture optimal data. The calibration test may additionally measure reaction time, and capture eye movement data during the calibration test for later evaluation by the machine learning/artificial intelligence algorithm. In some cases, a user's facial geometry may fall outside of the parameters that are testable by the apparatus.” in para. [0029]).
As to Claim 7, Fichtler teaches the following:
A computer implemented method (see “One embodiment of the disclosure is a method of using a testing apparatus and collecting data from the testing apparatus.” in para. [0013]), the method comprising:
accessing from memory (“automated impairment decision engine”) 205 a first plurality of images of a user captured using a camera, at least a portion of the first plurality of images captured while the camera was being between a side of the user's face to a front of the user's face (see “An output of the apparatus may be video recording and/or digital imagery which is not evaluated, used to train evaluative algorithms, or altered by the system. The video recording and/or imagery is captured by one or more cameras and stored either on the testing apparatus, on a mobile or desktop computing platform, or in cloud computing environments. This video and/or imagery can then be evaluated by existing human Drug Recognition Experts or other interested parties as required. Either visible or infrared cameras may be utilized to capture video and imagery from the automatically performed tests.” in para. [0050]);
locating the face in the first plurality of images using Haar cascades, a Histogram of Oriented Gradients, a Viola-Jones framework, and/or a first deep learning algorithm (see “Calibration—This test seeks to precisely calibrate the system to the face shape, eye characteristics and eye geometry of the user. The eye movement of the user is measured during a multi-part test that may include measuring interpupillary distance, eye tracking using a moving or stationary stimulus, the measurement of pupil size, blink rate, and other biometric or eye movement characteristics. Utilizing this data, the system may make adjustments to the software or hardware of the testing apparatus to precisely align the eye tracking sensors, eye cameras, and other biometric sensors to capture optimal data. The calibration test may additionally measure reaction time, and capture eye movement data during the calibration test for later evaluation by the machine learning/artificial intelligence algorithm.” in para. [0029]);
locating at least a first eye in the face using a plurality of located facial landmarks and/or a convolutional neural network (see “Examples of data which may be utilized to automatically identify a test subject includes eye measurement data such as interpupillary distance, or any other data captured by the system, such as eye movement characteristics, corneal print, image recognition, pupil size, or other biometric data. The system may utilize this test data to automatically match the test subject against a database of known test subjects. This test subject matching algorithm can then be utilized to group a test subject's test data together, to send test results to specific third parties, or to allow the system to administer certain tests which may be most relevant to that user.” in para. [0025]);
locating a pupil in the first eye (see “Calibration—This test seeks to precisely calibrate the system to the face shape, eye characteristics and eye geometry of the user. … Utilizing this data, the system may make adjustments to the software or hardware of the testing apparatus to precisely align the eye tracking sensors, eye cameras, and other biometric sensors to capture optimal data.” in para. [0029]);
determining movements of the first eye in the first plurality of images (see “While the test apparatus conducts the automatically performed tests, the apparatus collects gaze vector data, pupil size measurement data, and other optional biometric data. This data is then automatically evaluated by a processor, or the like, programmed to use statistical models, and/or machine learning or artificial intelligence algorithms to identify characteristics within the data that are consistent with impairment and/or sobriety.” in para. [0027]);
detecting eye jerking of at least the first eye over two or more images in the first plurality of images based at least in part on the determined movements of the first eye (see “Horizontal Gaze Nystagmus—tests for involuntary jerking movement of the eyes at the left and right periphery of vision. The system conducts this test by moving the stimulus to the left and/or right periphery of the user's vision one or more times. An impaired person's eyes may exhibit nystagmus, or sustained jerking motion, in the eyes during stationary focus.” in para. [0031]);
determining a distance of the detected eye jerking of the first eye and how long the detected eye jerking lasted (see “Onset of Horizontal Gaze Nystagmus Before 45 Degrees—tests for involuntary jerking of the eye using a stimulus held at 45 degrees or less horizontally from center. Similar to the Horizontal Gaze Nystagmus test above, the system moves the stimulus left and/or right one or more times. However, this test stops the stimulus at or before 45 degrees from center. The test may also stop the stimulus as soon as nystagmus is detected and record the angle of onset. An impaired person's eyes may exhibit nystagmus at angles of onset that are not on the periphery of vision.” in para. [0032]);
based at least in part on the determined distance of the detected eye jerking of the first eye and how long the detected eye jerking lasted, generating a first intoxication indicator (see “Onset of Horizontal Gaze Nystagmus Before 45 Degrees—tests for involuntary jerking of the eye using a stimulus held at 45 degrees or less horizontally from center. Similar to the Horizontal Gaze Nystagmus test above, the system moves the stimulus left and/or right one or more times. However, this test stops the stimulus at or before 45 degrees from center. The test may also stop the stimulus as soon as nystagmus is detected and record the angle of onset. An impaired person's eyes may exhibit nystagmus at angles of onset that are not on the periphery of vision.” in para. [0032]; and see “Vertical Gaze Nystagmus—tests for involuntary jerking of the eye at the upper or lower periphery of vision. The system conducts this test by moving the stimulus up and/or down to the upper and/or lower periphery of the user's vision one or more times. The system may also stop the stimulus as soon as nystagmus is detected and record the angle of onset. An impaired person's eyes may exhibit nystagmus on the upper or lower peripheries of vision, or at an angle of onset prior to the periphery.” in para. [0033]);
using the first intoxication indicator, determining whether the user has a capacity to consent to a first act (see “If a subject is found to be impaired or with abnormal eye movement due to one or multiple tests, the test results may be communicated automatically to the test administrator, or other third party as required. This is accomplished through any of the following methods: email, SMS/MMS messages, notification utilizing the companion software application, or any other digital means. The data may be utilized in evaluated form, or in raw form by test administrators, test subjects, or other interested parties as required.” in para. [0048]; and see “If a subject is found to be not impaired, the test data may similarly be made available, or stored for later reference. Test subjects could alternatively request that test data be deleted rather than stored.” in para. [0049]); and
at least partly in response to determining that the user lacks the capacity to consent to the first act, causing one or more messages to be generated and transmitted to one or more respective electronic destinations (see “If a subject is found to be impaired or with abnormal eye movement due to one or multiple tests, the test results may be communicated automatically to the test administrator, or other third party as required. This is accomplished through any of the following methods: email, SMS/MMS messages, notification utilizing the companion software application, or any other digital means. The data may be utilized in evaluated form, or in raw form by test administrators, test subjects, or other interested parties as required.” in para. [0048]).
As to Claim 8, Fichtler teaches the following:
determining a diameter of the pupil of the first eye in one or more of the first plurality of images using a number of pixels in a line defining a diameter of the pupil of the first eye see “The system measures pupil size throughout this test and may optionally monitor eye movement.” in para. [0035]); and
wherein using the first intoxication indicator in determining whether the user has the capacity to consent to the first act, further comprises using the determined diameter of the pupil in determining whether the user has the capacity to consent to the first act (see “The rate at which a user's pupil responds to a change in light conditions may indicate impairment. For example, a person's pupils may persist in a dilated or constricted state, depending on the substance a person is impaired on.” in para. [0035]).
As to Claim 9, Fichtler teaches the following:
detecting, in the first plurality of images, eye blinking of the first eye using positions of one or more eye landmarks and/or using changes in pixel intensities over time (see “The eye movement of the user is measured during a multi-part test that may include measuring interpupillary distance, eye tracking using a moving or stationary stimulus, the measurement of pupil size, blink rate, and other biometric or eye movement characteristics.” in para. [0029]);
determining an eye blink rate of at least the first eye based at least in part on the detected eye blinking (see “The eye movement of the user is measured during a multi-part test that may include measuring interpupillary distance, eye tracking using a moving or stationary stimulus, the measurement of pupil size, blink rate, and other biometric or eye movement characteristics.” in para. [0029]),
wherein using the first intoxication indicator in determining whether the user has the capacity to consent to the first act, further comprises using the determined eye blink rate in determining whether the user has the capacity to consent to the first act (see “If a subject is found to be impaired or with abnormal eye movement due to one or multiple tests, the test results may be communicated automatically to the test administrator, or other third party as required. This is accomplished through any of the following methods: email, SMS/MMS messages, notification utilizing the companion software application, or any other digital means. The data may be utilized in evaluated form, or in raw form by test administrators, test subjects, or other interested parties as required.” in para. [0048]; and see “If a subject is found to be not impaired, the test data may similarly be made available, or stored for later reference. Test subjects could alternatively request that test data be deleted rather than stored.” in para. [0049]).
As to Claim 11, Fichtler teaches the following:
wherein determining the distance of the detected eye jerking of the first eye and how long the detected eye jerking lasted, further comprises determining whether the user has nystagmus (see “Vertical Gaze Nystagmus—tests for involuntary jerking of the eye at the upper or lower periphery of vision. The system conducts this test by moving the stimulus up and/or down to the upper and/or lower periphery of the user's vision one or more times. The system may also stop the stimulus as soon as nystagmus is detected and record the angle of onset. An impaired person's eyes may exhibit nystagmus on the upper or lower peripheries of vision, or at an angle of onset prior to the periphery.” in para. [0033]).
As to Claim 13, Fichtler teaches the following:
electronically causing instructions to be audibly provided to the user to hold the camera in a left hand facing the user, position the camera at eye level, move the camera from far left of the user's face to directly in front of the user's face, and to hold the camera in a right hand facing the user, position the camera at eye level, move the camera from far right of the user's face to directly in front of the user's face, wherein at least a portion of the first plurality of images are captured during such camera movements (see “Calibration—This test seeks to precisely calibrate the system to the face shape, eye characteristics and eye geometry of the user. The eye movement of the user is measured during a multi-part test that may include measuring interpupillary distance, eye tracking using a moving or stationary stimulus, the measurement of pupil size, blink rate, and other biometric or eye movement characteristics. Utilizing this data, the system may make adjustments to the software or hardware of the testing apparatus to precisely align the eye tracking sensors, eye cameras, and other biometric sensors to capture optimal data. The calibration test may additionally measure reaction time, and capture eye movement data during the calibration test for later evaluation by the machine learning/artificial intelligence algorithm. In some cases, a user's facial geometry may fall outside of the parameters that are testable by the apparatus.” in para. [0029]).
As to Claim 14, Fichtler teaches the following:
Non-transitory computer readable memory (“automated impairment decision engine”) 205 having program instructions stored thereon that when executed by a computing device (“processor”) 210 (see “The system 200 includes a processor 210 and an automated impairment decision engine 205.” in para. [0056]) cause the computing device 210 to perform operations (see “The processor 210 may be integral to the automated impairment decision engine 205 as would be appreciated by one of ordinary skill in the art having the benefit of this disclosure. The processor 210 is configured to process the data from the one or more eye tracking sensors 235, the camera 245, and/or the one or more other sensors 240. The data may be received from the onboard storage 215 and/or the cloud storage 220.” in para. [0056]) comprising:
accessing from memory (“automated impairment decision engine”) 205 a first plurality of images of a user captured using a camera, at least a portion of the first plurality of images captured while the camera was being between a side of the user's face to a front of the user's face (see “An output of the apparatus may be video recording and/or digital imagery which is not evaluated, used to train evaluative algorithms, or altered by the system. The video recording and/or imagery is captured by one or more cameras and stored either on the testing apparatus, on a mobile or desktop computing platform, or in cloud computing environments. This video and/or imagery can then be evaluated by existing human Drug Recognition Experts or other interested parties as required. Either visible or infrared cameras may be utilized to capture video and imagery from the automatically performed tests.” in para. [0050]);
locating the face in the first plurality of images using Haar cascades, a Histogram of Oriented Gradients, a Viola-Jones framework, and/or a first deep learning algorithm (see “Calibration—This test seeks to precisely calibrate the system to the face shape, eye characteristics and eye geometry of the user. The eye movement of the user is measured during a multi-part test that may include measuring interpupillary distance, eye tracking using a moving or stationary stimulus, the measurement of pupil size, blink rate, and other biometric or eye movement characteristics. Utilizing this data, the system may make adjustments to the software or hardware of the testing apparatus to precisely align the eye tracking sensors, eye cameras, and other biometric sensors to capture optimal data. The calibration test may additionally measure reaction time, and capture eye movement data during the calibration test for later evaluation by the machine learning/artificial intelligence algorithm.” in para. [0029]);
locating at least a first eye in the face using a plurality of located facial landmarks and/or a convolutional neural network (see “Examples of data which may be utilized to automatically identify a test subject includes eye measurement data such as interpupillary distance, or any other data captured by the system, such as eye movement characteristics, corneal print, image recognition, pupil size, or other biometric data. The system may utilize this test data to automatically match the test subject against a database of known test subjects. This test subject matching algorithm can then be utilized to group a test subject's test data together, to send test results to specific third parties, or to allow the system to administer certain tests which may be most relevant to that user.” in para. [0025]);
locating a pupil in the first eye (see “Calibration—This test seeks to precisely calibrate the system to the face shape, eye characteristics and eye geometry of the user. … Utilizing this data, the system may make adjustments to the software or hardware of the testing apparatus to precisely align the eye tracking sensors, eye cameras, and other biometric sensors to capture optimal data.” in para. [0029]);
determining movements of the first eye (see “While the test apparatus conducts the automatically performed tests, the apparatus collects gaze vector data, pupil size measurement data, and other optional biometric data. This data is then automatically evaluated by a processor, or the like, programmed to use statistical models, and/or machine learning or artificial intelligence algorithms to identify characteristics within the data that are consistent with impairment and/or sobriety.” in para. [0027]);
detecting eye jerking of at least the first eye over two or more images in the first plurality of images based at least in part on the determined movements of the first eye (see “Horizontal Gaze Nystagmus—tests for involuntary jerking movement of the eyes at the left and right periphery of vision. The system conducts this test by moving the stimulus to the left and/or right periphery of the user's vision one or more times. An impaired person's eyes may exhibit nystagmus, or sustained jerking motion, in the eyes during stationary focus.” in para. [0031]);
determining a distance of the detected eye jerking of the first eye and how long the detected eye jerking lasted (see “Onset of Horizontal Gaze Nystagmus Before 45 Degrees—tests for involuntary jerking of the eye using a stimulus held at 45 degrees or less horizontally from center. Similar to the Horizontal Gaze Nystagmus test above, the system moves the stimulus left and/or right one or more times. However, this test stops the stimulus at or before 45 degrees from center. The test may also stop the stimulus as soon as nystagmus is detected and record the angle of onset. An impaired person's eyes may exhibit nystagmus at angles of onset that are not on the periphery of vision.” in para. [0032]);
based at least in part on the determined distance of the detected eye jerking of the first eye and how long the detected eye jerking lasted, generating a first intoxication indicator (see “Onset of Horizontal Gaze Nystagmus Before 45 Degrees—tests for involuntary jerking of the eye using a stimulus held at 45 degrees or less horizontally from center. Similar to the Horizontal Gaze Nystagmus test above, the system moves the stimulus left and/or right one or more times. However, this test stops the stimulus at or before 45 degrees from center. The test may also stop the stimulus as soon as nystagmus is detected and record the angle of onset. An impaired person's eyes may exhibit nystagmus at angles of onset that are not on the periphery of vision.” in para. [0032]; and see “Vertical Gaze Nystagmus—tests for involuntary jerking of the eye at the upper or lower periphery of vision. The system conducts this test by moving the stimulus up and/or down to the upper and/or lower periphery of the user's vision one or more times. The system may also stop the stimulus as soon as nystagmus is detected and record the angle of onset. An impaired person's eyes may exhibit nystagmus on the upper or lower peripheries of vision, or at an angle of onset prior to the periphery.” in para. [0033]);
using the first intoxication indicator, determining whether the user has a capacity to consent to a first act (see “If a subject is found to be impaired or with abnormal eye movement due to one or multiple tests, the test results may be communicated automatically to the test administrator, or other third party as required. This is accomplished through any of the following methods: email, SMS/MMS messages, notification utilizing the companion software application, or any other digital means. The data may be utilized in evaluated form, or in raw form by test administrators, test subjects, or other interested parties as required.” in para. [0048]; and see “If a subject is found to be not impaired, the test data may similarly be made available, or stored for later reference. Test subjects could alternatively request that test data be deleted rather than stored.” in para. [0049]); and
at least partly in response to determining that the user lacks the capacity to consent to the first act, causing one or more messages to be generated and transmitted to one or more respective electronic destinations (see “If a subject is found to be impaired or with abnormal eye movement due to one or multiple tests, the test results may be communicated automatically to the test administrator, or other third party as required. This is accomplished through any of the following methods: email, SMS/MMS messages, notification utilizing the companion software application, or any other digital means. The data may be utilized in evaluated form, or in raw form by test administrators, test subjects, or other interested parties as required.” in para. [0048]).
As to Claim 15, Fichtler teaches the following:
determining a diameter of the pupil of the first eye in one or more of the first plurality of images using a number of pixels in a line defining a diameter of the pupil of the first eye see “The system measures pupil size throughout this test and may optionally monitor eye movement.” in para. [0035]); and
wherein using the first intoxication indicator in determining whether the user has the capacity to consent to the first act, further comprises using the determined diameter of the pupil in determining whether the user has the capacity to consent to the first act (see “The rate at which a user's pupil responds to a change in light conditions may indicate impairment. For example, a person's pupils may persist in a dilated or constricted state, depending on the substance a person is impaired on.” in para. [0035]).
As to Claim 16, Fichtler teaches the following:
detecting, in the first plurality of images, eye blinking of the first eye using positions of one or more eye landmarks and/or using changes in pixel intensities over time (see “The eye movement of the user is measured during a multi-part test that may include measuring interpupillary distance, eye tracking using a moving or stationary stimulus, the measurement of pupil size, blink rate, and other biometric or eye movement characteristics.” in para. [0029]);
determining an eye blink rate of at least the first eye based at least in part on the detected eye blinking (see “The eye movement of the user is measured during a multi-part test that may include measuring interpupillary distance, eye tracking using a moving or stationary stimulus, the measurement of pupil size, blink rate, and other biometric or eye movement characteristics.” in para. [0029]),
wherein using the first intoxication indicator in determining whether the user has the capacity to consent to the first act, further comprises using the determined eye blink rate in determining whether the user has the capacity to consent to the first act (see “If a subject is found to be impaired or with abnormal eye movement due to one or multiple tests, the test results may be communicated automatically to the test administrator, or other third party as required. This is accomplished through any of the following methods: email, SMS/MMS messages, notification utilizing the companion software application, or any other digital means. The data may be utilized in evaluated form, or in raw form by test administrators, test subjects, or other interested parties as required.” in para. [0048]; and see “If a subject is found to be not impaired, the test data may similarly be made available, or stored for later reference. Test subjects could alternatively request that test data be deleted rather than stored.” in para. [0049]).
As to Claim 18, Fichtler teaches the following:
wherein determining the distance of the detected eye jerking of the first eye and how long the detected eye jerking lasted, further comprises determining whether the user has nystagmus (see “Vertical Gaze Nystagmus—tests for involuntary jerking of the eye at the upper or lower periphery of vision. The system conducts this test by moving the stimulus up and/or down to the upper and/or lower periphery of the user's vision one or more times. The system may also stop the stimulus as soon as nystagmus is detected and record the angle of onset. An impaired person's eyes may exhibit nystagmus on the upper or lower peripheries of vision, or at an angle of onset prior to the periphery.” in para. [0033]).
As to Claim 20, Fichtler teaches the following:
electronically causing instructions to be audibly presented to the user to hold the camera in a left hand facing the user, position the camera at eye level, move the camera from far left of the user's face to directly in front of the user's face, and to hold the camera in a right hand facing the user, position the camera at eye level, move the camera from far right of the user's face to directly in front of the user's face, wherein at least a portion of the first plurality of images are captured while the camera is being moved (see “Calibration—This test seeks to precisely calibrate the system to the face shape, eye characteristics and eye geometry of the user. The eye movement of the user is measured during a multi-part test that may include measuring interpupillary distance, eye tracking using a moving or stationary stimulus, the measurement of pupil size, blink rate, and other biometric or eye movement characteristics. Utilizing this data, the system may make adjustments to the software or hardware of the testing apparatus to precisely align the eye tracking sensors, eye cameras, and other biometric sensors to capture optimal data. The calibration test may additionally measure reaction time, and capture eye movement data during the calibration test for later evaluation by the machine learning/artificial intelligence algorithm. In some cases, a user's facial geometry may fall outside of the parameters that are testable by the apparatus.” in para. [0029]).
Allowable Subject Matter
6. Claims 4, 5, 10, 12, 17, and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
7. The following is a statement of reasons for the indication of allowable subject matter:
As to Claim 4, neither Fichtler nor the prior art of record teaches the system of base claim 1, including the following, in combination with all other limitations of the base claim:
wherein the system is configured to detect whether the user is smoothly moving the camera, while at least a portion of the first plurality of images are captured, using acceleration data from a three axis accelerometer associated with the camera, and at least partly in response to detecting that acceleration varies by more than a threshold amount, determine that the camera is not being smoothly moved by the user and generating a first message.
As to Claim 5, neither Fichtler nor the prior art of record teaches the system of base claim 1, including the following, in combination with all other limitations of the base claim:
wherein using the first intoxication indicator, the second intoxication indicator, and the third intoxication indicator, in determining whether the user has the capacity to consent to the first act, further comprises using an analysis of a voice recording from the user in determining whether the user has the capacity to consent to the first act.
As to Claim 10, neither Fichtler nor the prior art of record teaches the computer implemented method of base claim 7, including the following, in combination with all other limitations of the base claim:
analyzing a voice recording of the user; and
wherein using the first intoxication indicator in determining whether the user has the capacity to consent to the first act, further comprises using the analysis of the voice recording of the user in determining whether the user has the capacity to consent to the first act.
As to Claim 12, neither Fichtler nor the prior art of record teaches the computer implemented method of base claim 7, including the following, in combination with all other limitations of the base claim:
detecting whether the user is smoothly moving the camera while at least a portion of the first plurality of images is captured using acceleration data from a three axis accelerometer associated with the camera; and
at least partly in response to detecting that acceleration varies by more than a threshold amount:
determining that the camera is not being smoothly moved by the user; and
generate a first message.
As to Claim 17, neither Fichtler nor the prior art of record teaches the non-transitory computer readable memory of base claim 14, including the following, in combination with all other limitations of the base claim:
analyzing a voice recording of the user,
wherein using the first intoxication indicator in determining whether the user has the capacity to consent to the first act, further comprises using the analysis of the voice recording of the user in determining whether the user has the capacity to consent to the first act.
As to Claim 19, neither Fichtler nor the prior art of record teaches the non-transitory computer readable memory of base claim 14, including the following, in combination with all other limitations of the base claim:
detecting whether the user is smoothly moving the camera while at least a portion of the first plurality of images are captured using acceleration data from a three axis accelerometer associated with the camera; and
at least partly in response to detecting that acceleration varies by more than a threshold amount, determine that the camera is not being smoothly moved by the user and generate a first message.
Conclusion
8. Any inquiry concerning this communication or earlier communications from the examiner should be directed to NAVIN NATNITHITHADHA whose telephone number is (571)272-4732. The examiner can normally be reached Monday - Friday 8:00 am - 8:00 am - 4:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason M Sims can be reached at 571-272-7540. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NAVIN NATNITHITHADHA/Primary Examiner, Art Unit 3791 05/27/2026
Prosecution Insights