DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on 12/11/2025 has been entered. Claims 1, 3-9, and 11-20 remain pending the application.
Response to Arguments
Applicant's arguments filed on 12/11/2025 have been fully considered but they are not persuasive.
Applicant argues on pages 10-13 that Woods does not disclose the newly added limitations to the claims related to correlating eye movements to pilot schedule and specific types of remedial action. This argument is moot in view of the new grounds of rejection necessitated by amendment which relies on newly cited portions of Woods to disclose these limitations in the claims. Accordingly, this argument is not persuasive.
Applicant argues on page 14 that the double patenting rejection is obviated by the amendments to the independent claims. This argument is moot in view of the new grounds of rejection necessitated by amendment which relies on newly cited portions of Woods to disclose these limitations in the claims. Accordingly, this argument is not persuasive.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 3-9, and 11-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 1, 8, and 15, the claims recite the limitation “the identified location of the pilot's gaze”. There is insufficient antecedent basis for this limitation in the claim. No identified location of the pilot’s gaze has previously been set forth. For examination purposes, this limitation will be interpreted as referring to “an identified location of the pilot’s gaze”.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 3-9, and 11-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-15 of U.S. Patent No. US12124625 in view of Flaherty-Woods et al (US10376198, hereafter Woods).
Application 18611304
US12124625
1. A system comprising:
at least one eye tracking camera; and
at least one processor in data communication with the at least one eye tracking camera and a memory storing processor executable code to configure the at least one processor to:
12. A simulator comprising: at least one eye tracking camera; […]at least one processor in data communication with a memory storing processor executable code; and wherein the processor executable code configures the at least one processor to:
receive an image stream from the at least one eye tracking camera;
receive an image stream from the at least one eye tracking camera;
identify involuntary eye movements from the image stream; and
identify gaze and pupil dynamics from the image stream;
determine if the involuntary eye movements fail to exceed a minimum threshold for a period of time; and
determine when a duration of gaze is within a threshold of acceptable attention for the current task with reference to the physiological data.
continuously log the identified involuntary eye movements and correlate the involuntary eye movements to an individual duty schedule of a pilot;
N/A
initiate a remedial action comprising attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound, and displaying warning messages at the identified location of the pilot's gaze.
N/A
3. The system of claim 1, wherein the remedial action comprises specific instrument cues to redirect the pilot's attention.
N/A
4. The system of claim 1, wherein:
the at least one processor is further configured to receive a task or user specific profile of involuntary eye movements; and
the user specific profile defines the minimum threshold.
13. The simulator of claim 12, wherein:
the processor executable code further configures the at least one processor to receive a task or user specific profile of gaze, pupil dynamics, and physiological data; and
determining when the duration is within a threshold of acceptable attention includes reference to the task or user specific profile.
5. The system of claim 4, further comprising a data storage element in data communication with the at least one processor, wherein the at least one processor is further configured to: continuously store the identified involuntary eye movements; and
refine the user specific profile.
14. The simulator of claim 12, further comprising a data storage element in data communication with the at least one processor, wherein the processor executable code further configures the at least one processor to continuously store the gaze and pupil dynamics in the data storage element.
6. The system of claim 5, wherein the at least one processor is further configured to: record system and task data contemporaneously with the identified involuntary eye movements; and
correlate the system and task data with the identified involuntary eye movements.
12. […] correlate the gaze and pupil dynamics with a current task;
7. The system of claim 1, wherein the involuntary eye movements comprise eye lid movement and position.
N/A
8. A method comprising:
receiving an image stream from at least one eye tracking camera;
6. A method comprising:
receiving an image stream from at least one eye tracking camera;
identifying involuntary eye movements from the image stream;
identifying gaze and pupil dynamics from the image stream;
determining if the involuntary eye movements fail to exceed a minimum threshold for a period of time; and
determining when a duration of gaze is within a threshold of acceptable attention for the current task with reference to the physiological data.
continuously log the identified involuntary eye movements and correlate the involuntary eye movements to an individual duty schedule of a pilot;
N/A
initiate a remedial action comprising attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound, and displaying warning messages at the identified location of the pilot's gaze.
N/A
9. The method of claim 8, wherein the remedial action comprises rendering specific instrument cues to redirect the pilot's attention.
N/A
11. The method of claim 8, further comprising receiving a task or user specific profile of involuntary eye movements, wherein the user specific profile defines the minimum threshold.
7. The method of claim 6, further comprising receiving a task or user specific profile of gaze, pupil dynamics, and physiological data, wherein determining when the duration of gaze is within the threshold includes reference to the task or user specific profile.
12. The method of claim 11, further comprising:
continuously storing the identified involuntary eye movements; and
8. The method of claim 6, further comprising continuously storing the gaze and pupil dynamics in a data storage element.
refining the user specific profile.
N/A
13. The method of claim 12, further comprising:
recording system and task data contemporaneously with the identified involuntary eye movements;
correlating the system and task data with the identified involuntary eye movements; and
6. […] correlating the gaze and pupil dynamics with a current task
analyzing the correlated system and task data and identified involuntary eye movements with respect to crew rest, crew sleep rhythms, and flight schedules to refine the minimum threshold.
N/A
14. The method of claim 8, wherein the involuntary eye movements comprise eye lid movement and position.
N/A
15. A system for use on an aircraft comprising:
at least one eye tracking camera; and
12. A simulator comprising:
at least one eye tracking camera;
at least one processor in data communication with the at least one eye tracking camera and a memory storing processor executable code to configure the at least one processor to:
at least one processor in data communication with a memory storing processor executable code; and
wherein the processor executable code configures the at least one processor to:
receive an image stream of a pilot from the at least one eye tracking camera;
receive an image stream from the at least one eye tracking camera;
identify involuntary eye movements from the image stream;
identify gaze and pupil dynamics from the image stream;
determine if the involuntary eye movements fail to exceed a minimum threshold for a period of time; and
determine when a duration of gaze is within a threshold of acceptable attention for the current task with reference to the physiological data.
continuously log the identified involuntary eye movements and correlate the involuntary eye movements to an individual duty schedule of a pilot;
N/A
initiate a remedial action comprising attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound, and displaying warning messages at the identified location of the pilot's gaze.
N/A
16. The system of claim 15, wherein the remedial action comprises specific instrument cues to redirect the pilot's attention.
N/A
17. The system of claim 15, wherein: the at least one processor is further configured to receive a task or user specific profile of involuntary eye movements; and
the user specific profile defines the minimum threshold.
13. The simulator of claim 12, wherein:
the processor executable code further configures the at least one processor to receive a task or user specific profile of gaze, pupil dynamics, and physiological data; and
determining when the duration is within a threshold of acceptable attention includes reference to the task or user specific profile.
18. The system of claim 17, further comprising a data storage element in data communication with the at least one processor, wherein the at least one processor is further configured to:
refine the user specific profile.
14. The simulator of claim 12, further comprising a data storage element in data communication with the at least one processor, wherein the processor executable code further configures the at least one processor to continuously store the gaze and pupil dynamics in the data storage element.
19. The system of claim 18, wherein the at least one processor is further configured to:
record system and task data contemporaneously with the identified involuntary eye movements;
correlate the system and task data with the identified involuntary eye movements;
12. […] correlate the gaze and pupil dynamics with a current task;
analyze the correlated system and task data and identified involuntary eye movements with respect to crew rest, crew sleep rhythms, and flight schedules to refine the minimum threshold.
N/A
20. The system of claim 15, wherein the involuntary eye movements comprise eye lid movement and position.
N/A
Claims 1-15 of US12124625 do not disclose continuously logging identified involuntary eye movements and correlating the involuntary eye movements to an individual duty schedule of a pilot; and initiating a remedial action comprising attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound, and displaying warning messages at an identified location of the pilot's gaze.
In an analogous pilot biometric monitoring field of endeavor Woods discloses continuously logging identified involuntary eye movements and correlating the involuntary eye movements to an individual duty schedule of a pilot (Woods, Col 3, lines 48 – Col 4, line 4; “In at least one embodiment, the processor 102 transfers the stored biometric data and other correlated system and task data to an offline storage device for later analysis and correlation to historic data and other outside factors such as crew rest, crew sleet rhythms, flight schedules, etc. Such transfer may be in real time via the wireless communication device 112. Referring to FIG. 3, a flowchart of an exemplary embodiment of the inventive concepts disclosed herein is shown. A computer system implementing embodiments of the inventive concepts disclosed herein receives 300 biometric data from one or more vision based sensors and/or physiological sensors. The biometric data is continuously logged 302 and correlated to a specific flight task or an individual duty schedule of the pilot. […] Indicia of impaired performance may comprise data indicative of fatigue or attention tunneling such as characteristic heart rate, characteristic brain wave patterns, characteristic eye movement or lack of eye movement, eye lid position and blink rate, characteristic facial parameter changes or lack of facial parameter changes, or characteristic head movement or lack of head movement.”); and
initiating a remedial action comprising attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound, and displaying warning messages at an identified location of the pilot's gaze (Woods, Col 4, lines 4-33; “When indicia of impaired performance are identified, a corrective action may be initiated 306. In at least one embodiment, the corrective action is specific to the type of impaired performance identified. […] such cues may be organized according to an idealized instrument observation pattern to facilitate the periodic observation of critical instruments. Attention capture techniques such as colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound may be used to disengage attention tunneling habits and shift the pilot's gaze toward information critical to the operational scenario. Where applicable, eye tracking may be used to determine exactly where the pilot's gaze is focused, and display warning messages at the identified location.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the claims of the instant application to include continuously logging identified involuntary eye movements and correlating the involuntary eye movements to an individual duty schedule of a pilot; and initiating a remedial action comprising attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound, and displaying warning messages at an identified location of the pilot's gaze in order to improve a pilot’s cognitive processing of information and reduce complacency as taught by Woods (Woods, Col 1, lines 5-40) therefore improving safety.
Claims 1-15 of US12124625 do not disclose wherein the remedial action comprises specific instrument cues to redirect a pilot's attention.
In an analogous pilot biometric monitoring field of endeavor Woods discloses wherein the remedial action comprises specific instrument cues to redirect a pilot's attention (Woods, Col 4, lines 21-33; “In a situation where attention tunneling is identified, specific instrument cues may be utilized to redirect the pilot's attention. In at least one embodiment, such cues may be organized according to an idealized instrument observation pattern to facilitate the periodic observation of critical instruments. Attention capture techniques such as colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound may be used to disengage attention tunneling habits and shift the pilot's gaze toward information critical to the operational scenario. Where applicable, eye tracking may be used to determine exactly where the pilot's gaze is focused, and display warning messages at the identified location.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the claims of the instant application wherein the remedial action comprises specific instrument cues to redirect a pilot's attention in order to improve a pilot’s cognitive processing of information and reduce complacency as taught by Woods (Woods, Col 1, lines 5-40) therefore improving safety.
Claims 1-15 of US12124625 do not disclose wherein the involuntary eye movements comprise eye lid movement and position.
In an analogous pilot biometric monitoring field of endeavor Woods discloses wherein the involuntary eye movements comprise eye lid movement and position (Woods, Col 3, line 63 – Col 4, line 4; “In at least one embodiment, the biometric data is analyzed 304 to identify one or more indicia of impaired performance. Indicia of impaired performance may comprise data indicative of fatigue or attention tunneling such […] characteristic eye movement or lack of eye movement, eye lid position and blink rate”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the claims of the instant application wherein the involuntary eye movements comprise eye lid movement and position in order to improve a pilot’s cognitive processing of information and reduce complacency as taught by Woods (Woods, Col 1, lines 5-40) therefore improving safety.
Claims 1-15 of US12124625 do not disclose wherein the specific instrument cues are rendered to include attention capturing techniques.
In an analogous pilot biometric monitoring field of endeavor Woods discloses wherein the specific instrument cues are rendered to include attention capturing techniques (Woods, Col 4, lines 21-33; “In a situation where attention tunneling is identified, specific instrument cues may be utilized to redirect the pilot's attention. In at least one embodiment, such cues may be organized according to an idealized instrument observation pattern to facilitate the periodic observation of critical instruments. Attention capture techniques such as colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound may be used to disengage attention tunneling habits and shift the pilot's gaze toward information critical to the operational scenario. Where applicable, eye tracking may be used to determine exactly where the pilot's gaze is focused, and display warning messages at the identified location.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the claims of the instant application wherein the specific instrument cues are rendered to include attention capturing techniques in order to reduce complacency as taught by Woods (Woods, Col 1, lines 5-40) therefore improving safety.
Claims 1-15 of US12124625 do not disclose analyzing the correlated system and task data and identified involuntary eye movements with respect to crew rest, crew sleep rhythms, and flight schedules to refine the minimum threshold.
In an analogous pilot biometric monitoring field of endeavor Woods discloses analyzing the correlated system and task data and identified involuntary eye movements with respect to crew rest, crew sleep rhythms, and flight schedules to refine the minimum threshold (Woods, Col 3, lines 48-62; “In at least one embodiment, the processor 102 transfers the stored biometric data and other correlated system and task data to an offline storage device for later analysis and correlation to historic data and other outside factors such as crew rest, crew sleet rhythms, flight schedules, etc. Such transfer may be in real time via the wireless communication device 112. Referring to FIG. 3, a flowchart of an exemplary embodiment of the inventive concepts disclosed herein is shown. A computer system implementing embodiments of the inventive concepts disclosed herein receives 300 biometric data from one or more vision based sensors and/or physiological sensors. The biometric data is continuously logged 302 and correlated to a specific flight task or an individual duty schedule of the pilot.”) (Woods, Col 4, line 59 – Col 5, line 7; “In at least one embodiment, logged biometric data is continuously or periodically analyzed 314 to establish or refine a function specific to the user, or flight crew including the user, relating active duty time, tasks, and biometric data to crew rest procedures. Neural network machine learning algorithms may be employed to refine the relationships between biometric data and indicia of impaired performance, and identify trends in airline operational deficiencies regarding pilot duty and fatigue patterns. Relevant data from other avionics systems may also be logged and correlated to the contemporaneous biometric data to provide context for the phase of flight and operational scenario. For example, data pertaining to specific aircraft or avionics tasks that are known or believed to induce high workload may be correlated to biometric data to provide system designers a means to gain insight into tasks that need to be streamlined.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the claims of the instant application to include analyzing the correlated system and task data and identified involuntary eye movements with respect to crew rest, crew sleep rhythms, and flight schedules to refine the minimum threshold in order to improve a pilot’s cognitive processing of information and reduce complacency as taught by Woods (Woods, Col 1, lines 5-40) therefore improving safety as well indicating tasks that need to be streamlined as taught by Woods (Woods, Col 3, lines 48-62).
Claims 1, 3-9, and 11-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-15 of US10376198 in view of Flaherty-Woods et al (US10376198, hereafter Woods).
Application 18611304
US10376198
1. A system comprising:
at least one eye tracking camera; and
at least one processor in data communication with the at least one eye tracking camera and a memory storing processor executable code to configure the at least one processor to:
receive an image stream from the at least one eye tracking camera; identify involuntary eye movements from the image stream; and
1. A computer apparatus comprising: at least one processor in data communication with a memory storing processor executable code; one or more biometric data recording devices in data communication with the at least one processor; wherein the processor executable code configures the at least one processor to: receive biometric data comprises an eye movement of a pilot from the one or more biometric data recording devices; continuously store the biometric data in the data storage element
a data storage element in data communication with the at least one processor; and
determine if the involuntary eye movements fail to exceed a minimum threshold for a period of time.
analyze the biometric data to identify indicia of impaired performance including cognitive overload; analyze the stored biometric data over time to identify an individual specific correlation between time on duty and biometric data indicative of impaired performance during specific flight tasks and with respect to an individual duty schedule of the pilot; and
continuously log the identified involuntary eye movements and correlate the involuntary eye movements to an individual duty schedule of a pilot;
N/A
initiate a remedial action
and initiate a corrective action comprising reducing a level of automation to provide a stimulus to the pilot.
wherein the remedial action comprises attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound and displaying warning messages at the identified location of the pilot's gaze.
N/A
3. The system of claim 1, wherein the remedial action comprises specific instrument cues to redirect the pilot's attention.
N/A
4. The system of claim 1, wherein:
the at least one processor is further configured to receive a task or user specific profile of involuntary eye movements; and
the user specific profile defines the minimum threshold.
2. The computer apparatus of claim 1, wherein:
the processor executable code further configures the at least one processor to receive a biometric profile corresponding to desirable biometric data for a current task; and
analyzing the biometric data to identify indicia of impaired performance comprises comparing the biometric data to the biometric profile to identify discrepancies characteristic of cognitive overload.
3. The computer apparatus of claim 2, wherein:
the one or more biometric data recording devices comprises at least one vision based sensors; and
comparing the biometric data to the biometric profile comprises comparing an eye movement pattern of a pilot to a stored eye movement pattern of an expert pilot.
5. The system of claim 4, further comprising a data storage element in data communication with the at least one processor, wherein the at least one processor is further configured to: refine the user specific profile.
N/A
6. The system of claim 5, wherein the at least one processor is further configured to: record system and task data contemporaneously with the identified involuntary eye movements; and
correlate the system and task data with the identified involuntary eye movements.
1. […] analyze the stored biometric data over time to identify an individual specific correlation between time on duty and biometric data indicative of impaired performance during specific flight tasks and with respect to an individual duty schedule of the pilot;
7. The system of claim 1, wherein the involuntary eye movements comprise eye lid movement and position.
N/A
8. A method comprising:
receiving an image stream from at least one eye tracking camera; identifying involuntary eye movements from the image stream; determining if the involuntary eye movements fail to exceed a minimum threshold for a period of time; and
7. A method comprising:
receiving biometric data from one or more biometric data recording devices in an aircraft cockpit, including at least one vision based sensors; […]receiving a biometric profile corresponding to desirable biometric data for a current task;
analyzing the biometric data to identify indicia of impaired performance including cognitive overload by comparing the biometric data to the biometric profile to identify discrepancies in a pilot's actual eye movement as compared to a stored eye movement pattern in the biometric profile;
analyzing the stored biometric data over time to identify an individual specific correlation between time on duty and biometric data indicative of impaired performance during specific flight tasks and with respect to an individual duty schedule of the pilot; and
continuously log the identified involuntary eye movements and correlate the involuntary eye movements to an individual duty schedule of a pilot;
N/A
initiate a remedial action
and initiate a corrective action comprising reducing a level of automation to provide a stimulus to the pilot.
wherein the remedial action comprises attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound and displaying warning messages at the identified location of the pilot's gaze.
N/A
9. The method of claim 8, wherein the remedial action comprises rendering specific instrument cues to redirect the pilot's attention.
N/A
11. The method of claim 8, further comprising receiving a task or user specific profile of involuntary eye movements, wherein the user specific profile defines the minimum threshold.
7. […]analyzing the biometric data to identify indicia of impaired performance including cognitive overload by comparing the biometric data to the biometric profile to identify discrepancies in a pilot's actual eye movement as compared to a stored eye movement pattern in the biometric profile;
8. The method of claim 7, wherein the biometric profile comprises an eye movement pattern derived via the analysis of the stored biometric data.
12. The method of claim 11, further comprising:
continuously storing the identified involuntary eye movements; and
7. […] continuously storing the biometric data;
refining the user specific profile.
N/A
13. The method of claim 12, further comprising:
recording system and task data contemporaneously with the identified involuntary eye movements;
correlating the system and task data with the identified involuntary eye movements; and
7. […]analyzing the stored biometric data over time to identify an individual specific correlation between time on duty and biometric data indicative of impaired performance during specific flight tasks and with respect to an individual duty schedule of the pilot
analyzing the correlated system and task data and identified involuntary eye movements with respect to crew rest, crew sleep rhythms, and flight schedules to refine the minimum threshold.
9. The method of claim 7, further comprising:
correlating the biometric data to historic data of at least one of crew rest, crew sleep rhythms, and flight schedules.
14. The method of claim 8, wherein the involuntary eye movements comprise eye lid movement and position.
N/A
15. A system for use on an aircraft comprising:
at least one eye tracking camera; and at least one processor in data communication with the at least one eye tracking camera and a memory storing processor executable code to configure the at least one processor to: receive an image stream of a pilot from the at least one eye tracking camera; identify involuntary eye movements from the image stream; determine if the involuntary eye movements fail to exceed a minimum threshold for a period of time; and
10. An aircraft comprising: computer system comprising: at least one processor in data communication with a memory storing processor executable code; a data storage element in data communication with the at least one processor; and one or more biometric data recording devices in data communication with the at least one processor; wherein the processor executable code configures the at least one processor to: receive biometric data comprises an eye movement of a pilot from the one or more biometric data recording devices; continuously store the biometric data in the data storage element; analyze the biometric data to identify indicia of impaired performance; analyze the stored biometric data over time to identify an individual specific correlation between time on duty and biometric data indicative of impaired performance during specific flight tasks and with respect to an individual duty schedule of the pilot;
continuously log the identified involuntary eye movements and correlate the involuntary eye movements to an individual duty schedule of a pilot;
N/A
initiate a remedial action
and initiate a corrective action comprising reducing a level of automation to provide a stimulus to the pilot.
wherein the remedial action comprises attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound and displaying warning messages at the identified location of the pilot's gaze.
N/A
16. The system of claim 15, wherein the remedial action comprises specific instrument cues to redirect the pilot's attention.
N/A
17. The system of claim 15, wherein:
the at least one processor is further configured to receive a task or user specific profile of involuntary eye movements; and
the user specific profile defines the minimum threshold.
11. The aircraft of claim 10, wherein:
the processor executable code further configures the at least one processor to receive a biometric profile corresponding to desirable biometric data for a current task; and
analyzing the biometric data to identify indicia of impaired performance comprises comparing the biometric data to the biometric profile to identify discrepancies characteristic of pilot fatigue.
12. The aircraft of claim 11, wherein: the one or more biometric data recording devices comprises at least one vision based sensors; and comparing the biometric data to the biometric profile comprises comparing an eye movement pattern of the pilot to a stored eye movement pattern of an expert pilot.
18. The system of claim 17, further comprising a data storage element in data communication with the at least one processor, wherein the at least one processor is further configured to: refine user pilot specific profile
N/A
19. The system of claim 18, wherein the at least one processor is further configured to:
record system and task data contemporaneously with the identified involuntary eye movements;
correlate the system and task data with the identified involuntary eye movements;
10. […]continuously store the biometric data in the data storage element;
analyze the correlated system and task data and identified involuntary eye movements with respect to crew rest, crew sleep rhythms, and flight schedules to refine the minimum threshold.
10. […] analyze the stored biometric data over time to identify an individual specific correlation between time on duty and biometric data indicative of impaired performance during specific flight tasks and with respect to an individual duty schedule of the pilot;
20. The system of claim 15, wherein the involuntary eye movements comprise eye lid movement and position.
N/A
Claims 1-15 of US12124625 do not disclose continuously logging identified involuntary eye movements and correlating the involuntary eye movements to an individual duty schedule of a pilot; and initiating a remedial action comprising attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound, and displaying warning messages at an identified location of the pilot's gaze.
In an analogous pilot biometric monitoring field of endeavor Woods discloses continuously logging identified involuntary eye movements and correlating the involuntary eye movements to an individual duty schedule of a pilot (Woods, Col 3, lines 48 – Col 4, line 4; “In at least one embodiment, the processor 102 transfers the stored biometric data and other correlated system and task data to an offline storage device for later analysis and correlation to historic data and other outside factors such as crew rest, crew sleet rhythms, flight schedules, etc. Such transfer may be in real time via the wireless communication device 112. Referring to FIG. 3, a flowchart of an exemplary embodiment of the inventive concepts disclosed herein is shown. A computer system implementing embodiments of the inventive concepts disclosed herein receives 300 biometric data from one or more vision based sensors and/or physiological sensors. The biometric data is continuously logged 302 and correlated to a specific flight task or an individual duty schedule of the pilot. […] Indicia of impaired performance may comprise data indicative of fatigue or attention tunneling such as characteristic heart rate, characteristic brain wave patterns, characteristic eye movement or lack of eye movement, eye lid position and blink rate, characteristic facial parameter changes or lack of facial parameter changes, or characteristic head movement or lack of head movement.”); and
initiating a remedial action comprising attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound, and displaying warning messages at an identified location of the pilot's gaze (Woods, Col 4, lines 4-33; “When indicia of impaired performance are identified, a corrective action may be initiated 306. In at least one embodiment, the corrective action is specific to the type of impaired performance identified. […] such cues may be organized according to an idealized instrument observation pattern to facilitate the periodic observation of critical instruments. Attention capture techniques such as colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound may be used to disengage attention tunneling habits and shift the pilot's gaze toward information critical to the operational scenario. Where applicable, eye tracking may be used to determine exactly where the pilot's gaze is focused, and display warning messages at the identified location.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the claims of the instant application to include continuously logging identified involuntary eye movements and correlating the involuntary eye movements to an individual duty schedule of a pilot; and initiating a remedial action comprising attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound, and displaying warning messages at an identified location of the pilot's gaze in order to improve a pilot’s cognitive processing of information and reduce complacency as taught by Woods (Woods, Col 1, lines 5-40) therefore improving safety.
Claims 1-15 of US10376198 do not disclose wherein the remedial action comprises specific instrument cues to redirect a pilot's attention.
In an analogous pilot biometric monitoring field of endeavor Woods discloses wherein the remedial action comprises specific instrument cues to redirect a pilot's attention (Woods, Col 4, lines 21-33; “In a situation where attention tunneling is identified, specific instrument cues may be utilized to redirect the pilot's attention. In at least one embodiment, such cues may be organized according to an idealized instrument observation pattern to facilitate the periodic observation of critical instruments. Attention capture techniques such as colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound may be used to disengage attention tunneling habits and shift the pilot's gaze toward information critical to the operational scenario. Where applicable, eye tracking may be used to determine exactly where the pilot's gaze is focused, and display warning messages at the identified location.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the claims of the instant application wherein the remedial action comprises specific instrument cues to redirect a pilot's attention in order to improve a pilot’s cognitive processing of information and reduce complacency as taught by Woods (Woods, Col 1, lines 5-40) therefore improving safety.
Claims 1-15 of US10376198 do not disclose refining the user profile.
In an analogous pilot biometric monitoring field of endeavor Woods refining the user profile (Woods, Col 1, lines 49-53; “In a further aspect, biometric data is continuously logged and correlated with data from other avionics systems to refine formulas relating biometric data to states of alertness and crew rest procedures.) (Woods, Col 4, line 59 – Col 5, line 7; “In at least one embodiment, logged biometric data is continuously or periodically analyzed 314 to establish or refine a function specific to the user, or flight crew including the user, relating active duty time, tasks, and biometric data to crew rest procedures. Neural network machine learning algorithms may be employed to refine the relationships between biometric data and indicia of impaired performance, and identify trends in airline operational deficiencies regarding pilot duty and fatigue patterns. Relevant data from other avionics systems may also be logged and correlated to the contemporaneous biometric data to provide context for the phase of flight and operational scenario. For example, data pertaining to specific aircraft or avionics tasks that are known or believed to induce high workload may be correlated to biometric data to provide system designers a means to gain insight into tasks that need to be streamlined.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the claims of the instant application to include refining the user profile in order to improve a pilot’s cognitive processing of information and reduce complacency more effectively as taught by Woods (Woods, Col 1, lines 5-40) therefore improving safety.
Claims 1-15 of US12124625 do not disclose wherein the involuntary eye movements comprise eye lid movement and position.
In an analogous pilot biometric monitoring field of endeavor Woods discloses wherein the involuntary eye movements comprise eye lid movement and position (Woods, Col 3, line 63 – Col 4, line 4; “In at least one embodiment, the biometric data is analyzed 304 to identify one or more indicia of impaired performance. Indicia of impaired performance may comprise data indicative of fatigue or attention tunneling such […] characteristic eye movement or lack of eye movement, eye lid position and blink rate”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the claims of the instant application wherein the involuntary eye movements comprise eye lid movement and position in order to improve a pilot’s cognitive processing of information and reduce complacency as taught by Woods (Woods, Col 1, lines 5-40) therefore improving safety.
Claims 1-15 of US12124625 do not disclose wherein the specific instrument cues are rendered to include attention capturing techniques.
In an analogous pilot biometric monitoring field of endeavor Woods discloses wherein the specific instrument cues are rendered to include attention capturing techniques (Woods, Col 4, lines 21-33; “In a situation where attention tunneling is identified, specific instrument cues may be utilized to redirect the pilot's attention. In at least one embodiment, such cues may be organized according to an idealized instrument observation pattern to facilitate the periodic observation of critical instruments. Attention capture techniques such as colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound may be used to disengage attention tunneling habits and shift the pilot's gaze toward information critical to the operational scenario. Where applicable, eye tracking may be used to determine exactly where the pilot's gaze is focused, and display warning messages at the identified location.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the claims of the instant application wherein the specific instrument cues are rendered to include attention capturing techniques in order to reduce complacency as taught by Woods (Woods, Col 1, lines 5-40) therefore improving safety.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 3-9, and 11-20 are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Flaherty-Woods et al (US10376198, hereafter Woods).
Regarding claim 1, Woods discloses in Figures 1-3 a system (system 100) (Woods, Col 3, lines 7-19; “Referring to FIG. 1 and FIG. 2, a block diagram of a system 100 suitable for implementing embodiments of the incentive concepts disclosed herein, and a block environmental representation of an aircraft 200 including such system 100 are shown. The system 100 includes a processor 102, memory 104 in data communication with the processor 102 for storing processor executable code, a data storage device 106 in data communication with the processor 102 for storing biometric data and task or individual specific biometric profiles corresponding to idealized or otherwise acceptable biometric data that should be generated during a specific task, and one or more biometric data gathering devices 108, 110 for receiving biometric data about a pilot”) comprising:
at least one eye tracking camera (vision based sensor 108) (Woods, Col 3, lines 20-29; “In at least one embodiment, the one or more biometric data gathering devices 108, 110 includes one or more vision based sensors 108; for example, a pilot facing camera or eye tracking sensor to record head and facial parameters, eye movement/gaze of the pilot and eye lid position.”); and
at least one processor (processor 102) in data communication with the at least one eye tracking camera and a memory (memory 104) storing processor executable code to configure the at least one processor to: receive an image stream from the at least one eye tracking camera; identify involuntary eye movements (Woods, Col 4, lines 41-54; “Such biometric reference points may include recorded biometric data of experienced pilots during similar tasks or under similar situations; including recorded eye movement patterns and workload characteristics.”) from the image stream (Woods, Col 3, lines 37-47; “In at least one embodiment, the processor executable code configures the processor 102 to receive biometric data from the one or more vision based sensors 108 and/or physiological sensors 110, and continuously log the biometric data in the data storage element 106. The processor executable code configures the processor 102 to analyze the biometric data to identify one or more indicia of impaired performance, potentially in comparison to stored biometric profiles. When indicia of impaired performance are identified, the processor executable code configures the processor 102 to initiate a corrective action as more fully described herein.”); and
determine if the involuntary eye movements fail to exceed a minimum threshold for a period of time (Woods, Col 3, line 63 – Col 4, line 4; “In at least one embodiment, the biometric data is analyzed 304 to identify one or more indicia of impaired performance. Indicia of impaired performance may comprise data indicative of fatigue or attention tunneling such […] characteristic eye movement or lack of eye movement, eye lid position and blink rate”) (Woods, Col 4, Para 55-58; “The task or individual specific biometric profile is compared 310 to the biometric data to identify 312 discrepancies indicative of impaired performance, and a corrective action is initiated 306.”) (Woods, Col 1, lines 43-60; “In one aspect, embodiments of the inventive concepts disclosed herein are directed to an on aircraft computer system that records biometric data and analyzes such data for indicia of impaired performance, such as pilot fatigue, attention tunneling, or cognitive overload. Appropriate corrective action may be taken based on the type of impaired performance identified. In a further aspect, biometric data is continuously logged and correlated with data from other avionics systems to refine formulas relating biometric data to states of alertness and crew rest procedures.”);
continuously log the identified involuntary eye movements and correlating the involuntary eye movements to an individual duty schedule of a pilot (Woods, Col 3, lines 48 – Col 4, line 4; “In at least one embodiment, the processor 102 transfers the stored biometric data and other correlated system and task data to an offline storage device for later analysis and correlation to historic data and other outside factors such as crew rest, crew sleet rhythms, flight schedules, etc. Such transfer may be in real time via the wireless communication device 112. Referring to FIG. 3, a flowchart of an exemplary embodiment of the inventive concepts disclosed herein is shown. A computer system implementing embodiments of the inventive concepts disclosed herein receives 300 biometric data from one or more vision based sensors and/or physiological sensors. The biometric data is continuously logged 302 and correlated to a specific flight task or an individual duty schedule of the pilot. […] Indicia of impaired performance may comprise data indicative of fatigue or attention tunneling such as characteristic heart rate, characteristic brain wave patterns, characteristic eye movement or lack of eye movement, eye lid position and blink rate, characteristic facial parameter changes or lack of facial parameter changes, or characteristic head movement or lack of head movement.”); and
initiate a remedial action comprising attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound, and displaying warning messages at an identified location of the pilot's gaze (Woods, Col 4, lines 4-33; “When indicia of impaired performance are identified, a corrective action may be initiated 306. In at least one embodiment, the corrective action is specific to the type of impaired performance identified. […] such cues may be organized according to an idealized instrument observation pattern to facilitate the periodic observation of critical instruments. Attention capture techniques such as colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound may be used to disengage attention tunneling habits and shift the pilot's gaze toward information critical to the operational scenario. Where applicable, eye tracking may be used to determine exactly where the pilot's gaze is focused, and display warning messages at the identified location.”).
Regarding claim 3, Woods discloses all of the limitations of claim 1 as discussed above.
Woods further discloses wherein the remedial action comprises specific instrument cues to redirect the pilot's attention (Woods, Col 4, lines 21-33; “In a situation where attention tunneling is identified, specific instrument cues may be utilized to redirect the pilot's attention. In at least one embodiment, such cues may be organized according to an idealized instrument observation pattern to facilitate the periodic observation of critical instruments. Attention capture techniques such as colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound may be used to disengage attention tunneling habits and shift the pilot's gaze toward information critical to the operational scenario. Where applicable, eye tracking may be used to determine exactly where the pilot's gaze is focused, and display warning messages at the identified location.”).
Regarding claim 4, Woods discloses all of the limitations of claim 1 as discussed above.
Woods further discloses wherein: the at least one processor is further configured to receive a task or user specific profile of involuntary eye movements; and the user specific profile defines the minimum threshold (Woods, Col 4, line 41 – Col 5, line 7; “In at least one embodiment, the computer system also receives 308 a task or individual specific biometric profile including one more biometric reference points. Such biometric reference points may include recorded biometric data of experienced pilots during similar tasks or under similar situations; including recorded eye movement patterns and workload characteristics. Alternatively, or in addition, one or more biometric reference points may include biometric data of the specific pilot, recorded under ideal conditions. Further, the one more biometric reference points may comprise some combination of biometric data of experienced pilots, modified according to actual previously recoded biometric data of the specific pilot to produce a projected biometric profile for the specific pilot during a specific task. The task or individual specific biometric profile is compared 310 to the biometric data to identify 312 discrepancies indicative of impaired performance, and a corrective action is initiated 306. In at least one embodiment, logged biometric data is continuously or periodically analyzed 314 to establish or refine a function specific to the user, or flight crew including the user, relating active duty time, tasks, and biometric data to crew rest procedures. Neural network machine learning algorithms may be employed to refine the relationships between biometric data and indicia of impaired performance, and identify trends in airline operational deficiencies regarding pilot duty and fatigue patterns. Relevant data from other avionics systems may also be logged and correlated to the contemporaneous biometric data to provide context for the phase of flight and operational scenario. For example, data pertaining to specific aircraft or avionics tasks that are known or believed to induce high workload may be correlated to biometric data to provide system designers a means to gain insight into tasks that need to be streamlined.”).
Regarding claim 5, Woods discloses all of the limitations of claim 4 as discussed above.
Woods further discloses a data storage element in data communication with the at least one processor, wherein the at least one processor is further configured to: refine the user specific profile (Woods, Col 4, line 59 – Col 5, line 7; “In at least one embodiment, logged biometric data is continuously or periodically analyzed 314 to establish or refine a function specific to the user, or flight crew including the user, relating active duty time, tasks, and biometric data to crew rest procedures. Neural network machine learning algorithms may be employed to refine the relationships between biometric data and indicia of impaired performance, and identify trends in airline operational deficiencies regarding pilot duty and fatigue patterns. Relevant data from other avionics systems may also be logged and correlated to the contemporaneous biometric data to provide context for the phase of flight and operational scenario. For example, data pertaining to specific aircraft or avionics tasks that are known or believed to induce high workload may be correlated to biometric data to provide system designers a means to gain insight into tasks that need to be streamlined.”) (Woods, Col 1, lines 49-53; “In a further aspect, biometric data is continuously logged and correlated with data from other avionics systems to refine formulas relating biometric data to states of alertness and crew rest procedures.).
Regarding claim 6, Woods discloses all of the limitations of claim 5 as discussed above.
Woods further discloses wherein the at least one processor is further configured to: record system and task data contemporaneously with the identified involuntary eye movements; and correlate the system and task data with the identified involuntary eye movements (Woods, Col 3, lines 48-62; “In at least one embodiment, the processor 102 transfers the stored biometric data and other correlated system and task data to an offline storage device for later analysis and correlation to historic data and other outside factors such as crew rest, crew sleet rhythms, flight schedules, etc. Such transfer may be in real time via the wireless communication device 112. Referring to FIG. 3, a flowchart of an exemplary embodiment of the inventive concepts disclosed herein is shown. A computer system implementing embodiments of the inventive concepts disclosed herein receives 300 biometric data from one or more vision based sensors and/or physiological sensors. The biometric data is continuously logged 302 and correlated to a specific flight task or an individual duty schedule of the pilot.”) (Woods, Col 4, line 59 – Col 5, line 7; “In at least one embodiment, logged biometric data is continuously or periodically analyzed 314 to establish or refine a function specific to the user, or flight crew including the user, relating active duty time, tasks, and biometric data to crew rest procedures. Neural network machine learning algorithms may be employed to refine the relationships between biometric data and indicia of impaired performance, and identify trends in airline operational deficiencies regarding pilot duty and fatigue patterns. Relevant data from other avionics systems may also be logged and correlated to the contemporaneous biometric data to provide context for the phase of flight and operational scenario. For example, data pertaining to specific aircraft or avionics tasks that are known or believed to induce high workload may be correlated to biometric data to provide system designers a means to gain insight into tasks that need to be streamlined.”).
Regarding claim 7, Woods discloses all of the limitations of claim 1 as discussed above.
Woods further discloses wherein the involuntary eye movements comprise eye lid movement and position (Woods, Col 3, line 63 – Col 4, line 4; “In at least one embodiment, the biometric data is analyzed 304 to identify one or more indicia of impaired performance. Indicia of impaired performance may comprise data indicative of fatigue or attention tunneling such […] characteristic eye movement or lack of eye movement, eye lid position and blink rate”).
Regarding claim 8, Woods discloses in Figures 1-3 a method (Woods, Col 3, lines 7-19; “Referring to FIG. 1 and FIG. 2, a block diagram of a system 100 suitable for implementing embodiments of the incentive concepts disclosed herein, and a block environmental representation of an aircraft 200 including such system 100 are shown. The system 100 includes a processor 102, memory 104 in data communication with the processor 102 for storing processor executable code, a data storage device 106 in data communication with the processor 102 for storing biometric data and task or individual specific biometric profiles corresponding to idealized or otherwise acceptable biometric data that should be generated during a specific task, and one or more biometric data gathering devices 108, 110 for receiving biometric data about a pilot”) comprising:
receiving an image stream from at least one eye tracking camera (vision based sensor 108) (Woods, Col 3, lines 20-29; “In at least one embodiment, the one or more biometric data gathering devices 108, 110 includes one or more vision based sensors 108; for example, a pilot facing camera or eye tracking sensor to record head and facial parameters, eye movement/gaze of the pilot and eye lid position.”) (Woods, Col 4, lines 41-54; “Such biometric reference points may include recorded biometric data of experienced pilots during similar tasks or under similar situations; including recorded eye movement patterns and workload characteristics.”);
identifying involuntary eye movements from the image stream (Woods, Col 3, lines 37-47; “In at least one embodiment, the processor executable code configures the processor 102 to receive biometric data from the one or more vision based sensors 108 and/or physiological sensors 110, and continuously log the biometric data in the data storage element 106. The processor executable code configures the processor 102 to analyze the biometric data to identify one or more indicia of impaired performance, potentially in comparison to stored biometric profiles. When indicia of impaired performance are identified, the processor executable code configures the processor 102 to initiate a corrective action as more fully described herein.”);
determining if the involuntary eye movements fail to exceed a minimum threshold for a period of time (Woods, Col 3, line 63 – Col 4, line 4; “In at least one embodiment, the biometric data is analyzed 304 to identify one or more indicia of impaired performance. Indicia of impaired performance may comprise data indicative of fatigue or attention tunneling such […] characteristic eye movement or lack of eye movement, eye lid position and blink rate”) (Woods, Col 4, Para 55-58; “The task or individual specific biometric profile is compared 310 to the biometric data to identify 312 discrepancies indicative of impaired performance, and a corrective action is initiated 306.”);
continuously logging the identified involuntary eye movements and correlating the involuntary eye movements to an individual duty schedule of a pilot (Woods, Col 3, lines 48 – Col 4, line 4; “In at least one embodiment, the processor 102 transfers the stored biometric data and other correlated system and task data to an offline storage device for later analysis and correlation to historic data and other outside factors such as crew rest, crew sleet rhythms, flight schedules, etc. Such transfer may be in real time via the wireless communication device 112. Referring to FIG. 3, a flowchart of an exemplary embodiment of the inventive concepts disclosed herein is shown. A computer system implementing embodiments of the inventive concepts disclosed herein receives 300 biometric data from one or more vision based sensors and/or physiological sensors. The biometric data is continuously logged 302 and correlated to a specific flight task or an individual duty schedule of the pilot. […] Indicia of impaired performance may comprise data indicative of fatigue or attention tunneling such as characteristic heart rate, characteristic brain wave patterns, characteristic eye movement or lack of eye movement, eye lid position and blink rate, characteristic facial parameter changes or lack of facial parameter changes, or characteristic head movement or lack of head movement.”); and
initiating a remedial action comprising attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound, and displaying warning messages at an identified location of the pilot's gaze (Woods, Col 4, lines 4-33; “When indicia of impaired performance are identified, a corrective action may be initiated 306. In at least one embodiment, the corrective action is specific to the type of impaired performance identified. […] such cues may be organized according to an idealized instrument observation pattern to facilitate the periodic observation of critical instruments. Attention capture techniques such as colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound may be used to disengage attention tunneling habits and shift the pilot's gaze toward information critical to the operational scenario. Where applicable, eye tracking may be used to determine exactly where the pilot's gaze is focused, and display warning messages at the identified location.”).
Regarding claim 9, Woods discloses all of the limitations of claim 8 as discussed above.
Woods further discloses wherein the remedial action comprises specific instrument cues to redirect the pilot's attention (Woods, Col 4, lines 21-33; “In a situation where attention tunneling is identified, specific instrument cues may be utilized to redirect the pilot's attention. In at least one embodiment, such cues may be organized according to an idealized instrument observation pattern to facilitate the periodic observation of critical instruments. Attention capture techniques such as colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound may be used to disengage attention tunneling habits and shift the pilot's gaze toward information critical to the operational scenario. Where applicable, eye tracking may be used to determine exactly where the pilot's gaze is focused, and display warning messages at the identified location.”).
Regarding claim 11, Woods discloses all of the limitations of claim 8 as discussed above.
Woods further discloses receiving a task or user specific profile of involuntary eye movements; and the user specific profile defines the minimum threshold (Woods, Col 4, line 41 – Col 5, line 7; “In at least one embodiment, the computer system also receives 308 a task or individual specific biometric profile including one more biometric reference points. Such biometric reference points may include recorded biometric data of experienced pilots during similar tasks or under similar situations; including recorded eye movement patterns and workload characteristics. Alternatively, or in addition, one or more biometric reference points may include biometric data of the specific pilot, recorded under ideal conditions. Further, the one more biometric reference points may comprise some combination of biometric data of experienced pilots, modified according to actual previously recoded biometric data of the specific pilot to produce a projected biometric profile for the specific pilot during a specific task. The task or individual specific biometric profile is compared 310 to the biometric data to identify 312 discrepancies indicative of impaired performance, and a corrective action is initiated 306. In at least one embodiment, logged biometric data is continuously or periodically analyzed 314 to establish or refine a function specific to the user, or flight crew including the user, relating active duty time, tasks, and biometric data to crew rest procedures. Neural network machine learning algorithms may be employed to refine the relationships between biometric data and indicia of impaired performance, and identify trends in airline operational deficiencies regarding pilot duty and fatigue patterns. Relevant data from other avionics systems may also be logged and correlated to the contemporaneous biometric data to provide context for the phase of flight and operational scenario. For example, data pertaining to specific aircraft or avionics tasks that are known or believed to induce high workload may be correlated to biometric data to provide system designers a means to gain insight into tasks that need to be streamlined.”).
Regarding claim 12, Woods discloses all of the limitations of claim 11 as discussed above.
Woods further discloses continuously storing the identified involuntary eye movements; and refining the user specific profile (Woods, Col 4, line 59 – Col 5, line 7; “In at least one embodiment, logged biometric data is continuously or periodically analyzed 314 to establish or refine a function specific to the user, or flight crew including the user, relating active duty time, tasks, and biometric data to crew rest procedures. Neural network machine learning algorithms may be employed to refine the relationships between biometric data and indicia of impaired performance, and identify trends in airline operational deficiencies regarding pilot duty and fatigue patterns. Relevant data from other avionics systems may also be logged and correlated to the contemporaneous biometric data to provide context for the phase of flight and operational scenario. For example, data pertaining to specific aircraft or avionics tasks that are known or believed to induce high workload may be correlated to biometric data to provide system designers a means to gain insight into tasks that need to be streamlined.”) (Woods, Col 1, lines 49-53; “In a further aspect, biometric data is continuously logged and correlated with data from other avionics systems to refine formulas relating biometric data to states of alertness and crew rest procedures.).
Regarding claim 13, Woods discloses all of the limitations of claim 12 as discussed above.
Woods further discloses recording system and task data contemporaneously with the identified involuntary eye movements; correlating the system and task data with the identified involuntary eye movements; and analyzing the correlated system and task data and identified involuntary eye movements with respect to crew rest, crew sleep rhythms, and flight schedules to refine the minimum threshold (Woods, Col 3, lines 48-62; “In at least one embodiment, the processor 102 transfers the stored biometric data and other correlated system and task data to an offline storage device for later analysis and correlation to historic data and other outside factors such as crew rest, crew sleet rhythms, flight schedules, etc. Such transfer may be in real time via the wireless communication device 112. Referring to FIG. 3, a flowchart of an exemplary embodiment of the inventive concepts disclosed herein is shown. A computer system implementing embodiments of the inventive concepts disclosed herein receives 300 biometric data from one or more vision based sensors and/or physiological sensors. The biometric data is continuously logged 302 and correlated to a specific flight task or an individual duty schedule of the pilot.”) (Woods, Col 4, line 59 – Col 5, line 7; “In at least one embodiment, logged biometric data is continuously or periodically analyzed 314 to establish or refine a function specific to the user, or flight crew including the user, relating active duty time, tasks, and biometric data to crew rest procedures. Neural network machine learning algorithms may be employed to refine the relationships between biometric data and indicia of impaired performance, and identify trends in airline operational deficiencies regarding pilot duty and fatigue patterns. Relevant data from other avionics systems may also be logged and correlated to the contemporaneous biometric data to provide context for the phase of flight and operational scenario. For example, data pertaining to specific aircraft or avionics tasks that are known or believed to induce high workload may be correlated to biometric data to provide system designers a means to gain insight into tasks that need to be streamlined.”).
Regarding claim 14, Woods discloses all of the limitations of claim 8 as discussed above.
Woods further discloses wherein the involuntary eye movements comprise eye lid movement and position (Woods, Col 3, line 63 – Col 4, line 4; “In at least one embodiment, the biometric data is analyzed 304 to identify one or more indicia of impaired performance. Indicia of impaired performance may comprise data indicative of fatigue or attention tunneling such […] characteristic eye movement or lack of eye movement, eye lid position and blink rate”).
Regarding claim 15, Woods discloses in Figures 1-3 a system for use on an aircraft (system 100) (Woods, Col 3, lines 7-19; “Referring to FIG. 1 and FIG. 2, a block diagram of a system 100 suitable for implementing embodiments of the incentive concepts disclosed herein, and a block environmental representation of an aircraft 200 including such system 100 are shown. The system 100 includes a processor 102, memory 104 in data communication with the processor 102 for storing processor executable code, a data storage device 106 in data communication with the processor 102 for storing biometric data and task or individual specific biometric profiles corresponding to idealized or otherwise acceptable biometric data that should be generated during a specific task, and one or more biometric data gathering devices 108, 110 for receiving biometric data about a pilot”) comprising:
at least one eye tracking camera (vision based sensor 108) (Woods, Col 3, lines 20-29; “In at least one embodiment, the one or more biometric data gathering devices 108, 110 includes one or more vision based sensors 108; for example, a pilot facing camera or eye tracking sensor to record head and facial parameters, eye movement/gaze of the pilot and eye lid position.”); and
at least one processor (processor 102) in data communication with the at least one eye tracking camera and a memory (memory 104) storing processor executable code to configure the at least one processor to: receive an image stream from the at least one eye tracking camera; identify involuntary eye movements (Woods, Col 4, lines 41-54; “Such biometric reference points may include recorded biometric data of experienced pilots during similar tasks or under similar situations; including recorded eye movement patterns and workload characteristics.”) from the image stream (Woods, Col 3, lines 37-47; “In at least one embodiment, the processor executable code configures the processor 102 to receive biometric data from the one or more vision based sensors 108 and/or physiological sensors 110, and continuously log the biometric data in the data storage element 106. The processor executable code configures the processor 102 to analyze the biometric data to identify one or more indicia of impaired performance, potentially in comparison to stored biometric profiles. When indicia of impaired performance are identified, the processor executable code configures the processor 102 to initiate a corrective action as more fully described herein.”);
determine if the involuntary eye movements fail to exceed a minimum threshold for a period of time (Woods, Col 3, line 63 – Col 4, line 4; “In at least one embodiment, the biometric data is analyzed 304 to identify one or more indicia of impaired performance. Indicia of impaired performance may comprise data indicative of fatigue or attention tunneling such […] characteristic eye movement or lack of eye movement, eye lid position and blink rate”) (Woods, Col 4, Para 55-58; “The task or individual specific biometric profile is compared 310 to the biometric data to identify 312 discrepancies indicative of impaired performance, and a corrective action is initiated 306.”); and
continuously log the identified involuntary eye movements and correlating the involuntary eye movements to an individual duty schedule of a pilot (Woods, Col 3, lines 48 – Col 4, line 4; “In at least one embodiment, the processor 102 transfers the stored biometric data and other correlated system and task data to an offline storage device for later analysis and correlation to historic data and other outside factors such as crew rest, crew sleet rhythms, flight schedules, etc. Such transfer may be in real time via the wireless communication device 112. Referring to FIG. 3, a flowchart of an exemplary embodiment of the inventive concepts disclosed herein is shown. A computer system implementing embodiments of the inventive concepts disclosed herein receives 300 biometric data from one or more vision based sensors and/or physiological sensors. The biometric data is continuously logged 302 and correlated to a specific flight task or an individual duty schedule of the pilot. […] Indicia of impaired performance may comprise data indicative of fatigue or attention tunneling such as characteristic heart rate, characteristic brain wave patterns, characteristic eye movement or lack of eye movement, eye lid position and blink rate, characteristic facial parameter changes or lack of facial parameter changes, or characteristic head movement or lack of head movement.”); and
initiate a remedial action comprising attention capture techniques including colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound, and displaying warning messages at an identified location of the pilot's gaze (Woods, Col 4, lines 4-33; “When indicia of impaired performance are identified, a corrective action may be initiated 306. In at least one embodiment, the corrective action is specific to the type of impaired performance identified. […] such cues may be organized according to an idealized instrument observation pattern to facilitate the periodic observation of critical instruments. Attention capture techniques such as colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound may be used to disengage attention tunneling habits and shift the pilot's gaze toward information critical to the operational scenario. Where applicable, eye tracking may be used to determine exactly where the pilot's gaze is focused, and display warning messages at the identified location.”).
Regarding claim 16, Woods discloses all of the limitations of claim 15 as discussed above.
Woods further discloses wherein the remedial action comprises specific instrument cues to redirect the pilot's attention (Woods, Col 4, lines 21-33; “In a situation where attention tunneling is identified, specific instrument cues may be utilized to redirect the pilot's attention. In at least one embodiment, such cues may be organized according to an idealized instrument observation pattern to facilitate the periodic observation of critical instruments. Attention capture techniques such as colors, symbology, blinking or flashing indicators, motion, haptic feedback, or sound may be used to disengage attention tunneling habits and shift the pilot's gaze toward information critical to the operational scenario. Where applicable, eye tracking may be used to determine exactly where the pilot's gaze is focused, and display warning messages at the identified location.”).
Regarding claim 17, Woods discloses all of the limitations of claim 15 as discussed above.
Woods further discloses wherein: the at least one processor is further configured to receive a task or user specific profile of involuntary eye movements; and the user specific profile defines the minimum threshold (Woods, Col 4, line 41 – Col 5, line 7; “In at least one embodiment, the computer system also receives 308 a task or individual specific biometric profile including one more biometric reference points. Such biometric reference points may include recorded biometric data of experienced pilots during similar tasks or under similar situations; including recorded eye movement patterns and workload characteristics. Alternatively, or in addition, one or more biometric reference points may include biometric data of the specific pilot, recorded under ideal conditions. Further, the one more biometric reference points may comprise some combination of biometric data of experienced pilots, modified according to actual previously recoded biometric data of the specific pilot to produce a projected biometric profile for the specific pilot during a specific task. The task or individual specific biometric profile is compared 310 to the biometric data to identify 312 discrepancies indicative of impaired performance, and a corrective action is initiated 306. In at least one embodiment, logged biometric data is continuously or periodically analyzed 314 to establish or refine a function specific to the user, or flight crew including the user, relating active duty time, tasks, and biometric data to crew rest procedures. Neural network machine learning algorithms may be employed to refine the relationships between biometric data and indicia of impaired performance, and identify trends in airline operational deficiencies regarding pilot duty and fatigue patterns. Relevant data from other avionics systems may also be logged and correlated to the contemporaneous biometric data to provide context for the phase of flight and operational scenario. For example, data pertaining to specific aircraft or avionics tasks that are known or believed to induce high workload may be correlated to biometric data to provide system designers a means to gain insight into tasks that need to be streamlined.”).
Regarding claim 18, Woods discloses all of the limitations of claim 17 as discussed above.
Woods further discloses a data storage element in data communication with the at least one processor, wherein the at least one processor is further configured to: refine the user specific profile (Woods, Col 4, line 59 – Col 5, line 7; “In at least one embodiment, logged biometric data is continuously or periodically analyzed 314 to establish or refine a function specific to the user, or flight crew including the user, relating active duty time, tasks, and biometric data to crew rest procedures. Neural network machine learning algorithms may be employed to refine the relationships between biometric data and indicia of impaired performance, and identify trends in airline operational deficiencies regarding pilot duty and fatigue patterns. Relevant data from other avionics systems may also be logged and correlated to the contemporaneous biometric data to provide context for the phase of flight and operational scenario. For example, data pertaining to specific aircraft or avionics tasks that are known or believed to induce high workload may be correlated to biometric data to provide system designers a means to gain insight into tasks that need to be streamlined.”) (Woods, Col 1, lines 49-53; “In a further aspect, biometric data is continuously logged and correlated with data from other avionics systems to refine formulas relating biometric data to states of alertness and crew rest procedures.).
Regarding claim 19, Woods discloses all of the limitations of claim 18 as discussed above.
Woods further discloses wherein the at least one processor is further configured to: record system and task data contemporaneously with the identified involuntary eye movements; correlate the system and task data with the identified involuntary eye movements; analyze the correlated system and task data and identified involuntary eye movements with respect to crew rest, crew sleep rhythms, and flight schedules to refine the minimum threshold (Woods, Col 3, lines 48-62; “In at least one embodiment, the processor 102 transfers the stored biometric data and other correlated system and task data to an offline storage device for later analysis and correlation to historic data and other outside factors such as crew rest, crew sleet rhythms, flight schedules, etc. Such transfer may be in real time via the wireless communication device 112. Referring to FIG. 3, a flowchart of an exemplary embodiment of the inventive concepts disclosed herein is shown. A computer system implementing embodiments of the inventive concepts disclosed herein receives 300 biometric data from one or more vision based sensors and/or physiological sensors. The biometric data is continuously logged 302 and correlated to a specific flight task or an individual duty schedule of the pilot.”) (Woods, Col 4, line 59 – Col 5, line 7; “In at least one embodiment, logged biometric data is continuously or periodically analyzed 314 to establish or refine a function specific to the user, or flight crew including the user, relating active duty time, tasks, and biometric data to crew rest procedures. Neural network machine learning algorithms may be employed to refine the relationships between biometric data and indicia of impaired performance, and identify trends in airline operational deficiencies regarding pilot duty and fatigue patterns. Relevant data from other avionics systems may also be logged and correlated to the contemporaneous biometric data to provide context for the phase of flight and operational scenario. For example, data pertaining to specific aircraft or avionics tasks that are known or believed to induce high workload may be correlated to biometric data to provide system designers a means to gain insight into tasks that need to be streamlined.”).
Regarding claim 20, Woods discloses all of the limitations of claim 15 as discussed above.
Woods further discloses wherein the involuntary eye movements comprise eye lid movement and position (Woods, Col 3, line 63 – Col 4, line 4; “In at least one embodiment, the biometric data is analyzed 304 to identify one or more indicia of impaired performance. Indicia of impaired performance may comprise data indicative of fatigue or attention tunneling such […] characteristic eye movement or lack of eye movement, eye lid position and blink rate”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to John Li whose telephone number is (313)446-4916. The examiner can normally be reached Monday to Thursday; 5:30 AM to 3:30 PM Eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at (571) 272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN D LI/Primary Examiner, Art Unit 3798
Prosecution Insights