DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
This action is in response to the application filed on October 15, 2025. Claims 1, 3, 8, and 10 are amended, claims 2, 9, and 15 are cancelled. Thus, claim 1, 3-8, 10-14, and 16-20 are pending and have been examined.
Response to Amendments
Applicant’s remarks and amendments filed October 15, 2025, have been entered.
Applicant’s amendments regarding the 35 U.S.C. 112(f) interpretations previously set forth in the Non-Final Office Action mailed on July 15, 2025, regarding claims 2, 9, and 15, are persuasive. Accordingly, the 35 U.S.C. 112(f) rejections are withdrawn in response.
Applicant’s amendments regarding the 35 U.S.C. 101 rejections previously set forth in the Non-Final Office Action mailed on July 15, 2025, regarding claims 1, 3-8, 10-14, and 16-20, are persuasive. Accordingly, the 35 U.S.C. 101 rejections are withdrawn in response.
Response to Arguments
Applicant’s arguments filed October 15, 2025, regarding the rejection(s) of claim(s) 1, 3-8, 10-14, and 16-20 have been fully and completely considered but are moot because the arguments do not apply to the new combination of the references, facilitated by Applicant’s newly submitted amendments, including new prior art—Myers et al, US 20220230522—being used in the current rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-8, 10-14, and 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Myers et al, US 20220230522 in view of Kronberg et al, US 20150216466.
Regarding claim 1, Myers teaches a computer apparatus comprising:
at least one camera configured to track a pilot's gaze, pupil dynamics, eyelid position, and hand or arm movement (see Myers, Fig. 2 and Paragraph [0111], “system 100 includes an imaging camera 106 that is positioned on or in the cockpit instrument panel 107 and oriented to capture images of at least the pilot's and/or co-pilot's face in the infrared wavelength range to identify, locate and track one or more human facial features. Camera 106 may also image the pilot's and/or co-pilot's head and body to determine head and body position and movement,” Paragraph [0133], “Example facial features include nostrils, eyelid contours, pupils, irises, mouth corners and lips. Example body features include, head size, head shape, head orientation, neck shape and position, body shape, body posture, body position, body size, arm position and leg position”);
at least one physiological sensor comprising at least one of an electroencephalograph (EEG) or functional near-infrared spectroscopy (fNIRs) (see Myers, Paragraph [0206], “In some embodiments, system 100 includes or is adapted to receive information from additional hardware that monitors biometric information of the pilot 102 and/or co-pilot 103. In these embodiments, step 1304 may include module 122 receiving biometric information of the pilot 102 and/or co-pilot 103 from a biometric reader device to augment the alertness state data obtained in step 1302 to better understand the true attention state of the pilot/co-pilot. By way of example, the biometric information may include a signal indicative of brain activity received from an electroencephalography (EEG) device connected to the pilot 102 and/or co-pilot 103”);
and at least one processor in data communication with a memory storing processor executable code (see Myers, Paragraph [0120], “Controller 112 may be implemented as any form of computer processing device or portion of a device that processes electronic data, e.g., from registers and/or memory to transform that electronic data into other electronic data that, e.g., may be stored in registers and/or memory. As illustrated in FIG. 3, controller 112 includes a microprocessor 114, executing code stored in memory 116”);
and wherein the processor executable code configures the at least one processor to: receive an image stream from the at least one camera (see Myers Paragraph [0121], “Vision processor 118 is configured to process the captured images to perform the pilot/co-pilot monitoring,” captured images is considered to be an image stream);
determine a pilot pose estimate based on at least gaze and pupil dynamics and hand or arm movement determined from the image stream (see Myers, Paragraph [0122], “Vision processor 118 is configured to process the captured images to perform the pilot/co-pilot monitoring; for example to determine a three dimensional head pose, body pose, eye closure and/or eye gaze position of the pilot 102 within the monitoring environment. To determine the eye gaze position, vision processor 118 utilizes one or more eye gaze determination algorithms. … Vision processor 118 may also perform various other functions including determining attributes of the pilot 102 or co-pilot 103 such as blink rate and tracking the pilot's head position and motion, and body position and motion to detect pilot attention, cognitive state/load, sleepiness or other issues that may interfere with the pilot safely operating the aircraft”);
receive physiological data from the at least one physiological sensor (see Myers, Paragraph [0206], “the biometric information may include a signal indicative of brain activity received from an electroencephalography (EEG) device connected to the pilot 102 and/or co-pilot 103,” the biometric information is considered to be physiological data);
correlate the image stream and physiological data with task-specific data comprising at least one of an instrument reading, an alert condition, or a task segment identifier (see Myers, Fig. 13, Fig. 14, 122 Controlled Rest Determination Module, and Paragraph [0209], “at step 1305, controller 112 issues an audio, visual or audiovisual alert to the pilot 102 and/or co-pilot 103 and/or cabin crew based on the determination in step 1304”);
correlation (see Myers, Paragraph [0202], “Where appropriate permission is sought by pilots and aircraft operators, controller 112 may also be configured to receive input indicative of a sleep history and/or a recent duty history of the pilot 102 and/or co-pilot 103 as an input to the controlled rest determination algorithm … This historical data can be used to predict a likely future attention state of the pilot 102 or co-pilot 103 in determining the suitability of a controlled rest period and duration for the pilot 102 or co-pilot 10”);
compile avionics data required to implement at least one of the future pilot actions for presentation on a cockpit display or transmission to a remote control station (see Myers, Paragraph [0221], “At step 1504, system 100 issues an alert if the non-resting pilot/co-pilot enters a predetermined alertness state such as a distracted state, low vigilance drowsy state, asleep state or incapacitated attention state. The alert is preferably issued via the instrument panel 107 or other device in the cockpit and may be visual, audio, tactile, haptic or audiovisual in nature. The alert may also be issued to cabin crew outside the cockpit via a signal transmitted to a cabin crew notification system/device”).
Myers does not expressively teach
create a probability distribution of future actions
However, Kronberg in a similar invention in the same field of endeavor teaches
create a probability distribution of future actions (see Kronberg, Paragraph [0038], “The next step in the preparatory phase involves computation of a probability mass function (pmf) of each indicator (i.e. source of information) for the defined driver states. In FIGS. 4a and b, it is illustrated the pmf for two indicators, time to line crossing (TLC) and steering wheel reversal rate (SRR), using a two state parameterization {alert, drowsy}. In practice, pre collected data of actual drowsy and alert driving maybe used to create the pmfs for each indicator. Since data will always be inherently ambiguous (to some extent) there is a desire to calculate the pmf of the driver state rather than a discrete classification. In other words, the shape of the pmfs describing the indicator values are used to calculate the driver state pmf. Thus there will be several pmfs describing the distribution for the data values for each indicator, and one pmf describing the distribution between valid driver states,” a probability mass function is the probability distribution);
The combination of Myers and Kronberg are analogous art because they are both in the same field of endeavor of monitoring operator awareness. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to compute a probability mass function of each source of information for defined driver states in the method of Kronberg in the system of Myers to determine an overall operational state probability for the driver (see Kronberg, Abstract).
Regarding claim 3, Myers in view of Kronberg further teaches the computer apparatus of claim 1,
wherein: the processor executable code further configures the at least one processor to receive a task or user specific profile of pilot pose, physiological data, and subsequent pilot actions (see Myers, Fig. 10, Paragraph [0185], “Optionally, CFRMS 700 may receive input indicative of sleep history and/or a recent duty history of pilot 102 and/or co-pilot 103 … This sleep/duty history data may be used to augment the visual alertness state determined in step 802 to generate an overall alertness state and/or used to predict a future alertness state of the pilot/co-pilot”)
and creating the probability distribution includes reference to the task or user specific profile (see Kronberg, Paragraph [0038], “The next step in the preparatory phase involves computation of a probability mass function (pmf) of each indicator (i.e. source of information) for the defined driver states” a probability mass function was considered to be a probability distribution in the rejection of claim 1, therefore a probability distribution of the reference to the task or user specific profile can be done in a similar manner).
The rationale of claim 1 has been applied herein.
Regarding claim 4, Myers in view of Kronberg further teach the computer apparatus of claim 1,
wherein the probability distribution defines a plurality of windows of probability, each associated with a discreet future action or set of future actions (see Kronberg, Fig. 4a and 4b, Paragraph [0038], “In practice, pre collected data of actual drowsy and alert driving maybe used to create the pmfs for each indicator. Since data will always be inherently ambiguous (to some extent) there is a desire to calculate the pmf of the driver state rather than a discrete classification. In other words, the shape of the pmfs describing the indicator values are used to calculate the driver state pmf. Thus there will be several pmfs describing the distribution for the data values for each indicator, and one pmf describing the distribution between valid driver states,” the distribution is discrete therefore it is considered each indicator values defines a plurality of windows of probability ).
The rationale of claim 1 has been applied herein.
Regarding claim 5, Myers in view of Kronberg further teaches the computer apparatus of claim 4,
wherein the windows of probability are defined by threshold deviations from a peak probability (see Kronberg, Paragraph [0041], “The benefit with having a discrete state vector is twofold; the state can be designed to correspond to different interventions, and the probability mass function (pmf) can be calculated exactly rather than approximated,”).
The rationale of claim 4 has been applied herein.
Regarding claim 6, Myers in view of Kronberg further teaches the computer apparatus claim 1,
wherein the pose estimate corresponds to an automatized behavior (see Myers, Paragraph [0125], “Working in conjunction, device controller 120 and vision processor 118 provide for capturing and processing images of the pilot/co-pilot to obtain subject state information such as drowsiness, attention, body position and posture, head pose and motion, cognitive state/load and gaze position during an ordinary operation of aircraft 104,” ordinary operation of aircraft is considered to be an automized behavior).
The rationale of claim 1 has been applied herein.
Regarding claim 7, Myers in view of Kronberg further teaches the computer apparatus of claim 1,
wherein the processor executable code further configures the at least one processor as a machine learning neural network (see Myers, Paragraph [0157], “data processing and storage unit 702 utilizes or implements a machine learned classifier algorithm which receives the various inputs and performs a classification to produce an alertness state of the pilot/co-pilot”).
The rationale of claim 1 has been applied herein.
As per Claim 8, Claim 8 claims a method comprising the same limitations as claimed in Claim 1. Therefore the rejection and rationale are analogous to that made in Claim 1.
Myers further teaches, receiving an imaging stream from at least one camera (see Myers, Paragraph [0185], “At step 1301, camera 106 is controlled by device controller 120 to capture a plurality of digital images of the cockpit of aircraft 104 including one or both of the pilot 102 and a co-pilot 103”)
As per Claim 10, Claim 10 claims the same limitation as Claim 3 and is dependent on a similarly rejected independent claim. Therefore the rejection and rationale are analogous to that made in Claim 3.
As per Claim 11, Claim 11 claims the same limitation as Claim 4 and is dependent on a similarly rejected independent claim. Therefore the rejection and rationale are analogous to that made in Claim 4.
As per Claim 12, Claim 12 claims the same limitation as Claim 5 and is dependent on a similarly rejected independent claim. Therefore the rejection and rationale are analogous to that made in Claim 5.
As per Claim 13, Claim 13 claims the same limitation as Claim 6 and is dependent on a similarly rejected independent claim. Therefore the rejection and rationale are analogous to that made in Claim 6.
As per Claim 14, Claim 14 claims a pilot monitoring system comprising: the same limitations as claimed in Claim 1. Therefore the rejection and rationale are analogous to that made in Claim 1.
Myers further teaches, at least one camera (see Myers, Paragraph [0185], “At step 1301, camera 106 is controlled by device controller 120 to capture a plurality of digital images of the cockpit of aircraft 104 including one or both of the pilot 102 and a co-pilot 103”)
As per Claim 16, Claim 16 claims the same limitation as Claim 3 and is dependent on a similarly rejected independent claim. Therefore the rejection and rationale are analogous to that made in Claim 3.
As per Claim 17, Claim 17 claims the same limitation as Claim 4 and is dependent on a similarly rejected independent claim. Therefore the rejection and rationale are analogous to that made in Claim 4.
As per Claim 18, Claim 18 claims the same limitation as Claim 5 and is dependent on a similarly rejected independent claim. Therefore the rejection and rationale are analogous to that made in Claim 5.
As per Claim 19, Claim 19 claims the same limitation as Claim 6 and is dependent on a similarly rejected independent claim. Therefore the rejection and rationale are analogous to that made in Claim 6.
As per Claim 20, Claim 20 claims the same limitation as Claim 7 and is dependent on a similarly rejected independent claim. Therefore the rejection and rationale are analogous to that made in Claim 7.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DOMINIQUE JAMES whose telephone number is (703)756-1655. The examiner can normally be reached 9:00 am - 6:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571)270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DOMINIQUE JAMES/Examiner, Art Unit 2666 /EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666
Prosecution Insights