Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Status of the Application
The following is a non-Final Office Action.
In response to Examiner's communication of 11/12/2025, Applicant responded on 3/12/2026. Amended claims 1, 7, 9, 14, 16, 20. Cancelled claims 8, 15. Added claims 21-22.
Claims 1-7, 9-14, 16-22 are pending in this application and have been examined.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 3/12/2026 has been entered.
Response to Amendment
Applicant's amendments to claims 1, 7, 9, 14, 16, 20 are not sufficient to overcome the 101 rejections set forth in the previous action.
Applicant's amendments to claims 1, 7, 9, 14, 16, 20 are not sufficient to overcome the prior art rejections set forth in the previous action.
Response to Arguments – 35 USC § 101
Applicant’s arguments with respect to the rejections have been fully considered, but they are not persuasive.
Applicant submits, “…As discussed in Applicant's specification in paragraph [0014] current methods of evaluating a training subject by observing the training subject during a training performance pose several problems. Paragraph [0014] continues that "evaluation by observation can be difficult to provide information with traceability between actions performed by the training subject and the ICAO competency framework" which "can make it difficult to provide recommendations to help the training subject improve their training performance in a next training performance over a current training performance" and additionally, "a similar training performance can receive differing evaluations across different instructors" that need "to be present during the training performance for the observation." Therefore, evaluations by observation can be inconsistent and lack traceability. To address such concerns, Applicant discloses by example, at paragraph [0015] "utilizing computer-implemented behavioral elicitation events (BEEs) to automate the evaluation of a training performance of a training subject against a competency framework" where "[a] BEE is an event that elicits particular behaviors out of the training subject to demonstrate specified soft skill competencies in the competency framework." Paragraph [0015] continues that "[t]he training performance is performed by the training subject in a training environment that simulates a physical environment." The specification continues with a specific example in paragraph [0018] where an "aircraft training environment 102 is configured to simulate an aircraft environment for use in training pilots." Paragraph [0015] continues that a training computing system is configured to perform a computer simulated training scenario and to utilize a training interface for receiving user inputs in response to the computer simulated training scenario. Applicant believes that the human mind is not equipped to simulate a physical environment for a training subject to perform a training performance. Applicant also believes that the human mind is not equipped to perform a computer simulated training scenario and to receive user inputs from a training interface in response to the computer simulated training scenario. As stated in MPEP 2106.04(a)(2)IIIA, "[c]laims do not recite a mental process when they do not contain limitations that can practically be performed in the human mind, for instance when the human mind is not equipped to perform the claim limitations." Thus, amended claim 1 and claims depending therefrom are not directed towards an abstract idea of a mental process. As summarized in MPEP 2106.04(d), limitations the courts have found indicative that an additional element (or combination of elements) may have integrated a judicial exception into a practical application include an improvement in the functioning of a computer, or an improvement to other technology or technical field. Courts also have found claims that implement a judicial exception with, or that use a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim may integrate an abstract idea into a practical application. Applicant believes a training environment that simulates a physical environment including a training interface, and implements a computer simulated training scenario within the context of the simulated physical environment is a particular machine or manufacture that is integral to amended claim 1….amended claims 1, 9, 16, and claims depending therefrom define eligible subject matter. Applicant notes that amended claims 1, 9, and 16 contain meaningful limitations that do not monopolize the asserted abstract idea of a commercial activity that falls within the certain methods of organizing human activities….” The Examiner respectfully disagrees.
While Applicant’s amendments further prosecution, however the claims, by Applicant’s own admission, recite and direct to, …evaluating a training subject (i.e. human) by observing the training subject (i.e. human) during a training performance pose several problems.…provide data on the training performance to help an instructor (i.e. human) objectively assess soft skill competencies and increase consistency of inter-rater reliability over evaluations by observation of the instructor (i.e. human)… provide detailed evidence for use when asserting the proficiency of a training subject (i.e. human) performing the training performance… evaluation of the training performance in a more objective, reliable, and traceable manner than by observations of instructors (i.e. human)…, which is a problem directed to a mental process (i.e. human instructor observing and training human pilot actions and behaviors with training scenarios and learning simulations) and organizing human activities (i.e. h human instructor observing and training human pilot actions and behaviors with training scenarios and learning simulations), as established in Step 2A Prong 1. This problem does not specifically arise in the realm of computer technology, but rather, this problem existed and was addressed long before the advent of computers. Thus, the claims do not recite a technical improvement to a technical problem or necessarily roots in computing technologies. The alleged solutions are solutions directed to solving abstract ideas, which are still abstract ideas. Additionally, pursuant to the broadest reasonable interpretation, as an ordered combination, each of the additional elements are computing elements recited at high level of generality implementing the abstract idea, and thus, are no more than applying the abstract idea with generic computer components. Further, these additional elements generally link the abstract idea to a technical environment, namely the environment of a computer flight simulator, performing extra solution activities. Therefore, as a whole, the additional elements do not integrate the abstract ideas into a practical application in Step 2A Prong 2 or amount to significantly more under Step 2B.
Even novel and newly discovered judicial exceptions are still exceptions, despite their novelty. July 2015 Update, p. 3; see SAP America Inc. v. Investpic, LLC, No. 2017-2081, slip op. at 2 (Fed Cir. May 15, 2018).
Simply reciting specific limitations that narrow the abstract idea does not make an abstract idea non-abstract. 79 Fed. Reg. 74631; buySAFE Inc. v. Google, Inc., 765 F.3d 1350, 1355 (2014); see SAP America at p. 12. As discussed in SAP America, no matter how much of an advance the claims recite, when “the advance lies entirely in the realm of abstract ideas, with no plausibly alleged innovation in the non-abstract application realm,” “[a]n advance of that nature is ineligible for patenting.” Id. at p. 3.
Claims can recite a mental process even if they are claimed as being performed on a computer. The Supreme Court recognized this in Benson, determining that a mathematical algorithm for converting binary coded decimal to pure binary within a computer’s shift register was an abstract idea. The Court concluded that the algorithm could be performed purely mentally even though the claimed procedures “can be carried out in existing computers long in use, no new machinery being necessary.” 409 U.S at 67, 175 USPQ at 675. See also Mortgage Grader, 811 F.3d at 1324, 117 USPQ2d at 1699 (concluding that concept of “anonymous loan shopping” recited in a computer system claim is an abstract idea because it could be “performed by humans without a computer”).
An example of a case in which a computer was used as a tool to perform a mental process is Mortgage Grader, 811 F.3d. at 1324, 117 USPQ2d at 1699. The patentee in Mortgage Grader claimed a computer-implemented system for enabling borrowers to anonymously shop for loan packages offered by a plurality of lenders, comprising a database that stores loan package data from the lenders, and a computer system providing an interface and a grading module. The interface prompts a borrower to enter personal information, which the grading module uses to calculate the borrower’s credit grading, and allows the borrower to identify and compare loan packages in the database using the credit grading. 811 F.3d. at 1318, 117 USPQ2d at 1695. The Federal Circuit determined that these claims were directed to the concept of “anonymous loan shopping”, which was a concept that could be “performed by humans without a computer.” 811 F.3d. at 1324, 117 USPQ2d at 1699. Another example is Berkheimer v. HP, Inc., 881 F.3d 1360, 125 USPQ2d 1649 (Fed. Cir. 2018), in which the patentee claimed methods for parsing and evaluating data using a computer processing system. The Federal Circuit determined that these claims were directed to mental processes of parsing and comparing data, because the steps were recited at a high level of generality and merely used computers as a tool to perform the processes. 881 F.3d at 1366, 125 USPQ2d at 1652-53.
TLI Communications provides an example of a claim invoking computers and other machinery merely as a tool to perform an existing process. The court stated that the claims describe steps of recording, administration and archiving of digital images, and found them to be directed to the abstract idea of classifying and storing digital images in an organized manner. 823 F.3d at 612, 118 USPQ2d at 1747. The court then turned to the additional elements of performing these functions using a telephone unit and a server and noted that these elements were being used in their ordinary capacity (i.e., the telephone unit is used to make calls and operate as a digital camera including compressing images and transmitting those images, and the server simply receives data, extracts classification information from the received data, and stores the digital images based on the extracted information). 823 F.3d at 612-13, 118 USPQ2d at 1747-48. In other words, the claims invoked the telephone unit and server merely as tools to execute the abstract idea. Thus, the court found that the additional elements did not add significantly more to the abstract idea because they were simply applying the abstract idea on a telephone network without any recitation of details of how to carry out the abstract idea.
Other examples where the courts have found the additional elements to be mere instructions to apply an exception, because they do no more than merely invoke computers or machinery as a tool to perform an existing process include:
i. A commonplace business method or mathematical algorithm being applied on a general purpose computer, Alice Corp. Pty. Ltd. V. CLS Bank Int’l, 573 U.S. 208, 223, 110 USPQ2d 1976, 1983 (2014); Gottschalk v. Benson, 409 U.S. 63, 64, 175 USPQ 673, 674 (1972); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015);
iii. A process for monitoring audit log data that is executed on a general-purpose computer where the increased speed in the process comes solely from the capabilities of the general-purpose computer, FairWarning IP, LLC v. Iatric Sys., 839 F.3d 1089, 1095, 120 USPQ2d 1293, 1296 (Fed. Cir. 2016);
v. Requiring the use of software to tailor information and provide it to the user on a generic computer, Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1370-71, 115 USPQ2d 1636, 1642 (Fed. Cir. 2015)
Response to Arguments – Prior Art
Applicant’s arguments with respect to the rejections have been fully considered, but they are not persuasive.
Applicant submits, “…Kennedy discloses at paragraph [0023] training modules "that may provide instruction regarding scenarios that are applicable to a wide range of aircraft and situations" and, at paragraph [0024], that the embedded training modules could be taken during long-cruise segments in areas with minimal other traffic, in preparation for a flight, or at another time. Kennedy also discloses at paragraph [0049] that regulatory or audit data may map employees to various training exercises and, at paragraph [0050], that as the employees complete the training, it may be removed from their list of training exercises. Here, Kennedy discloses simulation modules applicable to a wide range of aircraft and that can be taken during flight. However, Kennedy is silent regarding evaluating a training performance of a training scenario. Thus, Kennedy fails to teach or suggest utilizing a BEE database to evaluate a training performance being performed in a training environment that simulates a physical environment as claimed.…Peyronnet fails to teach or suggest at least the elements of, for each active BEE, determine whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE, and query the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store information regarding the one or more corresponding observable behaviors of the completed BEE.…the combination of Kennedy and Peyronnet fails to teach or suggest at least the elements of amended claim 1 of evaluating a training performance of a training subject in response to a computer simulated training scenario, the training performance being performed in a training environment that simulates a physical environment, for each active BEE, determine whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE, and query the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store information regarding the one or more corresponding observable behaviors of the completed BEE….the combination of Kennedy and Peyronnet also fails to teach or suggest at least the elements of amended claim 9 of a training environment configured to simulate a physical environment, the training environment comprising a training computing system configured to perform a simulated training scenario including a simulation state, for each active BEE, determine whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE, and query the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store, in the learning record, information regarding the corresponding observable behaviors of the completed BEE….the combination of Kennedy and Peyronnet fails to teach or suggest at least the elements of evaluating a training performance of a training subject in response to a computer simulated training scenario, the training performance being performed in a training environment that simulates a physical environment, for each active BEE, determine whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE, and query the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store information regarding the one or more corresponding observable behaviors of the completed BEE. Therefore, the combination of Kennedy and Peyronnet also fails to teach or suggest at least the elements of amended claim 16 of evaluating a training performance of a training subject in response to a computer simulated training scenario, the training performance being performed in a training environment that simulates a physical environment, querying the BEE database to determine a second set of active BEEs based upon the second simulation state update, the first set of active BEEs including at least one active BEE that is not in the second set of active BEEs, comparing the first set of active BEEs to the second set of active BEEs, and for each active BEE in the first set of active BEEs and not in the second set of active BEEs, determine that the active BEE is an incomplete BEE, and store, in the learning record, information regarding the incomplete BEE and the second simulation state update…” The Examiner respectfully disagrees.
Respectfully, Applicant’s argument requires that the each of the features of supporting references are bodily incorporated into primary reference that teach and every element is individually taught by a single reference. However, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). The test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one single or in all of the references. See id. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See id.; In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Kennedy in view of Peyronnet teach the claim limitations as established further below in the office action.
Further, under the broadest reasonable interpretation, Kennedy teaches:
…a training subject in response to a computer simulated training scenario, the training performance being performed in a training environment that simulates a physical environment and comprising a training interface configured to receive user inputs from the training subject… (in at least [0052] Referring to FIG. 6, a conceptual diagram 600 depicting an embodiment of embedded training for commercial aviation is depicted. As shown in the diagram 600, a learning management system 610 and mid-fidelity simulation (MFS) training scenarios 630 may be used to provide embedded training 650 via mid-fidelity simulation. The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620.)
…
for each simulation action notification, determining whether an action represented by the simulation action notification … an elicited action for at least one active BEE in the set of active BEEs and storing, in a learning record, information regarding the elicited action as a response to the at least one active BEE in the set of active BEEs; (in at least [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight. The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, (i.e. simulation action notification) such as “You need to complete this module by [a particular date]—do you want to do it now?” or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” (i.e. elicited action) [0028] provide engaging, useful, and operationally relevant interactions on the flight deck. Given the sensitivities around providing training (and potential distraction), one, or a few modules, with clear operational relevance to a flight could be offered initially. These could be deployed on a voluntary basis. Tracking of when the modules are taken can be used to identify patterns of usage and adapt the modules and system to fit the needs of the pilots. Over time, an increasing number of modules can be introduced. [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. (i.e. action, and elicited action) [0039] The system 100 may be used in conjunction with a mid-fidelity simulator. For example, the training exercise 160 may include a mid-fidelity simulation 162. The mid-fidelity simulation 162 may provide simulations and scenarios that are common among multiple aircraft. This may enable the employee to receive conceptual scenario training as opposed to aircraft-specific training. (i.e. simulation action notification) [0050] As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. (i.e. active) [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
for each active BEE, determine whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE; and (in at least [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. [0050] As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. [0051] FIG. 5, an embodiment of the sample data 152 is depicted. The sample data 152 may map employees 510-512 to training histories 520, 530, 540. For example, a first employee 510 may be mapped to a first training history 520, a second employee 511 may be mapped to a second training history 530, and a third employee 512 may be mapped to a third training history 540. The learning management system 110 of FIG. 1 may use the sample data 152 to determining a common training exercise 172 associated with the employees 510-512. The training exercise 160 may correspond to and/or may be the common training exercise 172. Determining the common training exercise 172 associated with employees 510-512 may include performing an artificial intelligence analysis of the training histories 520, 530, 540. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
query the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store information regarding the one or more corresponding observable behaviors of the completed BEE. (in at least [0052] The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it (i.e. query). For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
…
for each active BEE, determine whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE, and (in at least [0028] provide engaging, useful, and operationally relevant interactions on the flight deck. Given the sensitivities around providing training (and potential distraction), one, or a few modules, with clear operational relevance to a flight could be offered initially. These could be deployed on a voluntary basis. Tracking of when the modules are taken can be used to identify patterns of usage and adapt the modules and system to fit the needs of the pilots. Over time, an increasing number of modules can be introduced. [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. [0050] As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. [0051] FIG. 5, an embodiment of the sample data 152 is depicted. The sample data 152 may map employees 510-512 to training histories 520, 530, 540. For example, a first employee 510 may be mapped to a first training history 520, a second employee 511 may be mapped to a second training history 530, and a third employee 512 may be mapped to a third training history 540. The learning management system 110 of FIG. 1 may use the sample data 152 to determining a common training exercise 172 associated with the employees 510-512. The training exercise 160 may correspond to and/or may be the common training exercise 172. Determining the common training exercise 172 associated with employees 510-512 may include performing an artificial intelligence analysis of the training histories 520, 530, 540. [0052] The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620.)
query the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store, in the learning record, information regarding the corresponding observable behaviors of the completed BEE. (in at least [0052] The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it (i.e. query). For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
…
for each active BEE in the first set of active BEEs and not in the second set of active BEEs, determine that the active BEE is an incomplete BEE, and store, in the learning record, information regarding the incomplete BEE and the second simulation state update. (in at least [0026] The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” (i.e. active and incomplete) [0036] In order to determine which training exercises may be applicable to an employee, the learning management system 110 may retrieve regulatory or audit data 142 from the regulatory or audit database 140. The regulatory or audit data 142 may indicate training exercises or concepts that are required for the employee and have not yet been performed (i.e. second BEE). Based on the regulatory data or audit data 142, the training exercise 160 may be selected. [0037] employee may alternatively or additionally include retrieving sample data 152 associated with multiple employees and determining a common training exercise 172 associated with the multiple employees. The training exercise 160 may be the common training exercise 172. Determining the common training exercise 172 associated with multiple employees may include performing an artificial intelligence analysis of a training history associated with the multiple employees using the AI learning model 170. [0038] Once the training exercise 160 has been selected (i.e. active), the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time.(i.e. first set of active BEEs not in the second set of active BEEs and store as incomplete) [0048] FIG. 3, an embodiment of the flight data 132 is depicted. The flight data 132 may associate, or otherwise map, flights 311, 312, with timelines 320, 330. For example, a first flight 311 may be mapped to a timeline 320 that includes aviation events such as an ocean crossing 322 and a new approach 324 to an airport. A second flight 312 may be mapped to a timeline 330 that includes aviation events such as a mountain crossing 332. The aviation events may be the basis for training concepts (e.g., the training concept 168 of FIG. 1). The learning management system 110 of FIG. 1 may use the flight data 132 to determine the training concept 168 and to select the training exercise 160 from the multiple training exercises 164. It should be noted that the ocean crossing 322, the new approach 324, and the mountain crossing 332 are only some examples (i.e. sets of active BEEs) of many different types of aviation events that may be associated with flights. Other potential scenarios may include fuel contaminations, ash encounters, technical problems, diversions due to sick passengers, etc. (i.e. sets of active BEEs) In practice, any type of aviation event may be associated with a flight and/or scheduled for training. [0049] FIG. 4, an embodiment of the regulatory or audit data 142 is depicted. The regulatory or audit data 142 may map employees 410-412 to various required or preferred training exercises 420-422, 430, 440, 441. For example, a first employee 410 may be mapped to a first set of required or preferred training exercises 410-422, a second employee may be mapped to a second required or preferred training exercise 430, and a third employee 412 may be mapped to a third set of required or preferred training exercises 440, 441. [0050] Regulatory authorities (such as the Federal Aviation Administration) may require that the employees 410-412 receive certain training and may perform audits to ensure that it has been done. Likewise, individual airlines or air service providers may require certain training. As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. [0051] FIG. 5, an embodiment of the sample data 152 is depicted. The sample data 152 may map employees 510-512 to training histories 520, 530, 540. For example, a first employee 510 may be mapped to a first training history 520, a second employee 511 may be mapped to a second training history 530, and a third employee 512 may be mapped to a third training history 540. The learning management system 110 of FIG. 1 may use the sample data 152 to determining a common training exercise 172 associated with the employees 510-512. The training exercise 160 may correspond to and/or may be the common training exercise 172. Determining the common training exercise 172 associated with employees 510-512 may include performing an artificial intelligence analysis of the training histories 520, 530, 540. [0052] The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios (i.e. first set of active BEEs) where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed (i.e. first set of active BEEs) and what training still needs to be completed. (i.e. stored as incomplete and second set of active BEEs) [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise (i.e. second simulation state update) to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training , or additional training, within a certain time frame. )
Although implied, Kennedy does not expressly disclose the following limitations, which however, are taught by Peyronnet,
…action … matches an elicited action for at least one active BEE in the set of active BEEs… (in at least [0067] FIG. 2 , the data correlation step 104 comprises a step 116 consisting in determining a trigger event originating from an operator, the trigger event originating from an operator is the occurrence of an event at the origin of an action of the at least one operator in response to this trigger event originating from an operator. The trigger parameter originating from an operator may thus be an endogenous datum, such as for example the initialization of technical dialog between the team of operators. However, it may also be contemplated, in a more generalized scenario, for the trigger event parameter to be an exogenous datum. [0068] The data correlation step 104 then comprises a step 118 consisting in determining a trigger event originating from the platform. The trigger event originating from the platform represents the occurrence of an event at the origin of a state of the platform, and may be interpreted as an exogenous datum. This is then followed by a step 120 of detecting trigger events originating from said at least one operator or from said platform, and a step 122 of selecting at least one trigger event. [0069] The detection 120 of a trigger event originating from an operator is based on the detection of an action from the at least one operator. The detection 120 of a trigger event originating from a platform is based on the detection of a state of the platform, such as for example a change of piloting mode, the extension or retraction of the landing gear, a fault, and on the departure from an envelope of dynamic parameters, such as for example speed, incline, attitude. [0070] In response to the trigger event, whether it is a trigger from the at least one operator or from the platform, the assessment method 100 then captures at least one action parameter represented by an endogenous datum and presenting the physical manifestation of a reaction of the at least one operator to the trigger parameter. Grouping together a trigger parameter, represented by an exogenous datum or an endogenous datum, and at least one action parameter, represented by endogenous data, thus makes it possible, during the correlation step 104, to generate at least one observable behavior datum reflecting, according to tangible parameters, the behavior of the at least one operator upon triggering of an event. [0079] In step 106, the assessment method 100 comprises a step consisting in analyzing the observable behavior data in predefined analysis sequences, each predefined analysis sequence being specific to a technical and non-technical skill to be assessed, and comprising at least one trigger event parameter and one action parameter for characterizing an expected observable behavior according to a predefined situation. The analysis 106 also makes it possible to generate a measurement indicator for each observed behavior. More specifically, step 106 analyzes observable behavior data under the prism of trigger event and action parameters by comparing (step 132) the detected observable behavior data with a predefined sequence defining the expected observable behavior, each predefined sequence representing at least one physical manifestation allocated to the expected behavior. The predefined sequences are contained in a correspondence database. This correspondence database thus comprises the predefined analysis sequences presenting observable behavior data known to those skilled in the art, as well as their assigned measurable and detectable physical manifestations. Each predefined analysis sequence thus comprises at least one trigger event parameter and at least one action parameter and other endogenous and exogenous data for characterizing a flight situation and a context for at least one operator, and also their expected reaction according to the predefined situation. This analysis provides the nature of the induced action, its temporal location and also its duration or its frequency. The correspondence database also comprises a reference table containing trigger event parameters associated with each behavior to be observed. [0080] in order to be able to analyze the behavior of at least one operator during a flight situation, the step 106 of analyzing the observable behavior data compares the detected endogenous and exogenous data, and more specifically the trigger event and action parameters, with the trigger event and action parameters and also the predefined endogenous and exogenous data. The predefined analysis sequences are specific to each technical and non-technical skill to be assessed. [0092] This generation of observed behavior indicators then makes it possible to initiate a step 108, shown in FIG. 1 , consisting in assessing a behavior of at least one operator. The assessment of the behavior of the at least one operator consists in comparing the observed behavior, which is based on a set of detected behavior elements, with predefined expected reference behaviors. The conformity of an observed behavior is assessed by comparison with known prior art of defined procedures or established protocols, contained in the correspondence database. [0093] The objectivity of the assessment of the technical and non-technical skills of an operator is thus based on the prior creation of the correspondence database between various observable behaviors and measurable physical variables in relation to these observable behaviors. [0094] The matching consists, for each observable behavior, in determining various ways of measuring same and then developing the tools necessary for each measurement. [0095] By way of indicative and non-exhaustive examples, various cases below are given to illustrate the matching of observable behaviors with measurable physical variables to allow behaviors to be assessed.)
At the time the invention was filed, it would have been obvious for one of ordinary skill in the art to have modified the teachings of Kennedy, as taught by Peyronnet above, with a reasonable expectation of success if arriving at the claimed invention. One of ordinary skill in the art would have been motivated to make this modification to the teachings of Kennedy with the motivation of, …analyzing the behavior of an operator in a simulation or training situation, allowing an observer to obtain statistical data that provide real-time information about the state and the behavior of the operator. Using these statistical data, the observer is able to carry out their own analysis of the technical and non-technical skills of the operator… to assist instructors in assessing these non-technical skills, the European Union Aviation Safety Agency (EASA) has published a list of Observable Behavior Indicators (OBI). These behavior indicators make it possible to objectify these various skills and provide a shared assessment framework between instructors, enabling a reduction in subjectivity in the assessment… allow the instructor to be easily directed to observable behavior data needed to assess skills…assessing and training commercial aviation pilots developed by players in the aeronautical world, based on an objective skills assessment (competency-based training)….to improve the assessment capabilities of the assessment method 100 by enriching the correspondence database for subsequent use of the assessment method…, as recited in Peyronnet.
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-7, 9-14, 16-22 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim 1 recite, “A method of evaluating a training performance of a training subject in response to a computer simulated training scenario, the training performance being performed in a training environment that simulates a physical environment and comprising a training interface configured to receive user inputs from the training subject, the method comprising:
receiving a simulation state update indicating a change in a simulation state of the …simulated training scenario from a training … of the training environment;
querying a behavioral elicitation event (BEE) database to determine, from a plurality of BEEs stored in the BEE database, a set of active BEEs based upon the simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions and one or more corresponding observable behaviors;
receiving a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface;
for each simulation action notification, determining whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the set of active BEEs and storing, in a learning record, information regarding the elicited action as a response to the at least one active BEE in the set of active BEEs;
for each active BEE, determine whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE; and
query the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store information regarding the one or more corresponding observable behaviors of the completed BEE.”
Claim 9 recites, A … comprising:
a training environment configured to simulate a physical environment, the training environment comprising
a training … configured to perform a simulated training scenario including a simulation state, and
a training interface configured to receive user inputs from a training subject in response to the simulated training scenario; and
a …, and
a … comprising,
a behavioral elicitation event (BEE) database comprising a plurality of BEEs, each BEE of the plurality of BEEs comprising one or more elicited actions defining expected user inputs as a response to the BEE, and one or more corresponding observable behaviors,
a learning record to store information about the simulation state of the training environment during the simulated training scenario and information related to a training performance of the training subject in response to the simulated training scenario, and
instructions executable by the … to
receive a simulation state update indicating a change in the simulation state of the simulated training scenario from the training …,
query the BEE database to determine, from the plurality of BEEs, a set of active BEEs based upon the simulation state update,
receive a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface,
for each simulation action notification, determine whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the set of active BEEs, and store, in the learning record, information regarding the elicited action as the response to the at least one active BEE in the set of active BEEs,
for each active BEE, determine whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE, and
query the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store, in the learning record, information regarding the corresponding observable behaviors of the completed BEE.
Claim 16 recites, A method of evaluating a training performance of a training subject in response to a … simulated training scenario, the training performance being performed in a training environment that simulates a physical environment and comprising a training interface configured to receive user inputs from the training subject, the method comprising:
receiving a first simulation state update from a training … of the training environment, the first simulation state update indicating a first change in a simulation state of the simulated training scenario;
querying a behavioral elicitation event (BEE) database to determine, from a plurality of BEEs stored in the BEE database, a first set of active BEEs based upon the first simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions defining expected user inputs as a response to the BEE, and one or more corresponding observable behaviors;
receiving a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface;
for each simulation action notification, determining whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the first set of active BEEs and storing, in a learning record, information regarding the elicited action and corresponding observable behaviors as the response to the at least one active BEE in the first set of active BEEs;
receiving a second simulation state update from the training computing system of the training environment, the second simulation state update indicating a second change in the simulation state of the simulated training scenario;
querying the BEE database to determine a second set of active BEEs based upon the second simulation state update, the first set of active BEEs including at least one active BEE that is not in the second set of active BEEs;
comparing the first set of active BEEs to the second set of active BEEs; and
for each active BEE in the first set of active BEEs and not in the second set of active BEEs, determine that the active BEE is an incomplete BEE, and store, in the learning record, information regarding the incomplete BEE and the second simulation state update.
Analyzing under Step 2A, Prong 1:
The limitations regarding, …evaluating a training performance of a training subject in response to a computer simulated training scenario, the training performance being performed in a training environment that simulates a physical environment and comprising a training interface configured to receive user inputs from the training subject, …receiving a simulation state update indicating a change in a simulation state of the …simulated training scenario from a training … of the training environment; querying a behavioral elicitation event (BEE) database to determine, from a plurality of BEEs stored in the BEE database, a set of active BEEs based upon the simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions and one or more corresponding observable behaviors; receiving a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface; for each simulation action notification, determining whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the set of active BEEs and storing, in a learning record, information regarding the elicited action as a response to the at least one active BEE in the set of active BEEs; for each active BEE, determine whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE; and query the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store information regarding the one or more corresponding observable behaviors of the completed BEE… a training environment configured to simulate a physical environment, the training environment comprising a training … configured to perform a simulated training scenario including a simulation state, and a training interface configured to receive user inputs from a training subject in response to the simulated training scenario; …a behavioral elicitation event (BEE) database comprising a plurality of BEEs, each BEE of the plurality of BEEs comprising one or more elicited actions defining expected user inputs as a response to the BEE, and one or more corresponding observable behaviors, a learning record to store information about the simulation state of the training environment during the simulated training scenario and information related to a training performance of the training subject in response to the simulated training scenario, …receive a simulation state update indicating a change in the simulation state of the simulated training scenario from the training …, query the BEE database to determine, from the plurality of BEEs, a set of active BEEs based upon the simulation state update, receive a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface, for each simulation action notification, determine whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the set of active BEEs, and store, in the learning record, information regarding the elicited action as the response to the at least one active BEE in the set of active BEEs, for each active BEE, determine whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE, and query the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store, in the learning record, information regarding the corresponding observable behaviors of the completed BEE…A method of evaluating a training performance of a training subject in response to a … simulated training scenario, the training performance being performed in a training environment that simulates a physical environment and comprising a training interface configured to receive user inputs from the training subject…receiving a first simulation state update from a training … of the training environment, the first simulation state update indicating a first change in a simulation state of the simulated training scenario; querying a behavioral elicitation event (BEE) database to determine, from a plurality of BEEs stored in the BEE database, a first set of active BEEs based upon the first simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions defining expected user inputs as a response to the BEE, and one or more corresponding observable behaviors; receiving a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface; for each simulation action notification, determining whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the first set of active BEEs and storing, in a learning record, information regarding the elicited action and corresponding observable behaviors as the response to the at least one active BEE in the first set of active BEEs; receiving a second simulation state update from the training computing system of the training environment, the second simulation state update indicating a second change in the simulation state of the simulated training scenario; querying the BEE database to determine a second set of active BEEs based upon the second simulation state update, the first set of active BEEs including at least one active BEE that is not in the second set of active BEEs; comparing the first set of active BEEs to the second set of active BEEs; and for each active BEE in the first set of active BEEs and not in the second set of active BEEs, determine that the active BEE is an incomplete BEE, and store, in the learning record, information regarding the incomplete BEE and the second simulation state update…,, under the broadest reasonable interpretation, can include a human using their mind and using pen and paper to perform the identified limitations; therefore, the claims recite a mental process.
Further, …evaluating a training performance of a training subject in response to a computer simulated training scenario, the training performance being performed in a training environment that simulates a physical environment and comprising a training interface configured to receive user inputs from the training subject, …receiving a simulation state update indicating a change in a simulation state of the …simulated training scenario from a training … of the training environment; querying a behavioral elicitation event (BEE) database to determine, from a plurality of BEEs stored in the BEE database, a set of active BEEs based upon the simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions and one or more corresponding observable behaviors; receiving a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface; for each simulation action notification, determining whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the set of active BEEs and storing, in a learning record, information regarding the elicited action as a response to the at least one active BEE in the set of active BEEs; for each active BEE, determine whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE; and query the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store information regarding the one or more corresponding observable behaviors of the completed BEE… a training environment configured to simulate a physical environment, the training environment comprising a training … configured to perform a simulated training scenario including a simulation state, and a training interface configured to receive user inputs from a training subject in response to the simulated training scenario; …a behavioral elicitation event (BEE) database comprising a plurality of BEEs, each BEE of the plurality of BEEs comprising one or more elicited actions defining expected user inputs as a response to the BEE, and one or more corresponding observable behaviors, a learning record to store information about the simulation state of the training environment during the simulated training scenario and information related to a training performance of the training subject in response to the simulated training scenario, …receive a simulation state update indicating a change in the simulation state of the simulated training scenario from the training …, query the BEE database to determine, from the plurality of BEEs, a set of active BEEs based upon the simulation state update, receive a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface, for each simulation action notification, determine whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the set of active BEEs, and store, in the learning record, information regarding the elicited action as the response to the at least one active BEE in the set of active BEEs, for each active BEE, determine whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE, and query the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store, in the learning record, information regarding the corresponding observable behaviors of the completed BEE…A method of evaluating a training performance of a training subject in response to a … simulated training scenario, the training performance being performed in a training environment that simulates a physical environment and comprising a training interface configured to receive user inputs from the training subject…receiving a first simulation state update from a training … of the training environment, the first simulation state update indicating a first change in a simulation state of the simulated training scenario; querying a behavioral elicitation event (BEE) database to determine, from a plurality of BEEs stored in the BEE database, a first set of active BEEs based upon the first simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions defining expected user inputs as a response to the BEE, and one or more corresponding observable behaviors; receiving a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface; for each simulation action notification, determining whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the first set of active BEEs and storing, in a learning record, information regarding the elicited action and corresponding observable behaviors as the response to the at least one active BEE in the first set of active BEEs; receiving a second simulation state update from the training computing system of the training environment, the second simulation state update indicating a second change in the simulation state of the simulated training scenario; querying the BEE database to determine a second set of active BEEs based upon the second simulation state update, the first set of active BEEs including at least one active BEE that is not in the second set of active BEEs; comparing the first set of active BEEs to the second set of active BEEs; and for each active BEE in the first set of active BEEs and not in the second set of active BEEs, determine that the active BEE is an incomplete BEE, and store, in the learning record, information regarding the incomplete BEE and the second simulation state update…, under the broadest reasonable interpretation, are human instructor observing and training human pilot actions and behaviors with training scenarios and learning simulations, therefore it is, managing personal behavior or relationships or interactions between people. Thus, the claims are directed to certain methods of organizing human activity.
Accordingly, the claims are directed to a mental process, certain methods of organizing human activity, and thus, the claims are directed to an abstract idea under the first prong of Step 2A.
Analyzing under Step 2A, Prong 2:
This judicial exception is not integrated into a practical application under the second prong of Step 2A.
In particular, the claims recite the additional elements beyond the recited abstract idea identified under Step 2A, Prong 1, such as:
Claim 1, 9, 16: computer simulated, training computing system, system, computing system comprising a logic system operably coupled to the training computing system, computer-readable storage system, logic system, computer-readable storage system
Claim 22: a fixed training device (FTD) flight simulator or a full motion simulator (FMS) flight simulator
, and pursuant to the broadest reasonable interpretation, as an ordered combination, each of the additional elements are computing elements recited at high level of generality implementing the abstract idea, and thus, are no more than applying the abstract idea with generic computer components. Further, these additional elements generally link the abstract idea to a technical environment, namely the environment of a computer.
Additionally, with respect to, “receiving …”, “input…”, “querying …”, “storing…”, “sending…”, “notification…”, these elements do not add a meaningful limitations to integrate the abstract idea into a practical application because they are extra-solution activity, pre and post solution activity - i.e. data gathering – “receiving …”, “input…”, “querying …”, “storing…”, data output – “sending…”, “notification…”
Analyzing under Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under Step 2B.
As noted above, the aforementioned additional elements beyond the recited abstract idea are not sufficient to amount to significantly more than the recited abstract idea because, as an order combination, the additional elements are no more than mere instructions to implement the idea using generic computer components (i.e. apply it).
Additionally, as an order combination, the additional elements append the recited abstract idea to well-understood, routine, and conventional activities in the field as individually evinced by the applicant’s own disclosure, as required by the Berkheimer Memo, in at least:
[0015] Accordingly, examples are disclosed that relate to utilizing computer- implemented behavioral elicitation events (BEEs) to automate the evaluation of a training performance of a training subject against a competency framework. A BEE is an event that elicits particular behaviors out of the training subject to demonstrate specified soft skill competencies in the competency framework. The training performance is performed by the training subject in a training environment that simulates a physical environment. Example training environments are a fixed training device (FTD) flight simulator, a full motion simulator (FMS) flight simulator, and a virtual training environment. As described in more detail below, the disclosed examples utilize a BEE database comprising a plurality of BEEs to evaluate the training performance. Each BEE of the plurality of BEEs comprises one or more triggers, one or more elicited actions, and one or more observable behaviors, as discussed in more detail with reference to FIG. 2. The disclosed examples receive simulation state updates from the training environment, and determine a set of active BEEs based on the simulation state update by querying the BEE database. The simulation state update indicates a change in a simulation state of the training environment.
[0049] In some examples the methods and processes described herein can be tied to a computing system of one or more computing devices. In particular, such methods and processes can be implemented in hardware as described above, as a computer-application program or service, an application-programming interface (API), a library, and/or other computer-program product.
[0050] FIG. 9 schematically shows a simplified representation of a computing system 900 configured to provide any to all of the compute functionality described herein. Computing system 900 can take the form of one or more personal computers, server computers, and computers integrated with aircraft, as examples. Computing system 100,training computing system 108, and computing system 402 are examples of computing system 900.
[0051] Computing system 900 includes a logic subsystem 902 and a storage subsystem 904. Computing system 900 can optionally include a display subsystem 906,input subsystem 908,communication subsystem 910, and/or other subsystems not shown in FIG. 9.
[0052] Logic subsystem 902 includes one or more physical devices configured to execute instructions. For example, the logic subsystem can be configured to execute instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions can be implemented to perform a task, implement a data type, transform the state of one or more components, achieve a technical effect, or otherwise arrive at a desired result.
[0053] The logic subsystem can include one or more hardware processors configured to execute software instructions. Additionally, or alternatively, the logic subsystem can include one or more hardware or firmware devices configured to execute hardware or firmware instructions. Processors of the logic subsystem can be single-core or multi-core, and the instructions executed thereon can be configured for sequential, parallel, and/or distributed processing. Individual components of the logic subsystem optionally can be distributed among two or more separate devices, which can be remotely located and/or configured for coordinated processing. Aspects of the logic subsystem can be virtualized and executed by remotely accessible, networked computing devices configured in a cloud-computing configuration.
[0054]Storage subsystem 904 includes one or more physical devices configured to temporarily and/or permanently hold computer information such as data and instructions executable by the logic subsystem. When the storage subsystem includes two or more devices, the devices can be collocated and/or remotely located. Storage subsystem 904 can include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content-addressable devices. Storage subsystem 904 can include removable and/or built-in devices. When the logic subsystem executes instructions, the state of storage subsystem 904 can be transformed - e.g., to hold different data.
[0055]Storage subsystem 904 can include removable and/or built-in devices. Storage subsystem 904 can include optical memory (e.g., CD, DVD, HD-DVD, Blu- Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and/or magnetic memory, among others. Storage subsystem 904 can include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content-addressable devices.
[0056] Aspects of logic subsystem 902 and storage subsystem 904 can be integrated together into one or more hardware-logic components. Such hardware-logic components can include program- and application-specific integrated circuits (PASIC/ASICs), program- and application-specific standard products (PSSP / ASSPs), system- on-a-chip (SOC), and complex programmable logic devices (CPLDs), for example.
[0057] The logic subsystem and the storage subsystem can cooperate to instantiate one or more logic machines. As used herein, the term "machine" is used to collectively refer to the combination of hardware, firmware, software, instructions, and/or any other components cooperating to provide computer functionality. In other words, "machines" are never abstract ideas and always have a tangible form. A machine can be instantiated by a single computing device, or a machine can include two or more sub-components instantiated by two or more different computing devices. In some implementations a machine includes a local component (e.g., software application executed by a computer processor) cooperating with a remote component (e.g., cloud computing service provided by a network of server computers). The software and/or other instructions that give a particular machine its functionality can optionally be saved as one or more unexecuted modules on one or more suitable storage devices.
[0058] When included, display subsystem 906 can be used to present a visual representation of data held by storage subsystem 904. This visual representation can take the form of a graphical user interface (GUI). As the herein described methods and processes change the data held by the storage subsystem, and thus transform the state of the storage subsystem, the state of display subsystem 906 can likewise be transformed to visually represent changes in the underlying data. Display subsystem 906 can include one or more display devices utilizing virtually any type of technology. Such display devices can be combined with the logic subsystem and the storage subsystem in a shared enclosure, or such display devices can be peripheral display devices.
[0059] When included, input subsystem 908 can comprise or interface with one or more input devices such as a keyboard and touch screen. In some examples, the input subsystem can comprise or interface with selected natural user input (NUI) componentry. Such componentry can be integrated or peripheral, and the transduction and/or processing of input actions can be handled on- or off-board. Example NUI componentry can include a microphone for speech and/or voice recognition; and an infrared, color, stereoscopic, and/or depth camera for machine vision and/or gesture recognition.
[0060] When included, communication subsystem 910 can be configured to communicatively couple computing system 900 with one or more other computing devices. Communication subsystem 910 can include wired and/or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem can be configured for communication via a wireless telephone network, or a wired or wireless local- or wide-area network. In some examples, the communication subsystem may allow computing system 900 to send and/or receive messages to and/or from other devices via a network such as the Internet.
[0082] It will be understood that the configurations and/or approaches described herein are exemplary in nature, and that these specific embodiments or examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. As such, various acts illustrated and/or described may be performed in the sequence illustrated and/or described, in other sequences, in parallel, or omitted. Likewise, the order of the above-described processes may be changed.
[0083] The subject matter of the present disclosure includes all novel and non- obvious combinations and sub-combinations of the various processes, systems and configurations, and other features, functions, acts, and/or properties disclosed herein, as well as any and all equivalents thereof.
Furthermore, as an ordered combination, these elements amount to generic computer components receiving or transmitting data over a network, performing repetitive calculations, electronic record keeping, and storing and retrieving information in memory, which, as held by the courts, are well-understood, routine, and conventional. See MPEP 2106.05(d).
Moreover, the remaining elements of dependent claims do not transform the recited abstract idea into a patent eligible invention because these remaining elements merely recite further abstract limitations that provide nothing more than simply a narrowing of the abstract idea recited in the independent claims.
Looking at these limitations as an ordered combination adds nothing additional that is sufficient to amount to significantly more than the recited abstract idea because they simply provide instructions to use a generic arrangement of generic computer components to “apply” the recited abstract idea, perform insignificant extra-solution activity, and generally link the abstract idea to a technical environment. Thus, the elements of the claims, considered both individually and as an ordered combination, are not sufficient to ensure that the claim as a whole amounts to significantly more than the abstract idea itself. Since there are no limitations in these claims that transform the exception into a patent eligible application such that these claims amount to significantly more than the exception itself, claims 1-7, 9-14, 16-22 are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Claim Rejections – 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
Determining the scope and contents of the prior art.
Ascertaining the differences between the prior art and the claims at issue.
Resolving the level of ordinary skill in the pertinent art.
Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-7, 9-14, 16-22 is/are rejected under 35 U.S.C. 103 as being unpatentable by US Patent Publication to US20210272469A1 to Kennedy et al., (hereinafter referred to as “Kennedy”) in view of US Patent Publication to US20240105076A1 to Peyronnet et al., (hereinafter referred to as “Peyronnet”)
As per Claim 1, Kennedy teaches: (Currently amended) A method of evaluating a training performance of a training subject in response to a computer simulated training scenario, the training performance being performed in a training environment that simulates a physical environment and comprising a training interface configured to receive user inputs from the training subject, the method comprising: (in at least [0052] Referring to FIG. 6, a conceptual diagram 600 depicting an embodiment of embedded training for commercial aviation is depicted. As shown in the diagram 600, a learning management system 610 and mid-fidelity simulation (MFS) training scenarios 630 may be used to provide embedded training 650 via mid-fidelity simulation. The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620.)
receiving a simulation state update indicating a change in a simulation state of the computer simulated training scenario from a training computing system of the training environment; (in at least [0025] training modules that could be available, and may be adapted to long-haul flights, include: a simulated Himalaya crossing during a Dubai to Beijing flight, or a drift-down procedures module during the same flight; a North Atlantic Tracks (NAT) refresher training for flights across the Atlantic Ocean; a refresher training on volcanic ash and related procedures for routes over areas with risk for volcanic activity; and polar operations for polar routes. [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight. The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” [0036] In order to determine which training exercises may be applicable to an employee, the learning management system 110 may retrieve regulatory or audit data 142 from the regulatory or audit database 140. The regulatory or audit data 142 may indicate training exercises or concepts that are required for the employee and have not yet been performed. [0053] Once training needs have been identified, at 620, the mid-fidelity simulation training scenarios 630 may be selected and calibrated. For example, the mid-fidelity simulation training scenarios 630 may upload actual data relevant to a trainee. To illustrate, during a flight associated with the trainee, the mid-fidelity simulation training scenario 630 may upload an aircraft position, at 632, and/or upload aircraft systems data, at 634. (i.e. change) An example of using uploaded flight data in mid-fidelity simulation is further described in U.S. patent application Ser. No. 16/275,723, filed on Feb. 14, 2019 and entitled “Mid-Fidelity Simulation Approach and Method for Flight Crew Training and Evaluation,” which has been incorporated by reference herein. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
querying a behavioral elicitation event (BEE) database to determine, from a plurality of BEEs stored in the BEE database, a set of active BEEs based upon the simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions and one or more corresponding observable behaviors; (in at least [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight (i.e. elicited actions and one or more observable behaviors). The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” (i.e. elicited actions and one or more observable behaviors) [0035] The learning management system 110 may select a training exercise 160 that is applicable to the employee. As an illustrative example, the learning management system 110 may retrieve flight data 132 from the flight database 130 for a flight associated with the employee and determine the training concept 168 associated with the flight. The learning management system 110 may select a training exercise 160 that is associated with the training concept 168 from the multiple training exercises 164. (i.e. BEE) [0036] In order to determine which training exercises may be applicable to an employee, the learning management system 110 may retrieve regulatory or audit data 142 from the regulatory or audit database 140. The regulatory or audit data 142 may indicate training exercises or concepts that are required for the employee and have not yet been performed. Based on the regulatory data or audit data 142, the training exercise 160 may be selected. [0037] Selecting the training exercise applicable to the employee may alternatively or additionally include retrieving sample data 152 associated with multiple employees and determining a common training exercise 172 associated with the multiple employees. The training exercise 160 may be the common training exercise 172. Determining the common training exercise 172 associated with multiple employees may include performing an artificial intelligence analysis of a training history associated with the multiple employees using the AI learning model 170. [0052] The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. (i.e. active BEE) The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
receiving a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface; (in at least [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight. The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. [0039] The system 100 may be used in conjunction with a mid-fidelity simulator. For example, the training exercise 160 may include a mid-fidelity simulation 162. The mid-fidelity simulation 162 may provide simulations and scenarios that are common among multiple aircraft. This may enable the employee to receive conceptual scenario training as opposed to aircraft-specific training. [0040] the learning management system 110 may receive feedback 186. This may be provided as user response data retrieved during performance of the training exercise 160, as survey data after performance of the training exercise 160, as instructor review data, or another form of evaluation data. The feedback 186 may be used to modify the training exercise 160 and/or the AI learning model 170. [0040] the learning management system 110 may receive feedback 186. This may be provided as user response data retrieved during performance of the training exercise 160, as survey data after performance of the training exercise 160, as instructor review data, or another form of evaluation data. The feedback 186 may be used to modify the training exercise 160 and/or the AI learning model 170. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
for each simulation action notification, determining whether an action represented by the simulation action notification … an elicited action for at least one active BEE in the set of active BEEs and storing, in a learning record, information regarding the elicited action as a response to the at least one active BEE in the set of active BEEs; (in at least [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight. The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, (i.e. simulation action notification) such as “You need to complete this module by [a particular date]—do you want to do it now?” or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” (i.e. elicited action) [0028] provide engaging, useful, and operationally relevant interactions on the flight deck. Given the sensitivities around providing training (and potential distraction), one, or a few modules, with clear operational relevance to a flight could be offered initially. These could be deployed on a voluntary basis. Tracking of when the modules are taken can be used to identify patterns of usage and adapt the modules and system to fit the needs of the pilots. Over time, an increasing number of modules can be introduced. [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. (i.e. action, and elicited action) [0039] The system 100 may be used in conjunction with a mid-fidelity simulator. For example, the training exercise 160 may include a mid-fidelity simulation 162. The mid-fidelity simulation 162 may provide simulations and scenarios that are common among multiple aircraft. This may enable the employee to receive conceptual scenario training as opposed to aircraft-specific training. (i.e. simulation action notification) [0050] As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. (i.e. active) [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
for each active BEE, determine whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE; and (in at least [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. [0050] As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. [0051] FIG. 5, an embodiment of the sample data 152 is depicted. The sample data 152 may map employees 510-512 to training histories 520, 530, 540. For example, a first employee 510 may be mapped to a first training history 520, a second employee 511 may be mapped to a second training history 530, and a third employee 512 may be mapped to a third training history 540. The learning management system 110 of FIG. 1 may use the sample data 152 to determining a common training exercise 172 associated with the employees 510-512. The training exercise 160 may correspond to and/or may be the common training exercise 172. Determining the common training exercise 172 associated with employees 510-512 may include performing an artificial intelligence analysis of the training histories 520, 530, 540. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
query the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store information regarding the one or more corresponding observable behaviors of the completed BEE. (in at least [0052] The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it (i.e. query). For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
Although implied, Kennedy does not expressly disclose the following limitations, which however, are taught by Peyronnet,
…action … matches an elicited action for at least one active BEE in the set of active BEEs… (in at least [0067] FIG. 2 , the data correlation step 104 comprises a step 116 consisting in determining a trigger event originating from an operator, the trigger event originating from an operator is the occurrence of an event at the origin of an action of the at least one operator in response to this trigger event originating from an operator. The trigger parameter originating from an operator may thus be an endogenous datum, such as for example the initialization of technical dialog between the team of operators. However, it may also be contemplated, in a more generalized scenario, for the trigger event parameter to be an exogenous datum. [0068] The data correlation step 104 then comprises a step 118 consisting in determining a trigger event originating from the platform. The trigger event originating from the platform represents the occurrence of an event at the origin of a state of the platform, and may be interpreted as an exogenous datum. This is then followed by a step 120 of detecting trigger events originating from said at least one operator or from said platform, and a step 122 of selecting at least one trigger event. [0069] The detection 120 of a trigger event originating from an operator is based on the detection of an action from the at least one operator. The detection 120 of a trigger event originating from a platform is based on the detection of a state of the platform, such as for example a change of piloting mode, the extension or retraction of the landing gear, a fault, and on the departure from an envelope of dynamic parameters, such as for example speed, incline, attitude. [0070] In response to the trigger event, whether it is a trigger from the at least one operator or from the platform, the assessment method 100 then captures at least one action parameter represented by an endogenous datum and presenting the physical manifestation of a reaction of the at least one operator to the trigger parameter. Grouping together a trigger parameter, represented by an exogenous datum or an endogenous datum, and at least one action parameter, represented by endogenous data, thus makes it possible, during the correlation step 104, to generate at least one observable behavior datum reflecting, according to tangible parameters, the behavior of the at least one operator upon triggering of an event. [0079] In step 106, the assessment method 100 comprises a step consisting in analyzing the observable behavior data in predefined analysis sequences, each predefined analysis sequence being specific to a technical and non-technical skill to be assessed, and comprising at least one trigger event parameter and one action parameter for characterizing an expected observable behavior according to a predefined situation. The analysis 106 also makes it possible to generate a measurement indicator for each observed behavior. More specifically, step 106 analyzes observable behavior data under the prism of trigger event and action parameters by comparing (step 132) the detected observable behavior data with a predefined sequence defining the expected observable behavior, each predefined sequence representing at least one physical manifestation allocated to the expected behavior. The predefined sequences are contained in a correspondence database. This correspondence database thus comprises the predefined analysis sequences presenting observable behavior data known to those skilled in the art, as well as their assigned measurable and detectable physical manifestations. Each predefined analysis sequence thus comprises at least one trigger event parameter and at least one action parameter and other endogenous and exogenous data for characterizing a flight situation and a context for at least one operator, and also their expected reaction according to the predefined situation. This analysis provides the nature of the induced action, its temporal location and also its duration or its frequency. The correspondence database also comprises a reference table containing trigger event parameters associated with each behavior to be observed. [0080] in order to be able to analyze the behavior of at least one operator during a flight situation, the step 106 of analyzing the observable behavior data compares the detected endogenous and exogenous data, and more specifically the trigger event and action parameters, with the trigger event and action parameters and also the predefined endogenous and exogenous data. The predefined analysis sequences are specific to each technical and non-technical skill to be assessed. [0092] This generation of observed behavior indicators then makes it possible to initiate a step 108, shown in FIG. 1 , consisting in assessing a behavior of at least one operator. The assessment of the behavior of the at least one operator consists in comparing the observed behavior, which is based on a set of detected behavior elements, with predefined expected reference behaviors. The conformity of an observed behavior is assessed by comparison with known prior art of defined procedures or established protocols, contained in the correspondence database. [0093] The objectivity of the assessment of the technical and non-technical skills of an operator is thus based on the prior creation of the correspondence database between various observable behaviors and measurable physical variables in relation to these observable behaviors. [0094] The matching consists, for each observable behavior, in determining various ways of measuring same and then developing the tools necessary for each measurement. [0095] By way of indicative and non-exhaustive examples, various cases below are given to illustrate the matching of observable behaviors with measurable physical variables to allow behaviors to be assessed.)
At the time the invention was filed, it would have been obvious for one of ordinary skill in the art to have modified the teachings of Kennedy, as taught by Peyronnet above, with a reasonable expectation of success if arriving at the claimed invention. One of ordinary skill in the art would have been motivated to make this modification to the teachings of Kennedy with the motivation of, …analyzing the behavior of an operator in a simulation or training situation, allowing an observer to obtain statistical data that provide real-time information about the state and the behavior of the operator. Using these statistical data, the observer is able to carry out their own analysis of the technical and non-technical skills of the operator… to assist instructors in assessing these non-technical skills, the European Union Aviation Safety Agency (EASA) has published a list of Observable Behavior Indicators (OBI). These behavior indicators make it possible to objectify these various skills and provide a shared assessment framework between instructors, enabling a reduction in subjectivity in the assessment… allow the instructor to be easily directed to observable behavior data needed to assess skills…assessing and training commercial aviation pilots developed by players in the aeronautical world, based on an objective skills assessment (competency-based training)….to improve the assessment capabilities of the assessment method 100 by enriching the correspondence database for subsequent use of the assessment method…, as recited in Peyronnet.
As per Claim 2, Kennedy teaches:. (Original) The method of claim 1,
wherein each BEE of the plurality of BEEs further comprises one or more triggers, and (in at least [0026] generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” [0052] The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620. [0053] Once training needs have been identified, at 620, the mid-fidelity simulation training scenarios 630 may be selected and calibrated. For example, the mid-fidelity simulation training scenarios 630 may upload actual data relevant to a trainee.)
wherein querying the BEE database to determine the set of active BEEs comprises,
querying the BEE database for one or more triggers that have a condition matching the simulation state update, (in at least [0052] The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620. [0053] Once training needs have been identified, at 620, the mid-fidelity simulation training scenarios 630 may be selected and calibrated. For example, the mid-fidelity simulation training scenarios 630 may upload actual data relevant to a trainee.)
storing the one or more triggers as a set of active triggers, and (in at least [0050] As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. [0052] The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed.)
querying the BEE database to determine whether any BEEs are active BEEs based on the set of active triggers. (in at least [0050] As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. [0052] The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed.)
As per Claim 3, Kennedy teaches: (Original) The method of claim 2, wherein querying the BEE database to determine the set of active BEEs further comprises
determining one or more inactive triggers in the set of active triggers by comparing a corresponding condition for each active trigger in the set of active triggers to the simulation state update, and (in at least [0050] As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. [0052] The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed.)
removing the one or more inactive triggers from the set of active triggers. (in at least [0050] As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. [0052] The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed.)
As per Claim 4, Kennedy teaches: (Previously presented) The method of claim 1, further comprising,
comparing a prior set of active BEEs to the set of active BEEs, and for each active BEE in the prior set of active BEEs and not in the set of active BEEs, storing the active BEE as an incomplete BEE and also storing, in the learning record, information regarding the training performance, the information comprising one or more missed elicited actions in response to the incomplete BEE. (in at least [0026] suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight. The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” (i.e. missed and incomplete) [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. [0041] Using the system 100, embedded training could be tracked, similar to other recurrent training, as part of a broad learning management plan. [0050] Regulatory authorities (such as the Federal Aviation Administration) may require that the employees 410-412 receive certain training and may perform audits to ensure that it has been done. Likewise, individual airlines or air service providers may require certain training. As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. [0051] FIG. 5, an embodiment of the sample data 152 is depicted. The sample data 152 may map employees 510-512 to training histories 520, 530, 540. For example, a first employee 510 may be mapped to a first training history 520, a second employee 511 may be mapped to a second training history 530, and a third employee 512 may be mapped to a third training history 540. The learning management system 110 of FIG. 1 may use the sample data 152 to determining a common training exercise 172 associated with the employees 510-512. The training exercise 160 may correspond to and/or may be the common training exercise 172. Determining the common training exercise 172 associated with employees 510-512 may include performing an artificial intelligence analysis of the training histories 520, 530, 540.)
As per Claim 5, Kennedy teaches: (Previously presented) The method of claim 1, further comprising, when the action represented by the simulation action notification … the elicited action for the at least one active BEE in the set of active BEEs,
determining whether the elicited action has a timing condition, and storing, in the learning record, information regarding a timing of the simulation action notification. (in at least [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight. The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” (i.e. timing condition) or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” [0027] In parallel with the previous examples, it may be desirable to have, for example, a drift-down simulation scenario for flights over the Himalayas, a volcanic ash scenario for flights passing over volcanic areas, a diversion scenario for flights over polar areas, and a loss of communication scenario for flights at risk for communication loss. All of these can have a briefing and debriefing portion. The simulations may be more effective in keeping the recipient's attention by providing information that is relevant to a current or impending flight. [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. [0041] Using the system 100, embedded training could be tracked, similar to other recurrent training, as part of a broad learning management plan. [0050] Regulatory authorities (such as the Federal Aviation Administration) may require that the employees 410-412 receive certain training and may perform audits to ensure that it has been done. Likewise, individual airlines or air service providers may require certain training. As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. [0051] FIG. 5, an embodiment of the sample data 152 is depicted. The sample data 152 may map employees 510-512 to training histories 520, 530, 540. For example, a first employee 510 may be mapped to a first training history 520, a second employee 511 may be mapped to a second training history 530, and a third employee 512 may be mapped to a third training history 540. The learning management system 110 of FIG. 1 may use the sample data 152 to determining a common training exercise 172 associated with the employees 510-512. The training exercise 160 may correspond to and/or may be the common training exercise 172. Determining the common training exercise 172 associated with employees 510-512 may include performing an artificial intelligence analysis of the training histories 520, 530, 540.)
Although implied, Kennedy does not expressly disclose the following limitations, which however, are taught by Peyronnet,
…action … matches an elicited action for at least one active BEE in the set of active BEEs… (in at least [0066] Each observable behavior datum therefore comprises at least one parameter called a trigger event parameter and one parameter called an action parameter. The trigger event parameter represents the action that is at the origin of a potential reaction of the at least one operator and their conduct, represented by at least one action parameter. A trigger event may also be an exceeded time delay, as part of an ongoing procedure. [0067] FIG. 2 , the data correlation step 104 comprises a step 116 consisting in determining a trigger event originating from an operator, the trigger event originating from an operator is the occurrence of an event at the origin of an action of the at least one operator in response to this trigger event originating from an operator. The trigger parameter originating from an operator may thus be an endogenous datum, such as for example the initialization of technical dialog between the team of operators. However, it may also be contemplated, in a more generalized scenario, for the trigger event parameter to be an exogenous datum. [0068] The data correlation step 104 then comprises a step 118 consisting in determining a trigger event originating from the platform. The trigger event originating from the platform represents the occurrence of an event at the origin of a state of the platform, and may be interpreted as an exogenous datum. This is then followed by a step 120 of detecting trigger events originating from said at least one operator or from said platform, and a step 122 of selecting at least one trigger event. [0069] The detection 120 of a trigger event originating from an operator is based on the detection of an action from the at least one operator. The detection 120 of a trigger event originating from a platform is based on the detection of a state of the platform, such as for example a change of piloting mode, the extension or retraction of the landing gear, a fault, and on the departure from an envelope of dynamic parameters, such as for example speed, incline, attitude. [0070] In response to the trigger event, whether it is a trigger from the at least one operator or from the platform, the assessment method 100 then captures at least one action parameter represented by an endogenous datum and presenting the physical manifestation of a reaction of the at least one operator to the trigger parameter. Grouping together a trigger parameter, represented by an exogenous datum or an endogenous datum, and at least one action parameter, represented by endogenous data, thus makes it possible, during the correlation step 104, to generate at least one observable behavior datum reflecting, according to tangible parameters, the behavior of the at least one operator upon triggering of an event. [0079] In step 106, the assessment method 100 comprises a step consisting in analyzing the observable behavior data in predefined analysis sequences, each predefined analysis sequence being specific to a technical and non-technical skill to be assessed, and comprising at least one trigger event parameter and one action parameter for characterizing an expected observable behavior according to a predefined situation. The analysis 106 also makes it possible to generate a measurement indicator for each observed behavior. More specifically, step 106 analyzes observable behavior data under the prism of trigger event and action parameters by comparing (step 132) the detected observable behavior data with a predefined sequence defining the expected observable behavior, each predefined sequence representing at least one physical manifestation allocated to the expected behavior. The predefined sequences are contained in a correspondence database. This correspondence database thus comprises the predefined analysis sequences presenting observable behavior data known to those skilled in the art, as well as their assigned measurable and detectable physical manifestations. Each predefined analysis sequence thus comprises at least one trigger event parameter and at least one action parameter and other endogenous and exogenous data for characterizing a flight situation and a context for at least one operator, and also their expected reaction according to the predefined situation. This analysis provides the nature of the induced action, its temporal location and also its duration or its frequency. The correspondence database also comprises a reference table containing trigger event parameters associated with each behavior to be observed. [0080] in order to be able to analyze the behavior of at least one operator during a flight situation, the step 106 of analyzing the observable behavior data compares the detected endogenous and exogenous data, and more specifically the trigger event and action parameters, with the trigger event and action parameters and also the predefined endogenous and exogenous data. The predefined analysis sequences are specific to each technical and non-technical skill to be assessed. [0092] This generation of observed behavior indicators then makes it possible to initiate a step 108, shown in FIG. 1 , consisting in assessing a behavior of at least one operator. The assessment of the behavior of the at least one operator consists in comparing the observed behavior, which is based on a set of detected behavior elements, with predefined expected reference behaviors. The conformity of an observed behavior is assessed by comparison with known prior art of defined procedures or established protocols, contained in the correspondence database. [0093] The objectivity of the assessment of the technical and non-technical skills of an operator is thus based on the prior creation of the correspondence database between various observable behaviors and measurable physical variables in relation to these observable behaviors. [0094] The matching consists, for each observable behavior, in determining various ways of measuring same and then developing the tools necessary for each measurement. [0095] By way of indicative and non-exhaustive examples, various cases below are given to illustrate the matching of observable behaviors with measurable physical variables to allow behaviors to be assessed.)
The reason and rationale to combine Kennedy and Peyronnet is the same as recited above.
As per Claim 6, Kennedy teaches: (Original) The method of claim 5, further comprising
comparing the timing of the simulation action notification with a timing of a corresponding trigger of the at least one active BEE, and storing information regarding the comparison of the timings. (in at least [0026] provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time.)
As per Claim 7, Kennedy teaches: (Currently amended) The method of claim 1,
wherein the simulation state indicates comprises a change in one or more of a simulated flight phase, simulated weather, or a state of a simulated aircraft. (in at least [0025] training modules that could be available, and may be adapted to long-haul flights, include: a simulated Himalaya crossing during a Dubai to Beijing flight, or a drift-down procedures module during the same flight; a North Atlantic Tracks (NAT) refresher training for flights across the Atlantic Ocean; a refresher training on volcanic ash and related procedures for routes over areas with risk for volcanic activity; and polar operations for polar routes. [0027] In parallel with the previous examples, it may be desirable to have, for example, a drift-down simulation scenario for flights over the Himalayas, a volcanic ash scenario for flights passing over volcanic areas, a diversion scenario for flights over polar areas, and a loss of communication scenario for flights at risk for communication loss. All of these can have a briefing and debriefing portion. The simulations may be more effective in keeping the recipient's attention by providing information that is relevant to a current or impending flight. [0028] provide engaging, useful, and operationally relevant interactions on the flight deck. Given the sensitivities around providing training (and potential distraction), one, or a few modules, with clear operational relevance to a flight could be offered initially. These could be deployed on a voluntary basis. Tracking of when the modules are taken can be used to identify patterns of usage and adapt the modules and system to fit the needs of the pilots. Over time, an increasing number of modules can be introduced.)
As per Claim 9, Kennedy teaches: (Currently amended) A system comprising:
a training environment configured to simulate a physical environment, the training environment comprising (in at least [0052] FIG. 6, a conceptual diagram 600 depicting an embodiment of embedded training for commercial aviation is depicted. As shown in the diagram 600, a learning management system 610 and mid-fidelity simulation (MFS) training scenarios 630 may be used to provide embedded training 650 via mid-fidelity simulation)
a training computing system configured to perform a simulated training scenario and send a simulation state update indicating a change in a simulated state of the simulated training scenario, and (in at least [0025] training modules that could be available, and may be adapted to long-haul flights, include: a simulated Himalaya crossing during a Dubai to Beijing flight, or a drift-down procedures module during the same flight; a North Atlantic Tracks (NAT) refresher training for flights across the Atlantic Ocean; a refresher training on volcanic ash and related procedures for routes over areas with risk for volcanic activity; and polar operations for polar routes. [0027] In parallel with the previous examples, it may be desirable to have, for example, a drift-down simulation scenario for flights over the Himalayas, a volcanic ash scenario for flights passing over volcanic areas, a diversion scenario for flights over polar areas, and a loss of communication scenario for flights at risk for communication loss. All of these can have a briefing and debriefing portion. The simulations may be more effective in keeping the recipient's attention by providing information that is relevant to a current or impending flight. [0052] Referring to FIG. 6, a conceptual diagram 600 depicting an embodiment of embedded training for commercial aviation is depicted. As shown in the diagram 600, a learning management system 610 and mid-fidelity simulation (MFS) training scenarios 630 may be used to provide embedded training 650 via mid-fidelity simulation. The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620.)
a training interface configured to receive user inputs from a training subject in response to the simulated training scenario; and (in at least [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight. The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” (i.e. reminder in response to receiving a user input of declining or delaying training) or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. [0039] The system 100 may be used in conjunction with a mid-fidelity simulator. For example, the training exercise 160 may include a mid-fidelity simulation 162. The mid-fidelity simulation 162 may provide simulations and scenarios that are common among multiple aircraft. This may enable the employee to receive conceptual scenario training as opposed to aircraft-specific training. [0052] Referring to FIG. 6, a conceptual diagram 600 depicting an embodiment of embedded training for commercial aviation is depicted. As shown in the diagram 600, a learning management system 610 and mid-fidelity simulation (MFS) training scenarios 630 may be used to provide embedded training 650 via mid-fidelity simulation. The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620.)
a computing system comprising a logic system operably coupled to the training computing system, and (in at least [0052] Referring to FIG. 6, a conceptual diagram 600 depicting an embodiment of embedded training for commercial aviation is depicted. As shown in the diagram 600, a learning management system 610 and mid-fidelity simulation (MFS) training scenarios 630 may be used to provide embedded training 650 via mid-fidelity simulation. The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620.)
a computer-readable storage system comprising, a behavioral elicitation event (BEE) database comprising a plurality of BEEs, each BEE of the plurality of BEEs comprising one or more elicited actions …, and one or more corresponding observable behaviors, and (in at least [0049] FIG. 4, an embodiment of the regulatory or audit data 142 is depicted. The regulatory or audit data 142 may map employees 410-412 to various required or preferred training exercises 420-422, 430, 440, 441. For example, a first employee 410 may be mapped to a first set of required or preferred training exercises 410-422, a second employee may be mapped to a second required or preferred training exercise 430, and a third employee 412 may be mapped to a third set of required or preferred training exercises 440, 441. [0052] Referring to FIG. 6, a conceptual diagram 600 depicting an embodiment of embedded training for commercial aviation is depicted. As shown in the diagram 600, a learning management system 610 and mid-fidelity simulation (MFS) training scenarios 630 may be used to provide embedded training 650 via mid-fidelity simulation. The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620.)
instructions executable by the logic system to
receive a simulation state update indicating a change in the simulation state of the simulated training scenario from the training computing system, (in at least [0055] If the trainee is available for a training module, the training module may be uploaded to the trainee's electronic device, at 640. In the depicted embodiment, the electronic device may be an iPad, or a similar portable electronic device, such as an electronic flight book. The trainee may then perform the embedded training 650. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
query the BEE database to determine, from the plurality of BEEs, a set of active BEEs based upon the simulation state update, (in at least [0025] training modules that could be available, and may be adapted to long-haul flights, include: a simulated Himalaya crossing during a Dubai to Beijing flight, or a drift-down procedures module during the same flight; a North Atlantic Tracks (NAT) refresher training for flights across the Atlantic Ocean; a refresher training on volcanic ash and related procedures for routes over areas with risk for volcanic activity; and polar operations for polar routes. [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight. The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” [0036] In order to determine which training exercises may be applicable to an employee, the learning management system 110 may retrieve regulatory or audit data 142 from the regulatory or audit database 140. The regulatory or audit data 142 may indicate training exercises or concepts that are required for the employee and have not yet been performed. [0053] Once training needs have been identified, at 620, the mid-fidelity simulation training scenarios 630 may be selected and calibrated. For example, the mid-fidelity simulation training scenarios 630 may upload actual data relevant to a trainee. To illustrate, during a flight associated with the trainee, the mid-fidelity simulation training scenario 630 may upload an aircraft position, at 632, and/or upload aircraft systems data, at 634. An example of using uploaded flight data in mid-fidelity simulation is further described in U.S. patent application Ser. No. 16/275,723, filed on Feb. 14, 2019 and entitled “Mid-Fidelity Simulation Approach and Method for Flight Crew Training and Evaluation,” which has been incorporated by reference herein. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
receive a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface, (in at least [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight. The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. [0039] The system 100 may be used in conjunction with a mid-fidelity simulator. For example, the training exercise 160 may include a mid-fidelity simulation 162. The mid-fidelity simulation 162 may provide simulations and scenarios that are common among multiple aircraft. This may enable the employee to receive conceptual scenario training as opposed to aircraft-specific training. [0040] the learning management system 110 may receive feedback 186. This may be provided as user response data retrieved during performance of the training exercise 160, as survey data after performance of the training exercise 160, as instructor review data, or another form of evaluation data. The feedback 186 may be used to modify the training exercise 160 and/or the AI learning model 170. [0040] the learning management system 110 may receive feedback 186. This may be provided as user response data retrieved during performance of the training exercise 160, as survey data after performance of the training exercise 160, as instructor review data, or another form of evaluation data. The feedback 186 may be used to modify the training exercise 160 and/or the AI learning model 170. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
for each simulation action notification, determine whether an action represented by the simulation action notification … an elicited action for at least one active BEE in the set of active BEEs, and store, in the learning record, information regarding the elicited action as the response to the at least one active BEE in the set of active BEEs, (in at least [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight. The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, (i.e. simulation action notification) such as “You need to complete this module by [a particular date]—do you want to do it now?” or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” (i.e. elicited action) [0028] provide engaging, useful, and operationally relevant interactions on the flight deck. Given the sensitivities around providing training (and potential distraction), one, or a few modules, with clear operational relevance to a flight could be offered initially. These could be deployed on a voluntary basis. Tracking of when the modules are taken can be used to identify patterns of usage and adapt the modules and system to fit the needs of the pilots. Over time, an increasing number of modules can be introduced. [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. (i.e. action, and elicited action) [0050] As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. (i.e. active) [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
for each active BEE, determine whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE, and (in at least [0028] provide engaging, useful, and operationally relevant interactions on the flight deck. Given the sensitivities around providing training (and potential distraction), one, or a few modules, with clear operational relevance to a flight could be offered initially. These could be deployed on a voluntary basis. Tracking of when the modules are taken can be used to identify patterns of usage and adapt the modules and system to fit the needs of the pilots. Over time, an increasing number of modules can be introduced. [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. [0050] As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. [0051] FIG. 5, an embodiment of the sample data 152 is depicted. The sample data 152 may map employees 510-512 to training histories 520, 530, 540. For example, a first employee 510 may be mapped to a first training history 520, a second employee 511 may be mapped to a second training history 530, and a third employee 512 may be mapped to a third training history 540. The learning management system 110 of FIG. 1 may use the sample data 152 to determining a common training exercise 172 associated with the employees 510-512. The training exercise 160 may correspond to and/or may be the common training exercise 172. Determining the common training exercise 172 associated with employees 510-512 may include performing an artificial intelligence analysis of the training histories 520, 530, 540. [0052] The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620.)
query the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store, in the learning record, information regarding the corresponding observable behaviors of the completed BEE. (in at least [0052] The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it (i.e. query). For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
Although implied, Kennedy does not expressly disclose the following limitations, which however, are taught by Peyronnet,
… defining expected user inputs as a response to the BEE… (in at least [0067] FIG. 2 , the data correlation step 104 comprises a step 116 consisting in determining a trigger event originating from an operator, the trigger event originating from an operator is the occurrence of an event at the origin of an action of the at least one operator in response to this trigger event originating from an operator. The trigger parameter originating from an operator may thus be an endogenous datum, such as for example the initialization of technical dialog between the team of operators. However, it may also be contemplated, in a more generalized scenario, for the trigger event parameter to be an exogenous datum. [0068] The data correlation step 104 then comprises a step 118 consisting in determining a trigger event originating from the platform. The trigger event originating from the platform represents the occurrence of an event at the origin of a state of the platform, and may be interpreted as an exogenous datum. This is then followed by a step 120 of detecting trigger events originating from said at least one operator or from said platform, and a step 122 of selecting at least one trigger event. [0069] The detection 120 of a trigger event originating from an operator is based on the detection of an action from the at least one operator. The detection 120 of a trigger event originating from a platform is based on the detection of a state of the platform, such as for example a change of piloting mode, the extension or retraction of the landing gear, a fault, and on the departure from an envelope of dynamic parameters, such as for example speed, incline, attitude. [0070] In response to the trigger event, whether it is a trigger from the at least one operator or from the platform, the assessment method 100 then captures at least one action parameter represented by an endogenous datum and presenting the physical manifestation of a reaction of the at least one operator to the trigger parameter. Grouping together a trigger parameter, represented by an exogenous datum or an endogenous datum, and at least one action parameter, represented by endogenous data, thus makes it possible, during the correlation step 104, to generate at least one observable behavior datum reflecting, according to tangible parameters, the behavior of the at least one operator upon triggering of an event. [0079] In step 106, the assessment method 100 comprises a step consisting in analyzing the observable behavior data in predefined analysis sequences, each predefined analysis sequence being specific to a technical and non-technical skill to be assessed, and comprising at least one trigger event parameter and one action parameter for characterizing an expected observable behavior according to a predefined situation. The analysis 106 also makes it possible to generate a measurement indicator for each observed behavior. More specifically, step 106 analyzes observable behavior data under the prism of trigger event and action parameters by comparing (step 132) the detected observable behavior data with a predefined sequence defining the expected observable behavior, each predefined sequence representing at least one physical manifestation allocated to the expected behavior. The predefined sequences are contained in a correspondence database. This correspondence database thus comprises the predefined analysis sequences presenting observable behavior data known to those skilled in the art, as well as their assigned measurable and detectable physical manifestations. Each predefined analysis sequence thus comprises at least one trigger event parameter and at least one action parameter and other endogenous and exogenous data for characterizing a flight situation and a context for at least one operator, and also their expected reaction according to the predefined situation. This analysis provides the nature of the induced action, its temporal location and also its duration or its frequency. The correspondence database also comprises a reference table containing trigger event parameters associated with each behavior to be observed. [0080] in order to be able to analyze the behavior of at least one operator during a flight situation, the step 106 of analyzing the observable behavior data compares the detected endogenous and exogenous data, and more specifically the trigger event and action parameters, with the trigger event and action parameters and also the predefined endogenous and exogenous data. The predefined analysis sequences are specific to each technical and non-technical skill to be assessed. [0092] This generation of observed behavior indicators then makes it possible to initiate a step 108, shown in FIG. 1 , consisting in assessing a behavior of at least one operator. The assessment of the behavior of the at least one operator consists in comparing the observed behavior, which is based on a set of detected behavior elements, with predefined expected reference behaviors. The conformity of an observed behavior is assessed by comparison with known prior art of defined procedures or established protocols, contained in the correspondence database. [0093] The objectivity of the assessment of the technical and non-technical skills of an operator is thus based on the prior creation of the correspondence database between various observable behaviors and measurable physical variables in relation to these observable behaviors. [0094] The matching consists, for each observable behavior, in determining various ways of measuring same and then developing the tools necessary for each measurement. [0095] By way of indicative and non-exhaustive examples, various cases below are given to illustrate the matching of observable behaviors with measurable physical variables to allow behaviors to be assessed.)
…action … matches an elicited action for at least one active BEE in the set of active BEEs… (in at least [0067] FIG. 2 , the data correlation step 104 comprises a step 116 consisting in determining a trigger event originating from an operator, the trigger event originating from an operator is the occurrence of an event at the origin of an action of the at least one operator in response to this trigger event originating from an operator. The trigger parameter originating from an operator may thus be an endogenous datum, such as for example the initialization of technical dialog between the team of operators. However, it may also be contemplated, in a more generalized scenario, for the trigger event parameter to be an exogenous datum. [0068] The data correlation step 104 then comprises a step 118 consisting in determining a trigger event originating from the platform. The trigger event originating from the platform represents the occurrence of an event at the origin of a state of the platform, and may be interpreted as an exogenous datum. This is then followed by a step 120 of detecting trigger events originating from said at least one operator or from said platform, and a step 122 of selecting at least one trigger event. [0069] The detection 120 of a trigger event originating from an operator is based on the detection of an action from the at least one operator. The detection 120 of a trigger event originating from a platform is based on the detection of a state of the platform, such as for example a change of piloting mode, the extension or retraction of the landing gear, a fault, and on the departure from an envelope of dynamic parameters, such as for example speed, incline, attitude. [0070] In response to the trigger event, whether it is a trigger from the at least one operator or from the platform, the assessment method 100 then captures at least one action parameter represented by an endogenous datum and presenting the physical manifestation of a reaction of the at least one operator to the trigger parameter. Grouping together a trigger parameter, represented by an exogenous datum or an endogenous datum, and at least one action parameter, represented by endogenous data, thus makes it possible, during the correlation step 104, to generate at least one observable behavior datum reflecting, according to tangible parameters, the behavior of the at least one operator upon triggering of an event. [0079] In step 106, the assessment method 100 comprises a step consisting in analyzing the observable behavior data in predefined analysis sequences, each predefined analysis sequence being specific to a technical and non-technical skill to be assessed, and comprising at least one trigger event parameter and one action parameter for characterizing an expected observable behavior according to a predefined situation. The analysis 106 also makes it possible to generate a measurement indicator for each observed behavior. More specifically, step 106 analyzes observable behavior data under the prism of trigger event and action parameters by comparing (step 132) the detected observable behavior data with a predefined sequence defining the expected observable behavior, each predefined sequence representing at least one physical manifestation allocated to the expected behavior. The predefined sequences are contained in a correspondence database. This correspondence database thus comprises the predefined analysis sequences presenting observable behavior data known to those skilled in the art, as well as their assigned measurable and detectable physical manifestations. Each predefined analysis sequence thus comprises at least one trigger event parameter and at least one action parameter and other endogenous and exogenous data for characterizing a flight situation and a context for at least one operator, and also their expected reaction according to the predefined situation. This analysis provides the nature of the induced action, its temporal location and also its duration or its frequency. The correspondence database also comprises a reference table containing trigger event parameters associated with each behavior to be observed. [0080] in order to be able to analyze the behavior of at least one operator during a flight situation, the step 106 of analyzing the observable behavior data compares the detected endogenous and exogenous data, and more specifically the trigger event and action parameters, with the trigger event and action parameters and also the predefined endogenous and exogenous data. The predefined analysis sequences are specific to each technical and non-technical skill to be assessed. [0092] This generation of observed behavior indicators then makes it possible to initiate a step 108, shown in FIG. 1 , consisting in assessing a behavior of at least one operator. The assessment of the behavior of the at least one operator consists in comparing the observed behavior, which is based on a set of detected behavior elements, with predefined expected reference behaviors. The conformity of an observed behavior is assessed by comparison with known prior art of defined procedures or established protocols, contained in the correspondence database. [0093] The objectivity of the assessment of the technical and non-technical skills of an operator is thus based on the prior creation of the correspondence database between various observable behaviors and measurable physical variables in relation to these observable behaviors. [0094] The matching consists, for each observable behavior, in determining various ways of measuring same and then developing the tools necessary for each measurement. [0095] By way of indicative and non-exhaustive examples, various cases below are given to illustrate the matching of observable behaviors with measurable physical variables to allow behaviors to be assessed.)
The reason and rationale to combine Kennedy and Peyronnet is the same as recited above.
As per Claim 16, Kennedy teaches: (Currently amended) A method of evaluating a training performance of a training subject in response to a computer simulated training scenario, the training performance being performed in a training environment that simulates a physical environment and comprising a training interface configured to receive user inputs from the training subject, the method comprising: (in at least [0052] Referring to FIG. 6, a conceptual diagram 600 depicting an embodiment of embedded training for commercial aviation is depicted. As shown in the diagram 600, a learning management system 610 and mid-fidelity simulation (MFS) training scenarios 630 may be used to provide embedded training 650 via mid-fidelity simulation. The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620.)
receiving a first simulation state update from a training computing system of the training environment, the first simulation state update indicating a first change in a simulation state of the simulated training scenario; (in at least [0025] training modules that could be available, and may be adapted to long-haul flights, include: a simulated Himalaya crossing during a Dubai to Beijing flight, or a drift-down procedures module during the same flight; a North Atlantic Tracks (NAT) refresher training for flights across the Atlantic Ocean; a refresher training on volcanic ash and related procedures for routes over areas with risk for volcanic activity; and polar operations for polar routes. [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight. The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” [0036] In order to determine which training exercises may be applicable to an employee, the learning management system 110 may retrieve regulatory or audit data 142 from the regulatory or audit database 140. The regulatory or audit data 142 may indicate training exercises or concepts that are required for the employee and have not yet been performed. [0052] mid-fidelity simulation (MFS) training scenarios 630 may be used to provide embedded training 650 via mid-fidelity simulation [0053] Once training needs have been identified, at 620, the mid-fidelity simulation training scenarios 630 may be selected and calibrated. For example, the mid-fidelity simulation training scenarios 630 may upload actual data relevant to a trainee. To illustrate, during a flight associated with the trainee, the mid-fidelity simulation training scenario 630 may upload an aircraft position, at 632, and/or upload aircraft systems data, at 634 (i.e. change). An example of using uploaded flight data in mid-fidelity simulation is further described in U.S. patent application Ser. No. 16/275,723, filed on Feb. 14, 2019 and entitled “Mid-Fidelity Simulation Approach and Method for Flight Crew Training and Evaluation,” which has been incorporated by reference herein. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
querying a behavioral elicitation event (BEE) database to determine, from a plurality of BEEs stored in the BEE database, a first set of active BEEs based upon the first simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions …, and one or more corresponding observable behaviors; (in at least [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight (i.e. elicited actions and one or more observable behaviors). The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” (i.e. elicited actions and one or more observable behaviors) [0035] The learning management system 110 may select a training exercise 160 that is applicable to the employee. As an illustrative example, the learning management system 110 may retrieve flight data 132 from the flight database 130 for a flight associated with the employee and determine the training concept 168 associated with the flight. The learning management system 110 may select a training exercise 160 that is associated with the training concept 168 from the multiple training exercises 164. (i.e. BEEs) [0036] In order to determine which training exercises may be applicable to an employee, the learning management system 110 may retrieve regulatory or audit data 142 from the regulatory or audit database 140. The regulatory or audit data 142 may indicate training exercises or concepts that are required for the employee and have not yet been performed. Based on the regulatory data or audit data 142, the training exercise 160 may be selected. [0037] Selecting the training exercise applicable to the employee may alternatively or additionally include retrieving sample data 152 associated with multiple employees and determining a common training exercise 172 associated with the multiple employees. The training exercise 160 may be the common training exercise 172. Determining the common training exercise 172 associated with multiple employees may include performing an artificial intelligence analysis of a training history associated with the multiple employees using the AI learning model 170. [0052] The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. (i.e. active BEE) The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
receiving a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface; (in at least [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight. The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. [0039] The system 100 may be used in conjunction with a mid-fidelity simulator. For example, the training exercise 160 may include a mid-fidelity simulation 162. The mid-fidelity simulation 162 may provide simulations and scenarios that are common among multiple aircraft. This may enable the employee to receive conceptual scenario training as opposed to aircraft-specific training. [0040] the learning management system 110 may receive feedback 186. This may be provided as user response data retrieved during performance of the training exercise 160, as survey data after performance of the training exercise 160, as instructor review data, or another form of evaluation data. The feedback 186 may be used to modify the training exercise 160 and/or the AI learning model 170. [0040] the learning management system 110 may receive feedback 186. This may be provided as user response data retrieved during performance of the training exercise 160, as survey data after performance of the training exercise 160, as instructor review data, or another form of evaluation data. The feedback 186 may be used to modify the training exercise 160 and/or the AI learning model 170. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
for each simulation action notification, determining whether an action represented by the simulation action notification … an elicited action for at least one active BEE in the first set of active BEEs and storing, in a learning record, information regarding the elicited action and corresponding observable behaviors as the response to the at least one active BEE in the first set of active BEEs; (in at least [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight. The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, (i.e. simulation action notification) such as “You need to complete this module by [a particular date]—do you want to do it now?” or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” (i.e. elicited action) [0028] provide engaging, useful, and operationally relevant interactions on the flight deck. Given the sensitivities around providing training (and potential distraction), one, or a few modules, with clear operational relevance to a flight could be offered initially. These could be deployed on a voluntary basis. Tracking of when the modules are taken can be used to identify patterns of usage and adapt the modules and system to fit the needs of the pilots. Over time, an increasing number of modules can be introduced. [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. (i.e. action, and elicited action) [0050] As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. (i.e. active) [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
receiving a second simulation state update from the training computing system of the training environment, the second simulation state update indicating a second change in the simulation state of the simulated training scenario; (in at least [0025] training modules that could be available, and may be adapted to long-haul flights, include: a simulated Himalaya crossing during a Dubai to Beijing flight, or a drift-down procedures module during the same flight; a North Atlantic Tracks (NAT) refresher training for flights across the Atlantic Ocean; a refresher training on volcanic ash and related procedures for routes over areas with risk for volcanic activity; and polar operations for polar routes. [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight. The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” or “This flight will pass over active volcanic areas—do you want more information on this?” or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” [0036] In order to determine which training exercises may be applicable to an employee, the learning management system 110 may retrieve regulatory or audit data 142 from the regulatory or audit database 140. The regulatory or audit data 142 may indicate training exercises or concepts that are required for the employee and have not yet been performed. [0053] Once training needs have been identified, at 620, the mid-fidelity simulation training scenarios 630 may be selected and calibrated. For example, the mid-fidelity simulation training scenarios 630 may upload actual data relevant to a trainee. To illustrate, during a flight associated with the trainee, the mid-fidelity simulation training scenario 630 may upload an aircraft position, at 632, and/or upload aircraft systems data, at 634 (i.e. second state update, second change). An example of using uploaded flight data in mid-fidelity simulation is further described in U.S. patent application Ser. No. 16/275,723, filed on Feb. 14, 2019 and entitled “Mid-Fidelity Simulation Approach and Method for Flight Crew Training and Evaluation,” which has been incorporated by reference herein. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
querying the BEE database to determine a second set of active BEEs based upon the second simulation state update, the first set of active BEEs including at least one active BEE that is not in the second set of active BEEs; (in at least [0026] The modules could be part of an overall learning management system that tracks pilots' training and examinations for regulatory and audit purposes. When linked to a rostering system, suggestions to pilots could be sent out before a flight and reminders provided for the flight. As an example, this system could remind a pilot that he has not done an Atlantic crossing in the last few months and offer the module as a refresher before or in flight (i.e. elicited actions and one or more observable behaviors). The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” (i.e. first active BEE) (i.e. missed and incomplete) or “This flight will pass over active volcanic areas—do you want more information on this?” (i.e. second active BEE, queried when flight will over volcanos or others) or “You have not passed over the Himalayas recently—do you want to have an update on this?” or “Many pilots have recently looked at the module on new approach procedures for Hong Kong—would you like to take this module?” (i.e. elicited actions and one or more observable behaviors) [0035] The learning management system 110 may select a training exercise 160 that is applicable to the employee. As an illustrative example, the learning management system 110 may retrieve flight data 132 from the flight database 130 for a flight associated with the employee and determine the training concept 168 associated with the flight. The learning management system 110 may select a training exercise 160 that is associated with the training concept 168 from the multiple training exercises 164. (i.e. BEE) [0036] In order to determine which training exercises may be applicable to an employee, the learning management system 110 may retrieve regulatory or audit data 142 from the regulatory or audit database 140. The regulatory or audit data 142 may indicate training exercises or concepts that are required for the employee and have not yet been performed. Based on the regulatory data or audit data 142, the training exercise 160 may be selected. [0037] Selecting the training exercise applicable to the employee may alternatively or additionally include retrieving sample data 152 associated with multiple employees and determining a common training exercise 172 associated with the multiple employees. The training exercise 160 may be the common training exercise 172. Determining the common training exercise 172 associated with multiple employees may include performing an artificial intelligence analysis of a training history associated with the multiple employees using the AI learning model 170. [0038] Once the training exercise 160 has been selected, the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time. [0041] Using the system 100, embedded training could be tracked, similar to other recurrent training, as part of a broad learning management plan. A benefit of the system 100 is that short periods of time available in operations, in flight or on the ground, may be used for training, and the training may be more closely linked to current operations. Another benefit for pilots would be a lower recurrent training load to handle on their own time. For both pilots and airlines, the provision of operationally relevant information at relevant times may increase situation awareness and decrease operational risk. [0042] The embedded training component may also rely on artificial-intelligence-driven systems that may track, propose, and recommend training based on operational risk and training evidence. Although regulatory requirements will remain necessary to fulfill, the road to do so can be traced along actual operational knowledge needs as determined through artificial intelligence. With the embedded training available through the system 100, regulatory training needs can be met when they are operationally relevant. For example, a ten-minute refresher module on hydraulic systems is probably more relevant when a pilot is sitting on the flight deck with the system in front of them. [0043] FIG. 2, an embodiment of the roster data 122 is depicted. The roster data 122 may associate, or otherwise map, employees to schedules. To illustrate, a first employee 210 may be associated with various scheduled events such as a layover 222, operational times 223, 225, 227, available time 224, and a downtime 226. For example, the first employee 210 may be a pilot and the operational times 223, 225, 227, the available time 224, and the downtime 226 may be associated with a flight 220. A first operational time 223 may be associated with a takeoff, a second operational time 225 may be associated with an in-flight piloting activity, such as a course change, etc., a third operational time 227 may be associated with a landing. During long-haul flights, there may be available time 224 where, when permitted by flight regulations, a pilot may have some free time available for training. Other times, a pilot may have scheduled downtime 226 during the flight 220 and may perform training then. [0048] FIG. 3, an embodiment of the flight data 132 is depicted. The flight data 132 may associate, or otherwise map, flights 311, 312, with timelines 320, 330. For example, a first flight 311 may be mapped to a timeline 320 that includes aviation events such as an ocean crossing 322 and a new approach 324 to an airport. A second flight 312 may be mapped to a timeline 330 that includes aviation events such as a mountain crossing 332. The aviation events may be the basis for training concepts (e.g., the training concept 168 of FIG. 1). The learning management system 110 of FIG. 1 may use the flight data 132 to determine the training concept 168 and to select the training exercise 160 from the multiple training exercises 164. It should be noted that the ocean crossing 322, the new approach 324, and the mountain crossing 332 are only some examples of many different types of aviation events that may be associated with flights. Other potential scenarios may include fuel contaminations, ash encounters, technical problems, diversions due to sick passengers, etc. In practice, any type of aviation event may be associated with a flight and/or scheduled for training. [0052] The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. (i.e. active BEE) The data from the crew roster 616 may enable the learning management system 610 to identify time periods when trainees may be available for training and to identify training scenarios that may be relevant to the trainee's scheduled flights. The learning management system 610 may correspond to the learning management system 110 of FIG. 1 and may rely on artificial intelligence to identify training needs, at 620. [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise (i.e. second set of active BEEs) to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training, or additional training, within a certain time frame.)
comparing the first set of active BEEs to the second set of active BEEs; and (in at least [0035] The learning management system 110 may select a training exercise 160 that is applicable to the employee. As an illustrative example, the learning management system 110 may retrieve flight data 132 from the flight database 130 for a flight associated with the employee and determine the training concept 168 associated with the flight (i.e. first active BEE). The learning management system 110 may select a training exercise 160 that is associated with the training concept 168 from the multiple training exercises 164 [0036] In order to determine which training exercises may be applicable to an employee, the learning management system 110 may retrieve regulatory or audit data 142 from the regulatory or audit database 140. The regulatory or audit data 142 may indicate training exercises or concepts that are required for the employee and have not yet been performed (i.e. second active BEE). Based on the regulatory data or audit data 142, the training exercise 160 may be selected. [0037] employee may alternatively or additionally include retrieving sample data 152 associated with multiple employees and determining a common training exercise 172 associated with the multiple employees. The training exercise 160 may be the common training exercise 172. Determining the common training exercise 172 associated with multiple employees may include performing an artificial intelligence analysis of a training history associated with the multiple employees using the AI learning model 170. [0049] FIG. 4, an embodiment of the regulatory or audit data 142 is depicted. The regulatory or audit data 142 may map employees 410-412 to various required or preferred training exercises 420-422, 430, 440, 441. For example, a first employee 410 may be mapped to a first set of required or preferred training exercises 410-422, a second employee may be mapped to a second required or preferred training exercise 430, and a third employee 412 may be mapped to a third set of required or preferred training exercises 440, 441. [0050] Regulatory authorities (such as the Federal Aviation Administration) may require that the employees 410-412 receive certain training and may perform audits to ensure that it has been done. Likewise, individual airlines or air service providers may require certain training. As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. [0051] FIG. 5, an embodiment of the sample data 152 is depicted. The sample data 152 may map employees 510-512 to training histories 520, 530, 540. For example, a first employee 510 may be mapped to a first training history 520, a second employee 511 may be mapped to a second training history 530, and a third employee 512 may be mapped to a third training history 540. The learning management system 110 of FIG. 1 may use the sample data 152 to determining a common training exercise 172 associated with the employees 510-512. The training exercise 160 may correspond to and/or may be the common training exercise 172. Determining the common training exercise 172 associated with employees 510-512 may include performing an artificial intelligence analysis of the training histories 520, 530, 540. [0052] The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed and what training still needs to be completed. [0052] The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios (i.e. first set of active BEEs) where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed (i.e. first set of active BEEs) and what training still needs to be completed. (i.e. second set of active BEEs))
for each active BEE in the first set of active BEEs and not in the second set of active BEEs, determine that the active BEE is an incomplete BEE, and store, in the learning record, information regarding the incomplete BEE and the second simulation state update. (in at least [0026] The system could then provide different levels of recommendations, based on regulatory requirements. For example, the system could generate a notification with various recommendations, such as “You need to complete this module by [a particular date]—do you want to do it now?” (i.e. active and incomplete) [0036] In order to determine which training exercises may be applicable to an employee, the learning management system 110 may retrieve regulatory or audit data 142 from the regulatory or audit database 140. The regulatory or audit data 142 may indicate training exercises or concepts that are required for the employee and have not yet been performed (i.e. second BEE). Based on the regulatory data or audit data 142, the training exercise 160 may be selected. [0037] employee may alternatively or additionally include retrieving sample data 152 associated with multiple employees and determining a common training exercise 172 associated with the multiple employees. The training exercise 160 may be the common training exercise 172. Determining the common training exercise 172 associated with multiple employees may include performing an artificial intelligence analysis of a training history associated with the multiple employees using the AI learning model 170. [0038] Once the training exercise 160 has been selected (i.e. active), the learning management system 110 may send a notification 182 to an electronic device 180 associated with the employee. The notification 182 may include an offer 184 to perform the training exercise. The employee may then have an opportunity to accept or reject the training exercise 160. If the employee rejects the training exercise 160, it may be scheduled for another time.(i.e. first set of active BEEs not in the second set of active BEEs and store as incomplete) [0048] FIG. 3, an embodiment of the flight data 132 is depicted. The flight data 132 may associate, or otherwise map, flights 311, 312, with timelines 320, 330. For example, a first flight 311 may be mapped to a timeline 320 that includes aviation events such as an ocean crossing 322 and a new approach 324 to an airport. A second flight 312 may be mapped to a timeline 330 that includes aviation events such as a mountain crossing 332. The aviation events may be the basis for training concepts (e.g., the training concept 168 of FIG. 1). The learning management system 110 of FIG. 1 may use the flight data 132 to determine the training concept 168 and to select the training exercise 160 from the multiple training exercises 164. It should be noted that the ocean crossing 322, the new approach 324, and the mountain crossing 332 are only some examples (i.e. sets of active BEEs) of many different types of aviation events that may be associated with flights. Other potential scenarios may include fuel contaminations, ash encounters, technical problems, diversions due to sick passengers, etc. (i.e. sets of active BEEs) In practice, any type of aviation event may be associated with a flight and/or scheduled for training. [0049] FIG. 4, an embodiment of the regulatory or audit data 142 is depicted. The regulatory or audit data 142 may map employees 410-412 to various required or preferred training exercises 420-422, 430, 440, 441. For example, a first employee 410 may be mapped to a first set of required or preferred training exercises 410-422, a second employee may be mapped to a second required or preferred training exercise 430, and a third employee 412 may be mapped to a third set of required or preferred training exercises 440, 441. [0050] Regulatory authorities (such as the Federal Aviation Administration) may require that the employees 410-412 receive certain training and may perform audits to ensure that it has been done. Likewise, individual airlines or air service providers may require certain training. As the employees 410-412 complete the training, it may be removed from their list of required or preferred training exercises. The learning management system 110 of FIG. 1 may select the training exercise 160 to correspond to the regulatory or audit data 142 for the employees 410-412. [0051] FIG. 5, an embodiment of the sample data 152 is depicted. The sample data 152 may map employees 510-512 to training histories 520, 530, 540. For example, a first employee 510 may be mapped to a first training history 520, a second employee 511 may be mapped to a second training history 530, and a third employee 512 may be mapped to a third training history 540. The learning management system 110 of FIG. 1 may use the sample data 152 to determining a common training exercise 172 associated with the employees 510-512. The training exercise 160 may correspond to and/or may be the common training exercise 172. Determining the common training exercise 172 associated with employees 510-512 may include performing an artificial intelligence analysis of the training histories 520, 530, 540. [0052] The learning management system 610 may receive a training browsing history 612, completed training records 614, and data from a crew roster 616. The training browsing history 612 may be associated with multiple trainees and can help the learning management system 610 identify common training topics and scenarios (i.e. first set of active BEEs) where training might be beneficial. The completed training records 614 may function as a checklist to indicate what training trainees have completed (i.e. first set of active BEEs) and what training still needs to be completed. (i.e. stored as incomplete and second set of active BEEs) [0056] During the embedded training 650, data logs 656 may be generated. Further, after the training, feedback 652 via in-situ analytics may be performed to evaluate the trainee's performance and the relevance of the mid-fidelity simulation training scenarios 630. Also, the trainee may engage in a debriefing 654 with an instructor, with an automated feedback application, or with another type of feedback collection method. Data and results associated with the data logs 656, the feedback 652, and the debriefing 654 may be provided to the learning management system 610 for future use in determine what training to provide and when to provide it. For example, training data from the use of MFS, which may be included in the data logs 656, can be used to determines a next exercise (i.e. second simulation state update) to be performed. If a trainee does well, they may not need further training in a particular area. Otherwise, the data logs 656 can be used by the learning management system 610 to determine whether to offer another training , or additional training, within a certain time frame. )
Although implied, Kennedy does not expressly disclose the following limitations, which however, are taught by Peyronnet,
… defining expected user inputs as a response to the BEE … (in at least [0067] FIG. 2 , the data correlation step 104 comprises a step 116 consisting in determining a trigger event originating from an operator, the trigger event originating from an operator is the occurrence of an event at the origin of an action of the at least one operator in response to this trigger event originating from an operator. The trigger parameter originating from an operator may thus be an endogenous datum, such as for example the initialization of technical dialog between the team of operators. However, it may also be contemplated, in a more generalized scenario, for the trigger event parameter to be an exogenous datum. [0068] The data correlation step 104 then comprises a step 118 consisting in determining a trigger event originating from the platform. The trigger event originating from the platform represents the occurrence of an event at the origin of a state of the platform, and may be interpreted as an exogenous datum. This is then followed by a step 120 of detecting trigger events originating from said at least one operator or from said platform, and a step 122 of selecting at least one trigger event. [0069] The detection 120 of a trigger event originating from an operator is based on the detection of an action from the at least one operator. The detection 120 of a trigger event originating from a platform is based on the detection of a state of the platform, such as for example a change of piloting mode, the extension or retraction of the landing gear, a fault, and on the departure from an envelope of dynamic parameters, such as for example speed, incline, attitude. [0070] In response to the trigger event, whether it is a trigger from the at least one operator or from the platform, the assessment method 100 then captures at least one action parameter represented by an endogenous datum and presenting the physical manifestation of a reaction of the at least one operator to the trigger parameter. Grouping together a trigger parameter, represented by an exogenous datum or an endogenous datum, and at least one action parameter, represented by endogenous data, thus makes it possible, during the correlation step 104, to generate at least one observable behavior datum reflecting, according to tangible parameters, the behavior of the at least one operator upon triggering of an event. [0079] In step 106, the assessment method 100 comprises a step consisting in analyzing the observable behavior data in predefined analysis sequences, each predefined analysis sequence being specific to a technical and non-technical skill to be assessed, and comprising at least one trigger event parameter and one action parameter for characterizing an expected observable behavior according to a predefined situation. The analysis 106 also makes it possible to generate a measurement indicator for each observed behavior. More specifically, step 106 analyzes observable behavior data under the prism of trigger event and action parameters by comparing (step 132) the detected observable behavior data with a predefined sequence defining the expected observable behavior, each predefined sequence representing at least one physical manifestation allocated to the expected behavior. The predefined sequences are contained in a correspondence database. This correspondence database thus comprises the predefined analysis sequences presenting observable behavior data known to those skilled in the art, as well as their assigned measurable and detectable physical manifestations. Each predefined analysis sequence thus comprises at least one trigger event parameter and at least one action parameter and other endogenous and exogenous data for characterizing a flight situation and a context for at least one operator, and also their expected reaction according to the predefined situation. This analysis provides the nature of the induced action, its temporal location and also its duration or its frequency. The correspondence database also comprises a reference table containing trigger event parameters associated with each behavior to be observed. [0080] in order to be able to analyze the behavior of at least one operator during a flight situation, the step 106 of analyzing the observable behavior data compares the detected endogenous and exogenous data, and more specifically the trigger event and action parameters, with the trigger event and action parameters and also the predefined endogenous and exogenous data. The predefined analysis sequences are specific to each technical and non-technical skill to be assessed. [0092] This generation of observed behavior indicators then makes it possible to initiate a step 108, shown in FIG. 1 , consisting in assessing a behavior of at least one operator. The assessment of the behavior of the at least one operator consists in comparing the observed behavior, which is based on a set of detected behavior elements, with predefined expected reference behaviors. The conformity of an observed behavior is assessed by comparison with known prior art of defined procedures or established protocols, contained in the correspondence database. [0093] The objectivity of the assessment of the technical and non-technical skills of an operator is thus based on the prior creation of the correspondence database between various observable behaviors and measurable physical variables in relation to these observable behaviors. [0094] The matching consists, for each observable behavior, in determining various ways of measuring same and then developing the tools necessary for each measurement. [0095] By way of indicative and non-exhaustive examples, various cases below are given to illustrate the matching of observable behaviors with measurable physical variables to allow behaviors to be assessed.)
…action … matches an elicited action for at least one active BEE in the set of active BEEs… (in at least [0067] FIG. 2 , the data correlation step 104 comprises a step 116 consisting in determining a trigger event originating from an operator, the trigger event originating from an operator is the occurrence of an event at the origin of an action of the at least one operator in response to this trigger event originating from an operator. The trigger parameter originating from an operator may thus be an endogenous datum, such as for example the initialization of technical dialog between the team of operators. However, it may also be contemplated, in a more generalized scenario, for the trigger event parameter to be an exogenous datum. [0068] The data correlation step 104 then comprises a step 118 consisting in determining a trigger event originating from the platform. The trigger event originating from the platform represents the occurrence of an event at the origin of a state of the platform, and may be interpreted as an exogenous datum. This is then followed by a step 120 of detecting trigger events originating from said at least one operator or from said platform, and a step 122 of selecting at least one trigger event. [0069] The detection 120 of a trigger event originating from an operator is based on the detection of an action from the at least one operator. The detection 120 of a trigger event originating from a platform is based on the detection of a state of the platform, such as for example a change of piloting mode, the extension or retraction of the landing gear, a fault, and on the departure from an envelope of dynamic parameters, such as for example speed, incline, attitude. [0070] In response to the trigger event, whether it is a trigger from the at least one operator or from the platform, the assessment method 100 then captures at least one action parameter represented by an endogenous datum and presenting the physical manifestation of a reaction of the at least one operator to the trigger parameter. Grouping together a trigger parameter, represented by an exogenous datum or an endogenous datum, and at least one action parameter, represented by endogenous data, thus makes it possible, during the correlation step 104, to generate at least one observable behavior datum reflecting, according to tangible parameters, the behavior of the at least one operator upon triggering of an event. [0079] In step 106, the assessment method 100 comprises a step consisting in analyzing the observable behavior data in predefined analysis sequences, each predefined analysis sequence being specific to a technical and non-technical skill to be assessed, and comprising at least one trigger event parameter and one action parameter for characterizing an expected observable behavior according to a predefined situation. The analysis 106 also makes it possible to generate a measurement indicator for each observed behavior. More specifically, step 106 analyzes observable behavior data under the prism of trigger event and action parameters by comparing (step 132) the detected observable behavior data with a predefined sequence defining the expected observable behavior, each predefined sequence representing at least one physical manifestation allocated to the expected behavior. The predefined sequences are contained in a correspondence database. This correspondence database thus comprises the predefined analysis sequences presenting observable behavior data known to those skilled in the art, as well as their assigned measurable and detectable physical manifestations. Each predefined analysis sequence thus comprises at least one trigger event parameter and at least one action parameter and other endogenous and exogenous data for characterizing a flight situation and a context for at least one operator, and also their expected reaction according to the predefined situation. This analysis provides the nature of the induced action, its temporal location and also its duration or its frequency. The correspondence database also comprises a reference table containing trigger event parameters associated with each behavior to be observed. [0080] in order to be able to analyze the behavior of at least one operator during a flight situation, the step 106 of analyzing the observable behavior data compares the detected endogenous and exogenous data, and more specifically the trigger event and action parameters, with the trigger event and action parameters and also the predefined endogenous and exogenous data. The predefined analysis sequences are specific to each technical and non-technical skill to be assessed. [0092] This generation of observed behavior indicators then makes it possible to initiate a step 108, shown in FIG. 1 , consisting in assessing a behavior of at least one operator. The assessment of the behavior of the at least one operator consists in comparing the observed behavior, which is based on a set of detected behavior elements, with predefined expected reference behaviors. The conformity of an observed behavior is assessed by comparison with known prior art of defined procedures or established protocols, contained in the correspondence database. [0093] The objectivity of the assessment of the technical and non-technical skills of an operator is thus based on the prior creation of the correspondence database between various observable behaviors and measurable physical variables in relation to these observable behaviors. [0094] The matching consists, for each observable behavior, in determining various ways of measuring same and then developing the tools necessary for each measurement. [0095] By way of indicative and non-exhaustive examples, various cases below are given to illustrate the matching of observable behaviors with measurable physical variables to allow behaviors to be assessed.)
The reason and rationale to combine Kennedy and Peyronnet is the same as recited above.
As per Claim 21, Kennedy teaches: (New) The system of claim 9,
wherein the training environment comprises a flight simulator. (in at least [0003] recurrent flight crew training and evaluation may include computer-based training (CBT) (usually in the form of digital slide presentations), aircraft part-task simulators, aircraft fixed-base simulators, and highly sophisticated and realistic full-motion flight simulators. [0004] CBT is effectively a medium for presenting manuals, checklists, and other paper-based information in a digital slide pack. It typically includes no simulation that would allow someone to interact with the aircraft systems. Training may be enhanced with part-task trainers that enable subjects to practice skills in the operation of specific flight deck systems, however, these part-task trainers fail to provide a full picture of how the aircraft functions in given scenarios and cannot realistically simulate interactions between highly coupled aircraft systems. Fixed-base and full-motion simulators are a good medium for practicing physical maneuvers. However, they cannot evaluate whether the flight crew possess the requisite competencies to manage unusual/non-normal situations. They also cannot predict whether the flight crew understands the underlying mechanics of how the aircraft functions. Such understanding is crucial for correct decision making to occur.)
As per Claim 22, Kennedy teaches: (New) The system of claim 21,
wherein the flight simulator comprises a fixed training device (FTD) flight simulator or a full motion simulator (FMS) flight simulator. (in at least [0003] recurrent flight crew training and evaluation may include computer-based training (CBT) (usually in the form of digital slide presentations), aircraft part-task simulators, aircraft fixed-base simulators, and highly sophisticated and realistic full-motion flight simulators. [0004] CBT is effectively a medium for presenting manuals, checklists, and other paper-based information in a digital slide pack. It typically includes no simulation that would allow someone to interact with the aircraft systems. Training may be enhanced with part-task trainers that enable subjects to practice skills in the operation of specific flight deck systems, however, these part-task trainers fail to provide a full picture of how the aircraft functions in given scenarios and cannot realistically simulate interactions between highly coupled aircraft systems. Fixed-base and full-motion simulators are a good medium for practicing physical maneuvers. However, they cannot evaluate whether the flight crew possess the requisite competencies to manage unusual/non-normal situations. They also cannot predict whether the flight crew understands the underlying mechanics of how the aircraft functions. Such understanding is crucial for correct decision making to occur.)
As per Claim 10-14 for A system (see at least Kennedy [0052]), substantially recite the subject matter of Claim 2-5, 7 and are rejected based on the same reasoning and rationale.
As per Claim 17-20 for method (see at least Kennedy [0052]), substantially recite the subject matter of Claim 1-3, 5 and are rejected based on the same reasoning and rationale.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PO HAN MAX LEE whose telephone number is (571) 272-3821. The examiner can normally be reached on Mon-Thurs 8:00 am - 7:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rutao Wu can be reached on (571) 272-6045. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PO HAN LEE/Primary Examiner, Art Unit 3623
Prosecution Insights