Prosecution Insights
Last updated: November 20, 2026
Application No. 18/364,380

UTILIZING BEHAVIORAL ELICITATION EVENTS FOR EVALUATING A TRAINING PERFORMANCE AGAINST A COMPETENCY FRAMEWORK

Final Rejection §101
Filed
Aug 02, 2023
Examiner
LEE, PO HAN
Art Unit
3623
Tech Center
3600 — Transportation & Electronic Commerce
Assignee
The Boeing Company
OA Round
4 (Final)
32%
Grant Probability
At Risk
5-6
OA Rounds
3m
Est. Remaining
73%
With Interview

Examiner Intelligence

Grants only 32% of cases
32%
Career Allowance Rate
54 granted / 169 resolved
-20.0% vs TC avg
Strong +41% interview lift
Without
With
+41.3%
Interview Lift
resolved cases with interview
Typical timeline
3y 7m
Avg Prosecution
37 currently pending
Career history
223
Total Applications
across all art units

Statute-Specific Performance

§101
45.1%
+5.1% vs TC avg
§103
36.0%
-4.0% vs TC avg
§102
11.5%
-28.5% vs TC avg
§112
5.8%
-34.2% vs TC avg
Black line = Tech Center average estimate • Based on career data from 169 resolved cases

Office Action

§101
Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . DETAILED ACTION Status of the Application The following is a Final Office Action. In response to Examiner's communication of 6/23/2026, Applicant responded on 9/23/2026. Amended claims 1, 4, 5, 9, 12, 13, 16, 19, 20, and 22. Cancelled claim 21. Added claim 23. Claims 1-7, 9-14, and 16-20, 22-23 are pending in this application and have been examined. Response to Amendment Applicant's amendments to claims 1, 4, 5, 9, 12, 13, 16, 19, 20, and 22 are not sufficient to overcome the 35 USC 101 rejections set forth in the previous action. Applicant's amendments to claims 1, 4, 5, 9, 12, 13, 16, 19, 20, and 22 are sufficient to overcome the prior art rejections set forth in the previous action. Response to Arguments – 35 USC § 101 Applicant’s arguments with respect to the rejections have been fully considered, but they are not persuasive. Applicant submits, “….the claims as presented herein improve the technology for a particular machine of flight simulation training devices (FSTD) by automatically evaluating a training performance performed in the training device….As summarized in MPEP 2106.04(d), limitations the courts have found indicative that an additional element (or combination of elements) may have integrated a judicial exception into a practical application include an improvement in the functioning of a computer, or an improvement to other technology or technical field. Courts also have found claims that implement a judicial exception with, or that use a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim may integrate an abstract idea into a practical application. As disclosed by example in Applicant's specification at paragraph [0018], the aircraft training environment 102, for use in training pilots, can be configured as a FTD (fixed training device) flight simulator or an FMS (full motion simulator) flight simulator that "simulate a flight deck within aircraft training environment 102 ... ." Additionally, as stated in Appendix F to the 14 CFR Part 60, which is utilized by the FAA (Federal Aviation Association), a flight training device (FTD) is a flight simulation training device (FSTD) having equipment and computer programs to represent aircraft operations, while a full flight simulator (FFS) is a FSTD that replicates a specific type, make, model, or series of aircraft along with providing cues for motion.….As such, one skilled in the art would understand that the training environment 102 configured as the FTD or the FMS is a flight simulation training device (FSTD) for training pilots. The specification further discloses by example at paragraphs [0026-0027] that the xAPI monitor 406 is configured to monitor the training environment for update statements and when an update statement comprises a simulation state update or a simulation action notification, sends the update statement to a BEE monitor for evaluation. The specification continues, in response, the BEE monitor determines and sends a BEE evaluation, in paragraph [0036], comprising information regarding elicited action(s) as a response to an active BEE, observable behavior evaluations, in paragraph [0039], for observable behaviors of a completed BEE, and observable behavior evaluation(s), in paragraph [0032], of an incomplete BEE. Therefore, the configuration of claim 1 utilizes the xAPI monitor and the BEE monitor "to automate the evaluation of a training performance of a training subject" where the training performance is being performed in a flight simulator (specification, paragraph [0015])…. For at least the above reasons, the claims as presented herein use the particular machine of a flight simulator device along with improving the technology of flight simulator devices by automatically evaluating training performances in the flight simulator. As such, amended independent claims 1, 9, 16, and claims depending therefrom define eligible subject matter. Applicant notes that amended claims 1, 9, and 16 contain meaningful limitations that do not monopolize the asserted abstract ideas of a mental process and certain methods of organizing human activities…..” The Examiner respectfully disagrees. While Applicant’s amendments further prosecution, however the claims, by Applicant’s own admission, recite and direct to, …evaluating a training subject (i.e. human) by observing the training subject (i.e. human) during a training performance pose several problems.…provide data on the training performance to help an instructor (i.e. human) objectively assess soft skill competencies and increase consistency of inter-rater reliability over evaluations by observation of the instructor (i.e. human)… provide detailed evidence for use when asserting the proficiency of a training subject (i.e. human) performing the training performance… evaluation of the training performance in a more objective, reliable, and traceable manner than by observations of instructors (i.e. human)…, which is a problem directed to a mental process (i.e. human instructor observing and training human pilot actions and behaviors with training scenarios and learning simulations) and organizing human activities (i.e. h human instructor observing and training human pilot actions and behaviors with training scenarios and learning simulations), as established in Step 2A Prong 1. This problem does not specifically arise in the realm of computer technology, but rather, this problem existed and was addressed long before the advent of computers. Thus, the claims do not recite a technical improvement to a technical problem or necessarily roots in computing technologies. The alleged solutions are solutions directed to solving abstract ideas, which are still abstract ideas. Additionally, pursuant to the broadest reasonable interpretation, as an ordered combination, each of the additional elements are computing elements recited at high level of generality implementing the abstract idea, and thus, are no more than applying the abstract idea with generic computer components. Further, these additional elements generally link the abstract idea to a technical environment, namely the environment of a computer flight simulator, performing extra solution activities. Therefore, as a whole, the additional elements do not integrate the abstract ideas into a practical application in Step 2A Prong 2 or amount to significantly more under Step 2B. Further, the generically claimed elements of, “…flight simulator…”, in claims 1, 9, 16, under the broadest reasonable interpretation can be interpreted to be simulations with human hands or simulations to be performed with pen and paper or model aircrafts, thus do not root in computing technologies and do not integrate the recited abstract ideas into a practical application or amount to significantly more. And, contrary to Applicant’s assertions, the generically recited additional elements applying the abstract idea such as, flight training device (FTD), full flight simulator, in claim 22, have long existed before the advent of computers, used for organizing human behavior, such as “…for use in training pilots…”, do not root in computing technologies and do not integrate the recited abstract ideas into a practical application or amount to significantly more. PNG media_image1.png 327 569 media_image1.png Greyscale PNG media_image2.png 784 878 media_image2.png Greyscale PNG media_image3.png 285 600 media_image3.png Greyscale PNG media_image4.png 560 598 media_image4.png Greyscale PNG media_image5.png 682 826 media_image5.png Greyscale PNG media_image6.png 655 841 media_image6.png Greyscale PNG media_image7.png 682 858 media_image7.png Greyscale PNG media_image8.png 713 861 media_image8.png Greyscale PNG media_image9.png 696 468 media_image9.png Greyscale PNG media_image10.png 818 548 media_image10.png Greyscale Even novel and newly discovered judicial exceptions are still exceptions, despite their novelty. July 2015 Update, p. 3; see SAP America Inc. v. Investpic, LLC, No. 2017-2081, slip op. at 2 (Fed Cir. May 15, 2018). Simply reciting specific limitations that narrow the abstract idea does not make an abstract idea non-abstract. 79 Fed. Reg. 74631; buySAFE Inc. v. Google, Inc., 765 F.3d 1350, 1355 (2014); see SAP America at p. 12. As discussed in SAP America, no matter how much of an advance the claims recite, when “the advance lies entirely in the realm of abstract ideas, with no plausibly alleged innovation in the non-abstract application realm,” “[a]n advance of that nature is ineligible for patenting.” Id. at p. 3. Claims can recite a mental process even if they are claimed as being performed on a computer. The Supreme Court recognized this in Benson, determining that a mathematical algorithm for converting binary coded decimal to pure binary within a computer’s shift register was an abstract idea. The Court concluded that the algorithm could be performed purely mentally even though the claimed procedures “can be carried out in existing computers long in use, no new machinery being necessary.” 409 U.S at 67, 175 USPQ at 675. See also Mortgage Grader, 811 F.3d at 1324, 117 USPQ2d at 1699 (concluding that concept of “anonymous loan shopping” recited in a computer system claim is an abstract idea because it could be “performed by humans without a computer”). An example of a case in which a computer was used as a tool to perform a mental process is Mortgage Grader, 811 F.3d. at 1324, 117 USPQ2d at 1699. The patentee in Mortgage Grader claimed a computer-implemented system for enabling borrowers to anonymously shop for loan packages offered by a plurality of lenders, comprising a database that stores loan package data from the lenders, and a computer system providing an interface and a grading module. The interface prompts a borrower to enter personal information, which the grading module uses to calculate the borrower’s credit grading, and allows the borrower to identify and compare loan packages in the database using the credit grading. 811 F.3d. at 1318, 117 USPQ2d at 1695. The Federal Circuit determined that these claims were directed to the concept of “anonymous loan shopping”, which was a concept that could be “performed by humans without a computer.” 811 F.3d. at 1324, 117 USPQ2d at 1699. Another example is Berkheimer v. HP, Inc., 881 F.3d 1360, 125 USPQ2d 1649 (Fed. Cir. 2018), in which the patentee claimed methods for parsing and evaluating data using a computer processing system. The Federal Circuit determined that these claims were directed to mental processes of parsing and comparing data, because the steps were recited at a high level of generality and merely used computers as a tool to perform the processes. 881 F.3d at 1366, 125 USPQ2d at 1652-53. TLI Communications provides an example of a claim invoking computers and other machinery merely as a tool to perform an existing process. The court stated that the claims describe steps of recording, administration and archiving of digital images, and found them to be directed to the abstract idea of classifying and storing digital images in an organized manner. 823 F.3d at 612, 118 USPQ2d at 1747. The court then turned to the additional elements of performing these functions using a telephone unit and a server and noted that these elements were being used in their ordinary capacity (i.e., the telephone unit is used to make calls and operate as a digital camera including compressing images and transmitting those images, and the server simply receives data, extracts classification information from the received data, and stores the digital images based on the extracted information). 823 F.3d at 612-13, 118 USPQ2d at 1747-48. In other words, the claims invoked the telephone unit and server merely as tools to execute the abstract idea. Thus, the court found that the additional elements did not add significantly more to the abstract idea because they were simply applying the abstract idea on a telephone network without any recitation of details of how to carry out the abstract idea. Examples that the courts have indicated may not be sufficient to show an improvement in computer-functionality: ii. Accelerating a process of analyzing audit log data when the increased speed comes solely from the capabilities of a general-purpose computer, FairWarning IP, LLC v. Iatric Sys., 839 F.3d 1089, 1095, 120 USPQ2d 1293, 1296 (Fed. Cir. 2016); iii. Mere automation of manual processes, such as using a generic computer to process an application for financing a purchase, Credit Acceptance Corp. v. Westlake Services, 859 F.3d 1044, 1055, 123 USPQ2d 1100, 1108-09 (Fed. Cir. 2017) Other examples where the courts have found the additional elements to be mere instructions to apply an exception, because they do no more than merely invoke computers or machinery as a tool to perform an existing process include: i. A commonplace business method or mathematical algorithm being applied on a general purpose computer, Alice Corp. Pty. Ltd. V. CLS Bank Int’l, 573 U.S. 208, 223, 110 USPQ2d 1976, 1983 (2014); Gottschalk v. Benson, 409 U.S. 63, 64, 175 USPQ 673, 674 (1972); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); iii. A process for monitoring audit log data that is executed on a general-purpose computer where the increased speed in the process comes solely from the capabilities of the general-purpose computer, FairWarning IP, LLC v. Iatric Sys., 839 F.3d 1089, 1095, 120 USPQ2d 1293, 1296 (Fed. Cir. 2016); v. Requiring the use of software to tailor information and provide it to the user on a generic computer, Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1370-71, 115 USPQ2d 1636, 1642 (Fed. Cir. 2015) Response to Arguments – Prior Art Applicant’s arguments with respect to the rejections have been fully considered. Examiner finds persuasive Applicant’s remarks on pg14-15. The closest prior art are US Patent Publication to US20210272469A1 to Kennedy et al., (hereinafter referred to as “Kennedy”) in view of US Patent Publication to US20240105076A1 to Peyronnet et al., (hereinafter referred to as “Peyronnet”). However, the teachings of the references do not teach the specific ordered sequence of limitations of independent claims 1, 9, 16, Claim 1: A method of evaluating a training performance of a training subject in response to a computer simulated training scenario, the training performance being performed in a flight simulator that simulates a physical flight deck and comprising a training interface configured to receive user inputs from the training subject, the method comprising: receiving a plurality of update statements from an experience application programming interface (xAPI) monitor of the flight simulator; for each update statement comprising a simulation state update indicating a change in a simulation state of the computer simulated training scenario, querying, using a behavioral elicitation event (BEE) monitor, a BEE database to determine, from a plurality of BEEs stored in the BEE database, a set of active BEEs based upon the simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions and one or more corresponding observable behaviors, wherein the one or more corresponding observable behaviors correspond to pilot competencies in a competency framework; for each update statement comprising a simulation action notification indicating a corresponding user input entered into the training interface, determining, using the BEE monitor, whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the set of active BEEs and storing, in a learning record, a BEE evaluation including information regarding the elicited action as a response to the at least one active BEE in the set of active BEEs; for each active BEE, determining, using the BEE monitor, whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determining that the active BEE is a completed BEE; querying, using the BEE monitor, the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and storing, in the learning record, an observable behavior evaluation including information regarding the one or more corresponding observable behaviors of the completed BEE; and automatically evaluating the training performance against the competency framework using the BEE evaluations and the observable behavior evaluations stored in the learning record. Claim 9: A system comprising: a flight simulator flight deck, the flight simulator comprising a training computing system configured to perform a simulated training scenario including a simulation state, a training interface configured to receive user inputs from a training subject in response to the simulated training scenario, and an experience application programming interface (xAPI) monitor configured to send an update statement in response to a change in the simulation state of the simulated training scenario or receiving the user inputs via the training interface; and a computing system comprising a logic system operably coupled to the training computing system, and a computer-readable storage system comprising, a behavioral elicitation event (BEE) database comprising a plurality of BEEs, each BEE of the plurality of BEEs comprising one or more elicited actions defining expected user inputs as a response to the BEE, and one or more corresponding observable behaviors, wherein the one or more corresponding observable behaviors correspond to pilot competencies in a competency framework, a learning record to store information about the simulation state of the simulated training scenario and information related to a training performance of the training subject in response to the simulated training scenario, and instructions executable by the logic system to receive a plurality of update statements from the xAPI monitor of the flight simulator, for each update statement comprising a simulation state update indicating the change in the simulation state of the simulated training scenario, query, using a BEE monitor, the BEE database to determine, from the plurality of BEEs, a set of active BEEs based upon the simulation state update, for each update statement comprising a simulation action notification indicating a corresponding user input entered into the training interface, determine, using the BEE monitor, whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the set of active BEEs, and store, in the learning record, a BEE evaluation including information regarding the elicited action as the response to the at least one active BEE in the set of active BEEs, for each active BEE, determine, using the BEE monitor, whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE, query, using the BEE monitor, the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store, in the learning record, an observable behavior evaluation including information regarding the corresponding observable behaviors of the completed BEE, and automatically evaluate the training performance against the competency framework using the BEE evaluations and the observable behavior evaluations stored in the learning record. Claim 16: A method of evaluating a training performance of a training subject in response to a computer simulated training scenario, the training performance being performed in a flight simulator that simulates a physical flight deck and comprising a training interface configured to receive user inputs from the training subject, the method comprising: receiving a first simulation state update from an experience application programming interface (xAPI) monitor of the flight simulator, the first simulation state update indicating a first change in a simulation state of the computer simulated training scenario; querying, using a behavioral elicitation event (BEE) monitor, a BEE database to determine, from a plurality of BEEs stored in the BEE database, a first set of active BEEs based upon the first simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions defining expected user inputs as a response to the BEE, and one or more corresponding observable behaviors, wherein the one or more corresponding observable behaviors correspond to pilot competencies in a competency framework; receiving, from the xAPI monitor, a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface; for each simulation action notification, determining, using the BEE monitor, whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the first set of active BEEs and storing, in a learning record, a BEE evaluation including information regarding the elicited action and corresponding observable behaviors as the response to the at least one active BEE in the first set of active BEEs; receiving a second simulation state update from the xAPI monitor, the second simulation state update indicating a second change in the simulation state of the computer simulated training scenario; querying, using the BEE monitor, the BEE database to determine a second set of active BEEs based upon the second simulation state update, the first set of active BEEs including at least one active BEE that is not in the second set of active BEEs; comparing the first set of active BEEs to the second set of active BEEs; for each active BEE in the first set of active BEEs and not in the second set of active BEEs, determining, using the BEE monitor, that the active BEE is an incomplete BEE, and storing, in the learning record, an observable behavior evaluation including information regarding the incomplete BEE and the second simulation state update; and automatically evaluating the training performance against the competency framework using the BEE evaluations and the observable behavior evaluations stored in the learning record. No Non-Patent literature teach the specific ordered sequence of limitations of independent claims 1, 9, 16. The prior art rejection is hereby withdrawn. Claim Rejections – 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-7, 9-14, and 16-20, 22-23 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. Claim 1 recite, “A method of evaluating a training performance of a training subject in response to a …simulated training scenario, the training performance being performed in a flight simulator that simulates a physical flight deck and comprising a training interface configured to receive user inputs from the training subject, the method comprising: receiving a plurality of update statements from an … monitor of the flight simulator; for each update statement comprising a simulation state update indicating a change in a simulation state of the … simulated training scenario, querying, using a behavioral elicitation event (BEE) monitor, a BEE database to determine, from a plurality of BEEs stored in the BEE database, a set of active BEEs based upon the simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions and one or more corresponding observable behaviors, wherein the one or more corresponding observable behaviors correspond to pilot competencies in a competency framework; for each update statement comprising a simulation action notification indicating a corresponding user input entered into the training interface, determining, using the BEE monitor, whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the set of active BEEs and storing, in a learning record, a BEE evaluation including information regarding the elicited action as a response to the at least one active BEE in the set of active BEEs; for each active BEE, determining, using the BEE monitor, whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determining that the active BEE is a completed BEE; querying, using the BEE monitor, the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and storing, in the learning record, an observable behavior evaluation including information regarding the one or more corresponding observable behaviors of the completed BEE; and automatically evaluating the training performance against the competency framework using the BEE evaluations and the observable behavior evaluations stored in the learning record.” Claim 9 recites, “A system comprising: a flight simulator configured to simulate a physical flight deck, the flight simulator comprising a … configured to perform a simulated training scenario including a simulation state, a training interface configured to receive user inputs from a training subject in response to the simulated training scenario, and an … monitor configured to send an update statement in response to a change in the simulation state of the simulated training scenario or receiving the user inputs via the training interface; and a …, and a … comprising, a behavioral elicitation event (BEE) database comprising a plurality of BEEs, each BEE of the plurality of BEEs comprising one or more elicited actions defining expected user inputs as a response to the BEE, and one or more corresponding observable behaviors, wherein the one or more corresponding observable behaviors correspond to pilot competencies in a competency framework, a learning record to store information about the simulation state of the simulated training scenario and information related to a training performance of the training subject in response to the simulated training scenario, and instructions executable by the logic system to receive a plurality of update statements from the … monitor of the flight simulator, for each update statement comprising a simulation state update indicating the change in the simulation state of the simulated training scenario, query, using a BEE monitor, the BEE database to determine, from the plurality of BEEs, a set of active BEEs based upon the simulation state update, for each update statement comprising a simulation action notification indicating a corresponding user input entered into the training interface, determine, using the BEE monitor, whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the set of active BEEs, and store, in the learning record, a BEE evaluation including information regarding the elicited action as the response to the at least one active BEE in the set of active BEEs, for each active BEE, determine, using the BEE monitor, whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE, query, using the BEE monitor, the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store, in the learning record, an observable behavior evaluation including information regarding the corresponding observable behaviors of the completed BEE, and automatically evaluate the training performance against the competency framework using the BEE evaluations and the observable behavior evaluations stored in the learning record.” Claim 16 recites, “A method of evaluating a training performance of a training subject in response to a … simulated training scenario, the training performance being performed in a flight simulator that simulates a physical flight deck and comprising a training interface configured to receive user inputs from the training subject, the method comprising: receiving a first simulation state update from an … monitor of the flight simulator, the first simulation state update indicating a first change in a simulation state of the … simulated training scenario; querying, using a behavioral elicitation event (BEE) monitor, a BEE database to determine, from a plurality of BEEs stored in the BEE database, a first set of active BEEs based upon the first simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions defining expected user inputs as a response to the BEE, and one or more corresponding observable behaviors, wherein the one or more corresponding observable behaviors correspond to pilot competencies in a competency framework; receiving, from the … monitor, a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface; for each simulation action notification, determining, using the BEE monitor, whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the first set of active BEEs and storing, in a learning record, a BEE evaluation including information regarding the elicited action and corresponding observable behaviors as the response to the at least one active BEE in the first set of active BEEs; receiving a second simulation state update from the … monitor, the second simulation state update indicating a second change in the simulation state of the … simulated training scenario; querying, using the BEE monitor, the BEE database to determine a second set of active BEEs based upon the second simulation state update, the first set of active BEEs including at least one active BEE that is not in the second set of active BEEs; comparing the first set of active BEEs to the second set of active BEEs; for each active BEE in the first set of active BEEs and not in the second set of active BEEs, determining, using the BEE monitor, that the active BEE is an incomplete BEE, and storing, in the learning record, an observable behavior evaluation including information regarding the incomplete BEE and the second simulation state update; and automatically evaluating the training performance against the competency framework using the BEE evaluations and the observable behavior evaluations stored in the learning record.” Analyzing under Step 2A, Prong 1: The limitations regarding, … a training performance of a training subject in response to a …simulated training scenario, the training performance being performed in a flight simulator that simulates a physical flight deck and comprising a training interface configured to receive user inputs from the training subject…receiving a plurality of update statements from an … monitor of the flight simulator; for each update statement comprising a simulation state update indicating a change in a simulation state of the … simulated training scenario, querying, using a behavioral elicitation event (BEE) monitor, a BEE database to determine, from a plurality of BEEs stored in the BEE database, a set of active BEEs based upon the simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions and one or more corresponding observable behaviors, wherein the one or more corresponding observable behaviors correspond to pilot competencies in a competency framework; for each update statement comprising a simulation action notification indicating a corresponding user input entered into the training interface, determining, using the BEE monitor, whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the set of active BEEs and storing, in a learning record, a BEE evaluation including information regarding the elicited action as a response to the at least one active BEE in the set of active BEEs; for each active BEE, determining, using the BEE monitor, whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determining that the active BEE is a completed BEE; querying, using the BEE monitor, the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and storing, in the learning record, an observable behavior evaluation including information regarding the one or more corresponding observable behaviors of the completed BEE; and automatically evaluating the training performance against the competency framework using the BEE evaluations and the observable behavior evaluations stored in the learning record.… a flight simulator configured to simulate a physical flight deck, the flight simulator…to perform a simulated training scenario including a simulation state, a training interface configured to receive user inputs from a training subject in response to the simulated training scenario, and an … monitor configured to send an update statement in response to a change in the simulation state of the simulated training scenario or receiving the user inputs via the training interface; a behavioral elicitation event (BEE) database comprising a plurality of BEEs, each BEE of the plurality of BEEs comprising one or more elicited actions defining expected user inputs as a response to the BEE, and one or more corresponding observable behaviors, wherein the one or more corresponding observable behaviors correspond to pilot competencies in a competency framework, a learning record to store information about the simulation state of the simulated training scenario and information related to a training performance of the training subject in response to the simulated training scenario…receive a plurality of update statements from the … monitor of the flight simulator, for each update statement comprising a simulation state update indicating the change in the simulation state of the simulated training scenario, query, using a BEE monitor, the BEE database to determine, from the plurality of BEEs, a set of active BEEs based upon the simulation state update, for each update statement comprising a simulation action notification indicating a corresponding user input entered into the training interface, determine, using the BEE monitor, whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the set of active BEEs, and store, in the learning record, a BEE evaluation including information regarding the elicited action as the response to the at least one active BEE in the set of active BEEs, for each active BEE, determine, using the BEE monitor, whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE, query, using the BEE monitor, the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store, in the learning record, an observable behavior evaluation including information regarding the corresponding observable behaviors of the completed BEE, and automatically evaluate the training performance against the competency framework using the BEE evaluations and the observable behavior evaluations stored in the learning record…evaluating a training performance of a training subject in response to a … simulated training scenario, the training performance being performed in a flight simulator that simulates a physical flight deck and comprising a training interface configured to receive user inputs from the training subject…receiving a first simulation state update from an … monitor of the flight simulator, the first simulation state update indicating a first change in a simulation state of the … simulated training scenario; querying, using a behavioral elicitation event (BEE) monitor, a BEE database to determine, from a plurality of BEEs stored in the BEE database, a first set of active BEEs based upon the first simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions defining expected user inputs as a response to the BEE, and one or more corresponding observable behaviors, wherein the one or more corresponding observable behaviors correspond to pilot competencies in a competency framework; receiving, from the … monitor, a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface; for each simulation action notification, determining, using the BEE monitor, whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the first set of active BEEs and storing, in a learning record, a BEE evaluation including information regarding the elicited action and corresponding observable behaviors as the response to the at least one active BEE in the first set of active BEEs; receiving a second simulation state update from the … monitor, the second simulation state update indicating a second change in the simulation state of the … simulated training scenario; querying, using the BEE monitor, the BEE database to determine a second set of active BEEs based upon the second simulation state update, the first set of active BEEs including at least one active BEE that is not in the second set of active BEEs; comparing the first set of active BEEs to the second set of active BEEs; for each active BEE in the first set of active BEEs and not in the second set of active BEEs, determining, using the BEE monitor, that the active BEE is an incomplete BEE, and storing, in the learning record, an observable behavior evaluation including information regarding the incomplete BEE and the second simulation state update; and automatically evaluating the training performance against the competency framework using the BEE evaluations and the observable behavior evaluations stored in the learning record…., under the broadest reasonable interpretation, can include a human using their mind and using pen and paper to perform the identified limitations; therefore, the claims recite a mental process. Further, …a training performance of a training subject in response to a …simulated training scenario, the training performance being performed in a flight simulator that simulates a physical flight deck and comprising a training interface configured to receive user inputs from the training subject…receiving a plurality of update statements from an … monitor of the flight simulator; for each update statement comprising a simulation state update indicating a change in a simulation state of the … simulated training scenario, querying, using a behavioral elicitation event (BEE) monitor, a BEE database to determine, from a plurality of BEEs stored in the BEE database, a set of active BEEs based upon the simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions and one or more corresponding observable behaviors, wherein the one or more corresponding observable behaviors correspond to pilot competencies in a competency framework; for each update statement comprising a simulation action notification indicating a corresponding user input entered into the training interface, determining, using the BEE monitor, whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the set of active BEEs and storing, in a learning record, a BEE evaluation including information regarding the elicited action as a response to the at least one active BEE in the set of active BEEs; for each active BEE, determining, using the BEE monitor, whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determining that the active BEE is a completed BEE; querying, using the BEE monitor, the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and storing, in the learning record, an observable behavior evaluation including information regarding the one or more corresponding observable behaviors of the completed BEE; and automatically evaluating the training performance against the competency framework using the BEE evaluations and the observable behavior evaluations stored in the learning record.… a flight simulator configured to simulate a physical flight deck, the flight simulator…to perform a simulated training scenario including a simulation state, a training interface configured to receive user inputs from a training subject in response to the simulated training scenario, and an … monitor configured to send an update statement in response to a change in the simulation state of the simulated training scenario or receiving the user inputs via the training interface; a behavioral elicitation event (BEE) database comprising a plurality of BEEs, each BEE of the plurality of BEEs comprising one or more elicited actions defining expected user inputs as a response to the BEE, and one or more corresponding observable behaviors, wherein the one or more corresponding observable behaviors correspond to pilot competencies in a competency framework, a learning record to store information about the simulation state of the simulated training scenario and information related to a training performance of the training subject in response to the simulated training scenario…receive a plurality of update statements from the … monitor of the flight simulator, for each update statement comprising a simulation state update indicating the change in the simulation state of the simulated training scenario, query, using a BEE monitor, the BEE database to determine, from the plurality of BEEs, a set of active BEEs based upon the simulation state update, for each update statement comprising a simulation action notification indicating a corresponding user input entered into the training interface, determine, using the BEE monitor, whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the set of active BEEs, and store, in the learning record, a BEE evaluation including information regarding the elicited action as the response to the at least one active BEE in the set of active BEEs, for each active BEE, determine, using the BEE monitor, whether the one or more elicited actions of the active BEE has been met, and when all corresponding elicited actions have been met, determine that the active BEE is a completed BEE, query, using the BEE monitor, the BEE database for the corresponding observable behaviors for each elicited action of the one or more elicited actions of the completed BEE, and store, in the learning record, an observable behavior evaluation including information regarding the corresponding observable behaviors of the completed BEE, and automatically evaluate the training performance against the competency framework using the BEE evaluations and the observable behavior evaluations stored in the learning record…evaluating a training performance of a training subject in response to a … simulated training scenario, the training performance being performed in a flight simulator that simulates a physical flight deck and comprising a training interface configured to receive user inputs from the training subject…receiving a first simulation state update from an … monitor of the flight simulator, the first simulation state update indicating a first change in a simulation state of the … simulated training scenario; querying, using a behavioral elicitation event (BEE) monitor, a BEE database to determine, from a plurality of BEEs stored in the BEE database, a first set of active BEEs based upon the first simulation state update, each BEE of the plurality of BEEs comprising one or more elicited actions defining expected user inputs as a response to the BEE, and one or more corresponding observable behaviors, wherein the one or more corresponding observable behaviors correspond to pilot competencies in a competency framework; receiving, from the … monitor, a plurality of simulation action notifications indicating a corresponding plurality of user inputs entered into the training interface; for each simulation action notification, determining, using the BEE monitor, whether an action represented by the simulation action notification matches an elicited action for at least one active BEE in the first set of active BEEs and storing, in a learning record, a BEE evaluation including information regarding the elicited action and corresponding observable behaviors as the response to the at least one active BEE in the first set of active BEEs; receiving a second simulation state update from the … monitor, the second simulation state update indicating a second change in the simulation state of the … simulated training scenario; querying, using the BEE monitor, the BEE database to determine a second set of active BEEs based upon the second simulation state update, the first set of active BEEs including at least one active BEE that is not in the second set of active BEEs; comparing the first set of active BEEs to the second set of active BEEs; for each active BEE in the first set of active BEEs and not in the second set of active BEEs, determining, using the BEE monitor, that the active BEE is an incomplete BEE, and storing, in the learning record, an observable behavior evaluation including information regarding the incomplete BEE and the second simulation state update; and automatically evaluating the training performance against the competency framework using the BEE evaluations and the observable behavior evaluations stored in the learning record…, under the broadest reasonable interpretation, are human instructor observing and training human pilot actions and behaviors with training scenarios and learning simulations, therefore it is, managing personal behavior or relationships or interactions between people. Thus, the claims are directed to certain methods of organizing human activity. Accordingly, the claims are directed to a mental process, certain methods of organizing human activity, and thus, the claims are directed to an abstract idea under the first prong of Step 2A. Analyzing under Step 2A, Prong 2: This judicial exception is not integrated into a practical application under the second prong of Step 2A. In particular, the claims recite the additional elements beyond the recited abstract idea identified under Step 2A, Prong 1, such as: Claim 1, 9, 16: computer simulated, computer, an experience application programming interface (xAPI), training computing system, system, computing system comprising a logic system operably coupled to the training computing system, computer-readable storage system, logic system, computer-readable storage system Claim 22: a fixed training device (FTD) flight simulator or a full motion simulator (FMS) flight simulator , and pursuant to the broadest reasonable interpretation, as an ordered combination, each of the additional elements are computing elements recited at high level of generality implementing the abstract idea, and thus, are no more than applying the abstract idea with generic computer components. Further, these additional elements generally link the abstract idea to a technical environment, namely the environment of a computer. Additionally, with respect to, “receiving …”, “input…”, “querying …”, “storing…”, “sending…”, “notification…”, “…send an update…”, these elements do not add a meaningful limitations to integrate the abstract idea into a practical application because they are extra-solution activity, pre and post solution activity - i.e. data gathering – “receiving …”, “input…”, “querying …”, “storing…”, data output – “sending…”, “notification…”, “…send an update…” Analyzing under Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under Step 2B. As noted above, the aforementioned additional elements beyond the recited abstract idea are not sufficient to amount to significantly more than the recited abstract idea because, as an order combination, the additional elements are no more than mere instructions to implement the idea using generic computer components (i.e. apply it). Additionally, as an order combination, the additional elements append the recited abstract idea to well-understood, routine, and conventional activities in the field as individually evinced by the applicant’s own disclosure, as required by the Berkheimer Memo, in at least: [0015] Accordingly, examples are disclosed that relate to utilizing computer- implemented behavioral elicitation events (BEEs) to automate the evaluation of a training performance of a training subject against a competency framework. A BEE is an event that elicits particular behaviors out of the training subject to demonstrate specified soft skill competencies in the competency framework. The training performance is performed by the training subject in a training environment that simulates a physical environment. Example training environments are a fixed training device (FTD) flight simulator, a full motion simulator (FMS) flight simulator, and a virtual training environment. As described in more detail below, the disclosed examples utilize a BEE database comprising a plurality of BEEs to evaluate the training performance. Each BEE of the plurality of BEEs comprises one or more triggers, one or more elicited actions, and one or more observable behaviors, as discussed in more detail with reference to FIG. 2. The disclosed examples receive simulation state updates from the training environment, and determine a set of active BEEs based on the simulation state update by querying the BEE database. The simulation state update indicates a change in a simulation state of the training environment. [0049] In some examples the methods and processes described herein can be tied to a computing system of one or more computing devices. In particular, such methods and processes can be implemented in hardware as described above, as a computer-application program or service, an application-programming interface (API), a library, and/or other computer-program product. [0050] FIG. 9 schematically shows a simplified representation of a computing system 900 configured to provide any to all of the compute functionality described herein. Computing system 900 can take the form of one or more personal computers, server computers, and computers integrated with aircraft, as examples. Computing system 100,training computing system 108, and computing system 402 are examples of computing system 900. [0051] Computing system 900 includes a logic subsystem 902 and a storage subsystem 904. Computing system 900 can optionally include a display subsystem 906,input subsystem 908,communication subsystem 910, and/or other subsystems not shown in FIG. 9. [0052] Logic subsystem 902 includes one or more physical devices configured to execute instructions. For example, the logic subsystem can be configured to execute instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions can be implemented to perform a task, implement a data type, transform the state of one or more components, achieve a technical effect, or otherwise arrive at a desired result. [0053] The logic subsystem can include one or more hardware processors configured to execute software instructions. Additionally, or alternatively, the logic subsystem can include one or more hardware or firmware devices configured to execute hardware or firmware instructions. Processors of the logic subsystem can be single-core or multi-core, and the instructions executed thereon can be configured for sequential, parallel, and/or distributed processing. Individual components of the logic subsystem optionally can be distributed among two or more separate devices, which can be remotely located and/or configured for coordinated processing. Aspects of the logic subsystem can be virtualized and executed by remotely accessible, networked computing devices configured in a cloud-computing configuration. [0054]Storage subsystem 904 includes one or more physical devices configured to temporarily and/or permanently hold computer information such as data and instructions executable by the logic subsystem. When the storage subsystem includes two or more devices, the devices can be collocated and/or remotely located. Storage subsystem 904 can include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content-addressable devices. Storage subsystem 904 can include removable and/or built-in devices. When the logic subsystem executes instructions, the state of storage subsystem 904 can be transformed - e.g., to hold different data. [0055]Storage subsystem 904 can include removable and/or built-in devices. Storage subsystem 904 can include optical memory (e.g., CD, DVD, HD-DVD, Blu- Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and/or magnetic memory, among others. Storage subsystem 904 can include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content-addressable devices. [0056] Aspects of logic subsystem 902 and storage subsystem 904 can be integrated together into one or more hardware-logic components. Such hardware-logic components can include program- and application-specific integrated circuits (PASIC/ASICs), program- and application-specific standard products (PSSP / ASSPs), system- on-a-chip (SOC), and complex programmable logic devices (CPLDs), for example. [0057] The logic subsystem and the storage subsystem can cooperate to instantiate one or more logic machines. As used herein, the term "machine" is used to collectively refer to the combination of hardware, firmware, software, instructions, and/or any other components cooperating to provide computer functionality. In other words, "machines" are never abstract ideas and always have a tangible form. A machine can be instantiated by a single computing device, or a machine can include two or more sub-components instantiated by two or more different computing devices. In some implementations a machine includes a local component (e.g., software application executed by a computer processor) cooperating with a remote component (e.g., cloud computing service provided by a network of server computers). The software and/or other instructions that give a particular machine its functionality can optionally be saved as one or more unexecuted modules on one or more suitable storage devices. [0058] When included, display subsystem 906 can be used to present a visual representation of data held by storage subsystem 904. This visual representation can take the form of a graphical user interface (GUI). As the herein described methods and processes change the data held by the storage subsystem, and thus transform the state of the storage subsystem, the state of display subsystem 906 can likewise be transformed to visually represent changes in the underlying data. Display subsystem 906 can include one or more display devices utilizing virtually any type of technology. Such display devices can be combined with the logic subsystem and the storage subsystem in a shared enclosure, or such display devices can be peripheral display devices. [0059] When included, input subsystem 908 can comprise or interface with one or more input devices such as a keyboard and touch screen. In some examples, the input subsystem can comprise or interface with selected natural user input (NUI) componentry. Such componentry can be integrated or peripheral, and the transduction and/or processing of input actions can be handled on- or off-board. Example NUI componentry can include a microphone for speech and/or voice recognition; and an infrared, color, stereoscopic, and/or depth camera for machine vision and/or gesture recognition. [0060] When included, communication subsystem 910 can be configured to communicatively couple computing system 900 with one or more other computing devices. Communication subsystem 910 can include wired and/or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem can be configured for communication via a wireless telephone network, or a wired or wireless local- or wide-area network. In some examples, the communication subsystem may allow computing system 900 to send and/or receive messages to and/or from other devices via a network such as the Internet. [0082] It will be understood that the configurations and/or approaches described herein are exemplary in nature, and that these specific embodiments or examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. As such, various acts illustrated and/or described may be performed in the sequence illustrated and/or described, in other sequences, in parallel, or omitted. Likewise, the order of the above-described processes may be changed. [0083] The subject matter of the present disclosure includes all novel and non- obvious combinations and sub-combinations of the various processes, systems and configurations, and other features, functions, acts, and/or properties disclosed herein, as well as any and all equivalents thereof. Furthermore, as an ordered combination, these elements amount to generic computer components receiving or transmitting data over a network, performing repetitive calculations, electronic record keeping, and storing and retrieving information in memory, which, as held by the courts, are well-understood, routine, and conventional. See MPEP 2106.05(d). Moreover, the remaining elements of dependent claims do not transform the recited abstract idea into a patent eligible invention because these remaining elements merely recite further abstract limitations that provide nothing more than simply a narrowing of the abstract idea recited in the independent claims. Looking at these limitations as an ordered combination adds nothing additional that is sufficient to amount to significantly more than the recited abstract idea because they simply provide instructions to use a generic arrangement of generic computer components to “apply” the recited abstract idea, perform insignificant extra-solution activity, and generally link the abstract idea to a technical environment. Thus, the elements of the claims, considered both individually and as an ordered combination, are not sufficient to ensure that the claim as a whole amounts to significantly more than the abstract idea itself. Since there are no limitations in these claims that transform the exception into a patent eligible application such that these claims amount to significantly more than the exception itself, claims 1-7, 9-14, and 16-20, 22-23 are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter. Conclusion Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to PO HAN MAX LEE whose telephone number is (571)272-3821. The examiner can normally be reached on Mon-Thurs 8:00 am - 7:00 pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rutao Wu can be reached on (571) 272-6045. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /PO HAN LEE/Primary Examiner, Art Unit 3623
Read full office action

Prosecution Timeline

Show 1 earlier event
Apr 24, 2025
Non-Final Rejection mailed — §101
Jul 24, 2025
Response Filed
Nov 12, 2025
Final Rejection mailed — §101
Mar 12, 2026
Request for Continued Examination
Mar 26, 2026
Response after Non-Final Action
Jun 23, 2026
Non-Final Rejection mailed — §101
Sep 23, 2026
Response Filed
Nov 02, 2026
Final Rejection mailed — §101 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12802027
OPTIMIZING TRUCK LOADING OF PALLETS
1y 9m to grant Granted Nov 17, 2026
Patent 12620040
VALUATION OF HOMES USING GEOGRAPHIC REGIONS OF VARYING GRANULARITY
2y 4m to grant Granted May 05, 2026
Patent 12619941
TECHNICAL CANDIDATE CERTIFICATION SYSTEM
1y 11m to grant Granted May 05, 2026
Patent 12602629
USING MACHINE LEARNING TO PREDICT FLEET MOVES IN HYDRAULIC FRACTURING OPERATIONS
3y 2m to grant Granted Apr 14, 2026
Patent 12548089
OPTIMIZATION OF HYBRID GROWING INFRASTRUCTURE FOR DIFFERENT WEATHER PROFILES AND MARKET CONDITIONS
3y 3m to grant Granted Feb 10, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

5-6
Expected OA Rounds
32%
Grant Probability
73%
With Interview (+41.3%)
3y 7m (~3m remaining)
Median Time to Grant
High
PTA Risk
Based on 169 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month