DETAILED ACTION
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 101
3. Non-Statutory (Directed to a Judicial Exception without an Inventive Concept/Significantly More)
35 U.S.C.101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
● Claims 1-20 are rejected under 35 U.S.C.101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1
The current claims fall within one of the four statutory categories of invention (MPEP 2106.03).
Step 2A [Wingdings font/0xE0] Prong One:
The claim(s) recite a judicial exception, namely an abstract idea, as shown below:
— Considering each of claims 1, 8 and 16 as the representative claims, the following claimed limitations recite an abstract idea:
Claim 1:
[collect] flight simulation data generated by an instructor user to produce a simulated flight demonstration, the flight simulation data including flight visual re-creation data for recreating flight visuals, physical control element states, instructor user body states, and auditory data;
validate the flight demonstration for correctness; and
[present] the validated flight demonstration to at least one trainee user, wherein the validated flight demonstration includes time-synchronized recorded flight simulation data generated by the instructor user.
Claim 8:
[collect] flight simulation data to produce a simulated flight demonstration, the flight simulation data including flight visual re-creation data for recreating flight visuals, physical control element states, instructor user body states, and auditory data;
validate the flight demonstration for correctness; and
[present] the validated flight demonstration to at least one trainee user, wherein the validated flight demonstration includes time-synchronized recorded flight simulation data.
Claim 16:
[collect] flight simulation data generated by a trainee user during a flight training session, the flight simulation data including flight visual re-creation data for recreating flight visuals, physical control element states, trainee user body states, and auditory data; and
[present] a simulation formed from the flight simulation data to the trainee user, wherein the simulation includes time-synchronized flight simulation data generated by the trainee user.
Thus, the limitations identified above recite an abstract idea since the limitations correspond to certain methods of organizing human activity, and/or mental processes, which are part of the enumerated groupings of abstract ideas identified according to the current eligibility standard (see MPEP 2106.04(a)). For instance, the current claims correspond to managing personal behavior; such as teaching, wherein educational content, which is created based on recording the activities that an instructor and/or a trainee are performing, is presented to the trainee after preparing and/or validating the content.
Similarly, given the limitations that recite: flight simulation data that includes flight visual re-creation data for recreating flight visuals, physical control element states, instructor user body states, and auditory data; validating the simulated flight demonstration for correctness; the validated simulated flight demonstration includes time-synchronized playback of the recorded flight simulation data, etc., the claims also correspond to mental processes—such as, an observation, an evaluation, and/or a judgment process.
Step 2A [Wingdings font/0xE0] Prong Two:
The claim(s) recite additional element(s), wherein a virtual reality system, which includes a flight simulation training device, is utilized as a tool to facilitate the recited functions/steps regarding: collection information (e.g., “recording flight simulation data generated . . . the flight simulation data including flight visual re-creation data for recreating flight visuals, physical control element states, instructor user body states, and auditory data”): evaluating/analyzing the collected information (e.g., “validating the simulated flight demonstration for correctness”); generating/presenting training content (e.g., “playing the validated simulated flight demonstration to at least one trainee user . . . simulated flight demonstration includes time-synchronized playback of the recorded flight simulation data generated . . .”), etc.
However, the claimed additional element(s) fail to integrate the abstract idea into a patent-eligible practical application since the additional element(s) are utilized merely as a tool to facilitate the abstract idea. Accordingly, when each of the claims is considered as a whole, the additional element(s) fail to impose meaningful limits on practicing the abstract idea. For instance, when each of the claims is considered as a whole, none of the claims provides an improvement over the relevant existing technology.
The observations above confirm that the claims are indeed directed to an abstract idea.
Step 2B:
Accordingly, when the claim(s) is considered as a whole (i.e., considering all claim elements both individually and in combination), the claimed additional elements do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amounts to “significantly more” than the abstract idea itself (also see MPEP 2106). The claimed additional elements are directed to conventional computer elements, which are serving merely to perform conventional computer functions.
Accordingly, when each of the current claims is considered as a whole (e.g., see the discussion under Prong Two above regarding such consideration of the claim as a whole), none of the claims recites an element—or a combination of elements—directed to an inventive concept.
It is worth noting that the use of the conventional computer/network technology to facilitate the generation and/or presentation of content to a user(s); such as, the process of collecting/recording one or more scenarios and generating one or more educational content materials, including presenting the generated educational content material(s) to a user after validating the content material(s), etc., is directed to a well-understood, routine, conventional activity in the art (e.g., see US 2010/0092926; US 2016/0019808, etc.).
The above observation confirms that the current claimed invention fails to amount to “significantly more” than an abstract idea.
It is worth noting that the above analysis already encompasses each of the current dependent claims (i.e., claims 2-7, 9-15 and 17-20). Particularly, each of the dependent claims also fails to amount to “significantly more” than the abstract idea since each dependent claim is directed to a further abstract idea, and/or a further conventional computer element(s) utilized to facilitate the abstract idea.
Accordingly, the findings above demonstrate that none of the claims implements an element—or a combination of elements—directed to an inventive concept (e.g., none of the current claims is reciting an element—or a combination of elements—that provides a technological improvement over the existing/conventional technology).
35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph
4. The following is a quotation of 35 U.S.C.112(f):
(f) ELEMENT IN CLAIM FOR A COMBINATION.—An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C.112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
● Claims 6 and 14 invoke 35 U.S.C.112(f) or pre AIA 35 U.S.C.112, sixth paragraph for the following reasons.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as "configured to" or "so that"; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are (see claims 6 and 14):
“a required data parser”, which is used for “extracting required data”;
“a replay data parser”, which is used for “extracting replay data”; and
“a validator”, which is used for “comparing the overlap data with the extracted replay”
However, the written description fails to: (i) disclose the corresponding structure, material, or acts that correspond to the above parsers and validator, and (ii) clearly link the corresponding (if any) structure, material, or acts to the claimed function(s) associated with each of the above parsers and validator. Instead, the specification appears to simply use the terminologies, as recited in the claims, without sufficient description regarding the structure and/or material (if any) that corresponds to any of the above elements (see [0015], [0023], [0072], etc.).
Claim Rejections - 35 USC § 103
5. The following is a quotation of 35 U.S.C.103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating
obviousness or nonobviousness.
Note that the one or more citations (paragraphs or columns) presented in this office action regarding the teaching of a cited reference(s) are exemplary only. Accordingly, such citation(s) are not intended to limit/restrict the teaching of the reference(s) to the cited portion(s) only. Applicant is required to evaluate the entire disclosure of each reference; such as additional portions that teach or suggest the claimed limitations.
● Claims 1-18 are rejected under 35 U.S.C.103 as being unpatentable over Fabling 2010/0092926 in view of Chavez 2016/0019808.
Regarding claim 1, Fabling teaches the following claimed limitations: a method of flight simulation having a realistic instructor demonstration, the method comprising: recording flight simulation data generated by an instructor user in a flight simulation training device (FSTD) to produce a simulated flight demonstration ([0028] to [0031]; [0037]; [0038]: e.g., a system/method for flight crew training; wherein the implementation involves a flight deck simulator and a video camera system; and accordingly, the actions that the flight crew members are performing, including that of the instructor, are recorded in order to create a flight training video. In this regard, at least one of the participants represent the instructor user; whereas, the flight deck simulator corresponds to the FSTD), the flight simulation data including flight visual re-creation data for recreating flight visuals, physical control element states, instructor user body states, and auditory data ([0040]; [0066] to [0070]; [0074]; [0085]: e.g., the scenario being recorded already includes: (i) flight visual re-creation data for recreating flight visuals—such as, display data from electronic flight [FIG 1, label “‘116”] and display data from flight deck display [FIG 1, label “112”], (ii) physical control element states—such as, operation or manipulation of the control panels, (iii) instructor user body states—such as, the actions of one of the flight crew members or the actions of the flight instructor, and (iv) auditory data—such as, communications exchanged between flight crew members, comment made by the flight instructor, etc.); validating the simulated flight demonstration for correctness; and playing the validated simulated flight demonstration to at least one trainee user, the [demonstration] being viewable by the trainee user on at least one visual interface, wherein the simulated flight demonstration includes time synchronized playback of the recorded flight simulation data generated by the instructor user ([0091] to [0094]: e.g., once the scenario is recorded as already discussed above, it is edited—such as: changing the flight displays being presented based on the action(s) of a flight crew member(s), indication of which controls are being manipulated, etc. Thus, the above process of editing corresponds to the process of validating the simulated flight demonstration for correctness. Of course, once the editing process is completed, the flight training video is distributed to students; and this indicates, at least implicitly, that the video is played to at least one trainee—i.e., the trainee views the training video via a visual interface; also see claim 7. Note that given the proper editing process discussed above, the training video being displayed already correlates one or more of the flight displays with a corresponding action(s) of a flight crew member(s), etc., also see claim 4; thus, the simulated flight demonstration already includes time synchronized playback of the recorded flight simulation data generated by the instructor user).
Fabling does not expressly teach the simulation above as a virtual reality simulation, wherein the virtual simulated flight demonstration is played in a virtual environment of the FSTD.
However, Chavez teaches a pilot training system that incorporates a physical flight simulator (FIG 1, label “5”), including a head-mounted display (FIG 1, label “4”) that a trainee wears; and thereby the system provides the trainee with training video scenarios via augmented reality, which coordinates the trainee’s physical actions with one or more video scenarios being displayed to the trainee ([0022] to [0024]; [0031]; [0032]).
Accordingly, given the above teaching, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Fabling in view of Chavez; for example, by upgrading the system’s algorithm—such as, implementing one or more well-known techniques to generate augmented reality scenarios based on the recorded video scenes; and wherein one or more additional devices—such as, a head-mounted display (HMD) that implements one or more sensors, is also incorporated; so that the trainee wears the HMD when conducting the training; and thereby, the system provides the trainee with one or more training scenarios in the form of augmented reality—such as, modifying one or more of the visual scenes in response to one or more detected head position/orientation and/or physical actions of the trainee, etc., so that the trainee would have the option to actively interact with one or more of the training scenarios; and such arrangement further advances the trainee’s skills.
Regarding claim 2, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 1.
The limitation, “the virtual environment includes a virtual reality (VR), augmented reality (AR), or mixed reality (MR) visualization of validated simulated flight demonstration”, is already addressed per the modification discussed with respect to claim 1. For instance, Fabling already validates the training video before it is being played ([0042]; [0092]); and furthermore, Fabling is already modified, per the teaching gleaned from Chavez, so that the system presents—to the trainee—the training scenario in the form of augmented reality (also see the motivation discussed per claim 1 since it also applies to claim 2).
Regarding claim 3, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 1.
The limitation, “rendering the simulated flight demonstration at least once, whereby the simulated flight demonstration provides free head movement in the
virtual environment”, is already addressed per the modification discussed with respect to claim 1. In particular, per the feature incorporated from Chavez, the trainee is already provided with a HMD, which the trainee wears when conducting the training; and thus, as the trainee is moving his/her head, the system generates one or more relevant video scenes based on the detected position/orientation of the trainee’s head ([0031]; [0032]; [0036]).
The above confirms that the modified system not only renders the simulated flight demonstration at least once, but also provides free head movement in the virtual environment (also see the motivation discussed per claim 1 since it also applies to claim 3).
Regarding claim 4, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 3.
However, similar to claim 3 above, the limitation, “wherein rendering the simulated flight demonstration further comprises depicting object positional data within the virtual environment”, is also addressed per the modification discussed with respect to claim 1, or alternatively claim 3. This is because the modified system is displaying one or more relevant visual scenes based on the detected head position/orientation of the trainee (again see [0031], [0032], [0036]). Accordingly, when the trainee turns his/her head to the left, elements that are to the trainee’s left are displayed via the HMD; and similarly, when the trainee turns his/her head to the right, elements that are to the trainee’s right are displayed to the trainee. The above confirms the depiction of object positional data within the virtual environment.
Regarding claim 5, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 1.
Claim 5 is referring merely to a picture depicted in the virtual scenario being displayed, i.e., “an image of the instructor user body is depicted in the virtual environment” (emphasis added).
In this regard, Fabling already teaches that video images of various participants, including flight crew members and an instructor, are recorded as the participants are performing one or more actions and/or interactions (see [0031]; [0038]). Moreover, based on the teaching gleaned from Chavez, Fabling is already modified to display one or more relevant virtual scenarios to the trainee based on detected actions and/or head position/orientation of the trainee ([0031; [0032]; [0036]).
Accordingly, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to further modify Fabling’s system; for example, by providing an option to display one or more of the participants as part of the augmented reality that the system is displaying to the trainee—such as, displaying an image representing the instructor when the trainee is turning to the right or left, etc., so that the trainee would have a more realistic training experience (e.g., a realistic scenario that depicts an instructor providing advice to the trainee, etc.).
Regarding claim 6, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 1.
Fabling further teaches, validating the simulated flight demonstration for correctness comprises: extracting required data with a required data parser; extracting replay data with a replay data parser ([0085]; [0086]; [0091]: e.g., during the recording process, the system gathers various parameters, each representing one or more specific events—such as, one or more actions of a user(s), one or more display elements, etc., and subsequently, based on the determined type of one or more lessons to be to be created, one or more relevant parameters are identified and selected. Thus, the process of identifying and/or selecting one of more relevant parameters form the recorded video indicates the process of extracting required data with a required data parser; whereas, the determining of the one or more lessons to be created—i.e., videos to be replayed once created—indicates the process of extracting replay data with a reply extractor. Note that neither the so-called “required data extractor” nor the so-called “replay data extractor” has any specific structural feature. Instead, each of the above is merely a placeholder); determining overlap data between the extracted required data and the extracted replay data; and comparing the overlap data with the extracted replay using a validator to thereby return a validated flight recording ([0092]; [0093]: e.g., the editing process analyzes parameters that represent one or more actions of a flight crew member(s) with parameters that represent one or more flight displays to be displayed/replayed; and thereby, it changes, based on a particular action(s) taken by a flight crew member(s), the flight displays being presented/replayed in the training video. Accordingly, the analysis of the two sets of parameters above indicates the process of determining overlap data between the extracted required data and the extracted replay data; whereas, the process of presenting/replaying one or more relevant flight displays based on the particular action(s) of the crew member(s) indicates the process of returning a validated video recording after comparing, using a validator, the overlap data with the extracted replay. Here also the so-called “validator”’ is does not have any specific structural feature; rather, it is merely a placeholder).
Regarding claim 7, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 1.
Fabling further teaches, mixing at least a portion of the fight visual recreation data with a subsequent recording to provide the simulated flight demonstration with layered data (see FIG 5; also [0077] to [0079]: e.g., although different view perspectives are captured during the video recording phase, the system displays, as part of presenting the training video to the trainee, the various displays—such as, the flight deck displays and the flight video display. Thus, the above indicates the process of mixing at least a portion of the fight visual recreation data with a subsequent recording to provide the simulated flight demonstration with layered data).
Regarding claim 8, Fabling teaches the following claimed limitations: a method of flight simulation having a realistic instructor demonstration, the method comprising: recording flight simulation data to produce a simulated flight demonstration ([0028]; [0029]; [0038]: e.g., a system/method for flight crew training; wherein the implementation involves recording, via a video camera system, the actions that the flight crew members are performing, including that of the instructor; and subsequently, a flight training video is created. In this regard, at least one of the participants represents the instructor user;), the flight simulation data including flight visual re-creation data for recreating flight visuals, physical control element states, instructor user body states, and auditory data ([0040]; [0066] to [0070]; [0074]; [0085]: e.g., the scenario being recorded already includes: (i) flight visual re-creation data for recreating flight visuals—such as, display data from electronic flight [FIG 1, label “‘116”] and display data from flight deck display [FIG 1, label “112”], (ii) physical control element states—such as, operation or manipulation of the control panels, (iii) instructor user body states—such as, the actions of one of the flight crew members or the actions of the flight instructor, and (iv) auditory data—such as, communications exchanged between flight crew members, comment made by the flight instructor, etc.); validating the simulated flight demonstration for correctness; and playing the validated simulated flight demonstration to at least one trainee user, the [demonstration] being viewable by the trainee user on at least one visual interface, wherein the validated simulated flight demonstration includes time-synchronized playback of the recorded flight simulation data ([0091] to [0094]: e.g., once the scenario is recorded as discussed above, it is edited—such as: changing the flight displays being presented based on the action(s) of a flight crew member(s), indication of which controls are being manipulated, etc. Thus, the above process of editing corresponds to the process of validating the simulated flight demonstration for correctness. Of course, once the editing process is completed, the flight training video is distributed to students; and this indicates, at least implicitly, that the video is played to at least one trainee—i.e., the trainee views the training video via a visual interface; also see claim 7. Note that given the proper editing process discussed above, the training video being displayed already correlates one or more of the flight displays with a corresponding action(s) of a flight crew member(s), etc., also see claim 4; thus, the simulated flight demonstration already includes time synchronized playback of the recorded flight simulation data generated by the instructor user).
Although Fabling implements a flight simulation training device (FSTD) (see [0030], [0031]: e.g., the flight deck simulator corresponds to the flight simulation training device or FSTD), Fabling does not expressly teach the simulation above as a virtual reality simulation, wherein the virtual simulated flight demonstration is played in a virtual environment of the FSTD.
However, Chavez teaches a pilot training system that incorporates a physical flight simulator (FIG 1, label “5”), including a head-mounted display (FIG 1, label “4”) that a trainee wears; and thereby the system provides the trainee with training video scenarios via augmented reality, which coordinates the trainee’s physical actions with one or more video scenarios being displayed to the trainee ([0022] to [0024]; [0031]; [0032]).
Accordingly, given the above teaching, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Fabling in view of Chavez; for example, by upgrading the system’s algorithm—such as, implementing one or more well-known techniques to generate augmented reality scenarios based on the recorded video scenes; and wherein one or more additional devices—such as, a head-mounted display (HMD) that implements one or more sensors, is also incorporated; so that the trainee wears the HMD when conducting the training; and thereby, the system provides the trainee with one or more training scenarios in the form of augmented reality—such as, modifying one or more of the visual scenes in response to one or more detected head position/orientation and/or physical actions of the trainee, etc., so that the trainee would have the option to actively interact with one or more of the training scenarios; and such arrangement further advances the trainee’s skills.
Regarding claim 9, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 8.
Fabling further teaches, the flight simulation data is generated by at least one of: an instructor user in the FSTD; a non-simulated flight in an aircraft; or the trainee user in a previous flight simulation ([0029]; [0037]; [0038]: e.g., as part of creating the flight training video, the data being captured already involves one or more activities that at least one of the participants is performing in the flight deck simulator—such as, the actions of a flight crew member and/or that of the instructor. Thus, the above indicates that the flight simulation data is generated by at least an instructor user in the FSTD).
Regarding claim 10, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 9.
The limitation, “when the flight simulation data is generated form the non-simulated flight in the aircraft, at least a portion of the flight visual re-creation data is derived from at least one of: a sensor carried on the aircraft; or aircraft data from an aircraft bus” (emphasis added), is an optional limitation since the above is referring to one of the alternatives that claim 9 requires. In particular, given the claim language “at least one of”, claim 9 does not necessarily require all listed alternatives, but only one of the listed alternatives. In this regard, Fabling already teaches one of the listed alternatives; namely, “an instructor user in the FSTD” (see the discussion per claim 9 above). Thus, the teaching of Fabling discussed per claim 9 above already addresses claim 10 since claim 10 is directed to an optional limitation.
Regarding claim 11, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 8.
The limitation, “the virtual environment includes a virtual reality (VR), augmented reality (AR), or mixed reality (MR) visualization of validated simulated flight demonstration”, is addressed per the modification discussed with respect to claim 8 above. In particular, Fabling already validates the training video before it is being played ([0042]; [0092]); and furthermore, Fabling is modified, per the teaching gleaned from Chavez, so that the system presents—to the trainee—the training scenario in the form of augmented reality (also see the motivation discussed per claim 8 since it also applies to claim 11).
Regarding claim 12, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 8.
The limitation, “rendering the simulated flight demonstration at least once, whereby the simulated flight demonstration provides free head movement in the
virtual environment”, is already addressed per the modification discussed with respect to claim 8. This is because Fabling already incorporates a HMD, which the trainee wears when conducting the training; and accordingly, as the trainee is moving his/her head to one or more directions, the system generates one or more relevant video scenes based on the detected position/orientation of the trainee’s head ([0031]; [0032]; [0036]).
Thus, besides rendering the simulated flight demonstration at least once, the modified system also provides free head movement in the virtual environment (also see the motivation discussed per claim 8 since it also applies to claim 12).
Regarding claim 13, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 12.
The limitation, “rendering the simulated flight demonstration further comprises depicting object positional data within the virtual environment”, is also addressed per the modification discussed with respect to claim 8 (or claim 12). This is because the modified system is displaying one or more relevant visual scenes based on the detected head position/orientation of the trainee ([0031], [0032], [0036]). Accordingly, when the trainee turns his/her head to: (i) the left, elements that are to the trainee’s left are displayed via the HMD, (ii) the right, elements that are to the trainee’s right are displayed to the trainee.
Thus, the above indicates the process of depicting object positional data within the virtual environment.
Regarding claim 14, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 8.
Fabling further teaches, validating the simulated flight demonstration for correctness comprises: extracting required data with a required data parser; extracting replay data with a replay data parser ([0085]; [0086]; [0091]: e.g., during the recording process, the system gathers various parameters, each representing one or more specific events—such as, one or more actions of a user(s), one or more display elements, etc., and subsequently, based on the determined type of one or more lessons to be to be created, one or more relevant parameters are identified and selected. Thus, the process of identifying and/or selecting one of more relevant parameters form the recorded video indicates the process of extracting required data with a required data parser; whereas, the determining of the one or more lessons to be created—i.e., videos to be replayed once created—indicates the process of extracting replay data with a reply extractor. Note that neither the so-called “required data extractor” nor the so-called “replay data extractor” has any specific structural feature. Instead, each of the above is merely a placeholder); determining overlap data between the extracted required data and the extracted replay data; and comparing the overlap data with the extracted replay using a validator to thereby return a validated flight recording ([0092]; [0093]: e.g., the editing process analyzes parameters that represent one or more actions of a flight crew member(s) with parameters that represent one or more flight displays to be displayed/replayed; and thereby, it changes, based on a particular action(s) taken by a flight crew member(s), the flight displays being presented/replayed in the training video. Accordingly, the analysis of the two sets of parameters above indicates the process of determining overlap data between the extracted required data and the extracted replay data; whereas, the process of presenting/replaying one or more relevant flight displays based on the particular action(s) of the crew member(s) indicates the process of returning a validated video recording after comparing, using a validator, the overlap data with the extracted replay. Here also the so-called “validator”’ is does not have any specific structural feature; rather, it is merely a placeholder).
Regarding claim 15, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 8.
Fabling further teaches, mixing at least a portion of the fight visual recreation data with a subsequent recording to provide the simulated flight demonstration with layered data (see FIG 5; also [0077] to [0079]: e.g., although different view perspectives are captured during the video recording phase, the system displays, as part of presenting the training video to the trainee, the various displays—such as, the flight deck displays and the flight video display. Thus, the above indicates the process of mixing at least a portion of the fight visual recreation data with a subsequent recording to provide the simulated flight demonstration with layered data).
Regarding claim 16, Fabling teaches the following claimed limitations: a method of flight simulation having recording and realistic playback capabilities, the method comprising: recording flight simulation data generated by a trainee user in a flight simulation training device (FSTD) during a flight training session ([0028] to [0030]; [0037]; [0038]: e.g., a system/method for flight crew training; wherein the implementation involves a flight deck simulator and a video camera system; and accordingly, the actions that the flight crew members are performing are recorded in order to create a flight training video. In this regard, at least one of the participants, such as one of the flight crew members, represent the trainee user; and the flight deck simulator corresponds to the FSTD), the flight simulation data including flight visual re-creation data for recreating flight visuals, physical control element states, trainee user body states, and auditory data ([0040]; [0066] to [0070]; [0074]; [0085]: e.g., the scenario being recorded already includes: (i) flight visual re-creation data for recreating flight visuals—such as, display data from electronic flight [FIG 1, label “‘116”] and display data from flight deck display [FIG 1, label “112”], (ii) physical control element states—such as, manipulation of the control panels, (iii) trainee user body states—such as, the actions of one of the flight crew members, and (iv) auditory data—such as, communications exchanged between flight crew members, etc.); and playing a recorded simulation formed from the recorded flight simulation data to the trainee user, the [simulation] being viewable by the trainee user on at least one visual interface, wherein the recorded simulation includes time-synchronized playback of the recorded flight simulation data generated by the trainee user ([0091] to [0094]: e.g., once the scenario is recorded and edited, the flight training video is distributed to students; and this indicates, at least implicitly, the playing of the recorded simulation to the trainee—i.e., the trainee views the training video via a visual interface; also see claim 7. In addition, the editing process already correlates one or more of the flight displays with a corresponding action(s) of a flight crew member(s), etc., also see claim 4; thus, the recorded simulation already includes time-synchronized playback of the recorded flight simulation data generated by the trainee user).
Fabling does not expressly teach the flight simulation above as a virtual reality flight simulation, wherein the recorded simulation is played to the trainee in a virtual environment of the FSTD.
However, Chavez teaches a pilot training system that incorporates a physical flight simulator (FIG 1, label “5”), including a head-mounted display (FIG 1, label “4”) that a trainee wears; and thereby the system provides the trainee with training video scenarios via augmented reality, which coordinates the trainee’s physical actions with one or more video scenarios being displayed to the trainee ([0022] to [0024]; [0031]; [0032]).
Accordingly, given the above teaching, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Fabling in view of Chavez; for example, by upgrading the system’s algorithm—such as, implementing one or more well-known techniques to generate augmented reality scenarios based on the recorded video scenes; and wherein one or more additional devices—such as, a head-mounted display (HMD) that implements one or more sensors, is also incorporated; so that the trainee wears the HMD when conducting the training; and thereby, the system provides the trainee with one or more training scenarios in the form of augmented reality—such as, modifying one or more of the visual scenes in response to one or more detected head position/orientation and/or physical actions of the trainee, etc., so that the trainee would have the option to actively interact with one or more of the training scenarios; and such arrangement further advances the trainee’s skills.
Regarding claim 17, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 16.
Fabling further teaches, generating at least one additional recording; combining the at least one additional recording with the recorded flight simulation data; and playing the combined simulated flight demonstration with the at least one additional recording ([0092] to [0094]: e.g., once the simulation scenario is generated by recording the activities that one or more of the flight crew members are performing in the flight deck simulator, the recorded scenario is further edited, which includes adding one or more image and/or textual parts—such as, providing, as part of the flight training video, an indication of which controls on a control panel are being manipulated during flight crew interactions, etc. Thus, besides generating at least one additional recording, the additional recording is combined with the recorded flight simulation data; and the combined simulated flight demonstration is played with the additional recording).
Regarding claim 18, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 17.
The limitation, “the at least one additional recording further comprises at least one of: a voice recording, a text comment, an image, or a video recording”, is already addressed per the discussion presented above with respect to claim 17. This is because the data being added during the editing process ([0092], [0093]), which includes adding an indication of which controls on a control panel are being manipulated during flight crew interactions, etc., indicates that the additional recording comprises at least one of: a text comment, am image, etc.
● Claims 19 and 20 are rejected under 35 U.S.C.103 as being unpatentable over Fabling 2010/0092926 in view of Chavez 2016/0019808 and in view of Guckenberger 2012/0206577.
Regarding claim 19, Fabling in view of Chavez teaches the claimed limitations as discussed above per claim 17.
Although Fabling already teaches that the combined simulated flight demonstration with the at least one additional recording depicts image of at least one of the trainee user or an instructor user (see FIG 5, labels “508” or “506”), Fabling does describe the image above as an avatar.
However, Guckenberger teaches a training system for training a trainee; wherein the training system displays—via a virtual reality HMD—one or more training scenarios, including a semi-transparent expert/trainer avatar being superimposed on an opaque image representing the trainee (or vice versa), so that the trainer avatar demonstrates to the trainee how to correctly perform one or more tasks ([0033] to [0037]).
Accordingly, given the above teaching, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to further modify the invention of Fabling in view of Guckenberger; for example, by upgrading the system’s algorithm, so that it provides the user—such as the trainee—with an option to convert the image of one or more of the participants—such as the image of one of the crew members and/or the instructor, into an avatar; wherein the system also provides the user with an option to adjust the degree of transparency, etc., so that the trainee would be able to easily view and/or compare, without facing significant obstruction, the action(s) that one or more of the participants are performing; e.g., comparing an action(s) that a semi-transparent avatar, which represents the instructor, is performing against that an opaque avatar, which represents one of the flight crew members, is performing, etc.
Regarding claim 20, Fabling in view of Chavez and in view Guckenberger teaches the claimed limitations as discussed above per claim 19.
The limitation, “wherein at least one of the trainee user or the instructor user is
depicted partially transparent in the virtual environment”, is already addressed per the modification discussed with respect to claim 19. This is because the modified system of Fabling already incorporates an option that allows the user to adjust the degree of transparency of each of the one or more avatars—such as, making the instructor avatar a semi-transparent avatar, etc. (also see the motivation discussed above with respect to claim 19 since it also applies to claim 20).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRUK A GEBREMICHAEL whose telephone number is (571) 270-3079. The examiner can normally be reached from 7:00 AM - 3:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, PETER VASAT can be reached on (571) 270-7625. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRUK A GEBREMICHAEL/Primary Examiner, Art Unit 3715
Prosecution Insights