DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This is a Final Office Action on the merits. Claims 1-9 and 14-24 are currently pending and are addressed below.
Response to Amendment
The drawings were objected to due to minor informalities. Applicant amended the drawings accordingly; therefore, the drawings objection is withdrawn.
The specification was objected to due to minor informalities. Applicant amended the specification accordingly; therefore, the specification objection is withdrawn.
Claims 1, 3-4, 7, and 14 were objected to due to minor informalities. Applicant amended the claims accordingly; therefore, the objection is withdrawn.
Claims 1-18 were rejected under 35 U.S.C. 112 for being indefinite. Applicant amended the claims accordingly; therefore, the rejection is withdrawn.
Claims 1-18 were rejected under 35 U.S.C. 101 for being directed to an abstract idea. Applicant amended the claims accordingly; therefore, the rejection is withdrawn.
Response to Arguments
Applicant’s arguments on pages 14-17 of the response, with respect to the rejection(s) of claim(s) 14-17 under 35 U.S.C. 102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Robinson and Dumas.
Applicant’s arguments on pages 18-19 of the response, with respect to the rejection(s) of claim(s) 7-9 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Robinson and Brinkman.
Applicant’s arguments on pages 19 of the response, with respect to the rejection(s) of claim(s) 18 under 35 U.S.C. 103 have been fully considered but they are not persuasive. Applicant argues that Dupont is silent regarding camera use. However, as mentioned by the Applicant, the Dupont reference was cited to teach “an altimeter connected to a pressure sensor for capturing aircraft altitude data”. Therefore, this argument is not directed to the claim as written since claim 18 does not recite a camera.
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 04/27/2026 is/are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement(s) is/are being considered by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
the data capture means in claims 14-18:
“…for obtaining aircraft position, orientation, altitude, and velocity data…”
the data analysis module in claims 14-18:
“…for determining aircraft relative position and velocity…”
the communication module in claims 14-18:
“…for transmitting and receiving data with a computing device in a second aircraft…”
“…transmitting and receiving data with an operatively coupled augmented reality display device…”
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION. —The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 7-9, 14-18, and 22-24 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 7 recites the limitation "the trailing aircraft" in lines 5-6. There is insufficient antecedent basis for this limitation in the claim.
Claim 7 recites “…a trailing aircraft augmented reality display…” and “…the trailing aircraft operatively coupled augmented reality display…”. It is unclear if these two augmented reality displays are referring to the same display or if they are referring to different displays.
Claim 7 recites “…the display…” and “…the trailing aircraft augmented reality display…”. It is unclear if these two displays are referring to the same display or if they are referring to different displays.
Claim 14 recites “…using an operatively coupled augmented reality display device…”. It is unclear which element or component is operatively coupled to the augmented reality display device.
Claim 23 recites “…actual sizes of the aircraft”. It is unclear which aircraft is being referred to.
Claim 24 recites “…the trailing aircraft display…”. It is unclear if this is referring to the same display recited in parent claim 7: “…the trailing aircraft augmented reality display…”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 7-9 and 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Robinson of US 20210049925 A1, filed 10/30/2020, hereinafter “Robinson”, in view of Brinkman of US 20100283635 A1, filed 05/05/2009, hereinafter “Brinkman”.
Regarding claim 7, Robinson teaches:
A method for identifying a leading aircraft on a trailing aircraft augmented reality display and arranging formation flight information on the trailing aircraft operatively coupled augmented reality display to be worn by a pilot in an aircraft flying in formation accordingly comprising: (See at least [0161]: “FIG. 13 shows an augmented reality view 1300 shown to a real pilot (such as pilot 1302), in accordance with an exemplary embodiment. Further, the augmented reality view 1300 may be generated and displayed over a virtual reality display. For example, the virtual reality display may include a head-mounted display (HMD), eyeglasses, Head-Up Display (HUD), smart contact lenses, a virtual retinal display, an eye tap, a Primary Flight Display (PFD) and a cockpit glass etc. Further, the augmented reality view 1300 may assist a pilot 1302 in flying a civilian aircraft 1304.”)
forming a subsequent couple using computing devices, wherein a computing device on the leading aircraft is a lead computing device and a computing device on the trailing aircraft is a trail computing device; (See at least [0063]: “Systems and methods according to the principles of the present invention may involve training a plurality of pilots, each in a separate real aircraft, where the plurality of separate aircraft share a common physical environment. This may be useful in a training situation where two or more planes are flying in close proximity and are being presented with a common enemy asset in augmented reality. This could be a dog fight, missile aversion, target bombing, etc. Such systems and methods may include providing a head mounted see-through computer display (HMD) to each of the plurality of pilots such that each of the plurality of pilots can view a common virtual environment with computer rendered training content. Each of the aircraft may track and report its own location, attitude, speed, or other information to a computer simulation system such that the simulation system can manage the training simulation. The simulation system may position the computer rendered training content at a geospatial location within a visual range of each of the plurality of pilots or one of the pilots and the content may be presented to the HMD of each of the plurality of pilots, wherein the presentation in each individual HMD is dependent on an alignment of each respective HMD and the computer rendered content geospatial location.”)
determining a location of the leading aircraft on the trailing aircraft augmented reality display using data from the lead computing device, data from the trail computing device, and data from one or more gyroscopes and/or one or more compasses associated with the trailing augmented reality display; (See at least [0067]: “In embodiments, the presentation of the computer generated content to each of the plurality of pilots may be based on an alignment between each of the plurality of pilots viewing direction and the computer generated content's geospatial location such that each pilot sees the computer generated content when each pilot's aircraft position, pilot viewing direction and the content's geospatial location align in an unobstructed line of sight. For example, if a plane is flying level and within visual range of the content's geospatial location, the pilot may see the content if it is in front of the plane and above the plane horizon such that the pilot can see the geospatial location through the cockpit window. If, on the other hand, the content is directly behind the plane and the pilot cannot turn his head to view the goespatial location of the content, than the content may not be presented in the pilot's HMD” & [0031]: “A system and method according to the principles of the present invention update the relative geometric understanding describing the relationship between the vehicle and the virtual marker. The system may further include in the relative geometric understanding the vehicle operator's head location and viewing position and/or eye position. To maintain an accurate geometric understanding, a system and method may track information from sensors mounted within the vehicle, including a one or more sensors such as…gyroscopes…”)
arranging formation flight information for a virtual overlay on the trailing aircraft augmented reality display such that the formation flight information does not obstruct the view of the leading aircraft, wherein the size of the leading aircraft on the display is proportional to its actual size; and displaying, on the virtual overlay on the trailing aircraft augmented reality display, the formation flight information arrangement. (See at least [0170]: “Further, the augmented reality view 1300 may include one or more live aircraft (representing real pilots flying real aircraft), one or more virtual aircraft (representing real people on the ground, flying virtual aircraft) and one or more constructed aircraft (representing aircraft generated and controlled using computer graphics and processing systems). Further, the augmented reality view 1300 shown to a pilot (such as the pilot 1302) in a first aircraft (such as the civilian aircraft 1304) may be modified based on sensor data received from another aircraft (such as another airplane 1310). The sensor data may include data received from one or more internal sensors to track and localize the pilot's head within the cockpit of the aircraft. Further, the sensor data may include data received from one or more external sensors to track the position and orientation of the aircraft. Further, the data received from the one or more internal sensors and the one or more external sensors may be combined to provide a highly usable augmented reality solution in a fast-moving environment.”)
Robinson does not explicitly teach:
determining an actual size of the leading aircraft using data from the lead computing device and/or data from the trail computing device;
However, Robinson does teach an augmented reality view that shows a nearby aircraft, wherein “the augmented reality view 1300 shown to a pilot (such as the pilot 1302) in a first aircraft (such as the civilian aircraft 1304) may be modified based on sensor data received from another aircraft (such as another airplane 1310)” and wherein the sensor data may include a combination of the position and orientation of the aircraft and the tracking/localization data of the pilot’s head (See at least Fig. 13 & [0170]). Furthermore, Robinson teaches “continuously updating the augmented and virtual reality content shown to the real pilot (on the ground) flying the virtual aircraft based on the tracking the real pilot and the real aircraft, the operational state (e.g. location, speed, direction of travel, etc.) of the virtual aircraft flown by the real people (on the ground) and the operational state (e.g. location, speed, direction of travel, etc.) of the constructed aircraft” (See at least [0145-0148]). Therefore, since the augmented reality view shows another aircraft and is updated based on sensor data from that aircraft, the teachings of Robinson render obvious determining the size of an aircraft so it can be displayed in the augmented reality view, which provides the benefit of “drawing an accurate picture of an enemy aircraft in augmented and virtual reality display device 106 worn by a pilot” (See [0059] of Robinson).
Additionally, Brinkman teaches:
determining an actual size of the leading aircraft using data from the lead computing device and/or data from the trail computing device; (See at least [0026]: “Notably, in certain embodiments, onboard radar 32 may be utilized to approximate a neighboring aircraft's size, heading, and/or speed even when external source data (e.g., ADS-B data provided from the neighboring aircraft or TIS-B data provided from an external control source) is currently unavailable.”)
One having ordinary skill in the art, before the effective filing date of the claimed invention, would have found it obvious to combine Robinson’s method with Brinkman’s technique of determining an actual size of the leading aircraft using data from the lead computing device and/or data from the trail computing device. Doing so would be obvious “for visually expressing flight characteristics pertaining to neighboring aircraft in an intuitive and readily comprehendible manner” (See [0036] of Brinkman).
Regarding claim 8, Robinson and Brinkman in combination teach all the limitations of claim 7 as discussed above.
Robinson additionally teaches:
wherein the formation flight information on the virtual overlay of the trailing aircraft augmented reality display is continually rearranged in response to changes in the location and proximity of the coupled leading aircraft. (See at least [0145-0147]: “Moreover, at 1010, the method 1000 may include continuously updating the augmented and virtual reality content shown to the real pilot flying the real aircraft based on the tracking of the real pilot and the real aircraft. In some embodiments, the augmented and virtual reality content shown to the real pilot flying the real aircraft may be updated based on the operational state (e.g. location, speed, direction of travel, etc.) of the virtual aircraft flown by the real people (on the ground) and the operational state (e.g. location, speed, direction of travel, etc.) of the constructed aircraft. In some embodiments, the method 1000 may include continuously updating the augmented and virtual reality content shown to the real pilot (on the ground) flying the virtual aircraft based on the tracking the real pilot and the real aircraft, the operational state (e.g. location, speed, direction of travel, etc.) of the virtual aircraft flown by the real people (on the ground) and the operational state (e.g. location, speed, direction of travel, etc.) of the constructed aircraft.”)
Regarding claim 9, Robinson and Brinkman in combination teach all the limitations of claim 8 as discussed above.
Robinson additionally teaches:
wherein the data collected from the lead computing device and trail computing device to determine location of the leading aircraft on the trailing aircraft augmented reality display comprises position and altitude data. (See at least [0078]: “…the at least one first sensor 310 may be configured for sensing at least one first physical variable associated with the first vehicle 308. Further, the at least one second sensor 320 may be configured for sensing at least one second physical variable associated with the second vehicle. In further embodiments, the at least one first physical variable may include one or more of…a first location…” & [0121]: “...based on the first location including a first altitude associated with the first vehicle 708…”.)
Regarding claim 23, Robinson and Brinkman in combination teach all the limitations of claim 7 as discussed above.
Brinkman additionally teaches:
further comprising: estimating the spacing of the leading aircraft from the trailing aircraft based on actual sizes of the aircraft. (See at least [0019]: “To commence main process 40 (STEP 44, FIG. 2), processor 22 first determines the position of the closest neighboring aircraft utilizing data provided by at least one of the air traffic data sources, such as signals provided from onboard radar 32, TIS-B data provided from one or more external control sources and received via wireless transceiver 28, and/or ADS-B data provided from the neighboring aircraft and also received via wireless transceiver 28. If the data received during STEP 44 indicates that a neighboring aircraft is not within a predetermined distance of the host aircraft (STEP 46 FIG. 2), processor 22 returns to STEP 44 and the process repeats. If, instead, the data indicates that a neighboring aircraft is within a predetermined distance of the host aircraft, processor 22 generates a graphic indicative of the neighboring aircraft's current detected position on monitor 24 (STEP 48). For example, and with reference to FIG. 3, processor 22 may generate a three dimensional rendered model 51 representing the neighboring aircraft's current detected position.”)
Claim(s) 14-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Robinson in view of Dumas of US 20190094538 A1, filed 03/28/2019, hereinafter “Dumas”.
Regarding claim 14, Robinson teaches:
A system for calculating and displaying formation flight information in an aircraft, using an operatively coupled augmented reality display device to be worn by a pilot in an aircraft flying in formation, the system comprising: (See at least Abstract: “…The training system may include an aircraft sensor system affixed to the aircraft adapted to provide a location of the aircraft, including an altitude of the aircraft, speed of the aircraft, and directional attitude of the aircraft…The helmet may include a see-through computer display through which the pilot sees an environment outside of the aircraft with computer content overlaying the environment to create an augmented reality view of the environment for the pilot…”)
one or more augmented reality display devices wearable by a pilot of an aircraft, (See at least [0054]: “The augmented and virtual reality display device 106 may display content to a pilot flying the aircraft 200. The augmented and virtual reality display device 106 may be one of a head-mounted display (HMD)…”)
each located within a different aircraft in the formation flight, (See at least [0159]: “…each pilot may see other pilot's virtual image in their augmented and virtual reality equipment.”)
each augmented reality display device comprising a display screen at least partially transparent which is configured to display formation flight information,(See at least [0100]: “…the first head mount display 400 may include a display device 406 to present visuals. The display device may a first see-through display device”)
one or more computing devices, each located within a different aircraft in the formation, to include at least one aircraft in the formation containing the augmented reality display device, (See at least [0081]: “…the first presentation device may include a first head mount display. Further, the second presentation device may include a second head mount display” & [0098]: “…the first processor may be configured for controlling presentation of the at least one first presentation data on the at least one first presentation device 314…the second processor may be configured for controlling presentation of the at least one second presentation data on the at least one second presentation device 324…”)
each computing device comprising and/or operatively connected to: data capture means for obtaining aircraft position, orientation, altitude, and velocity data; (See at least [0087]: “…the at least one first sensor 310 may be communicatively coupled to a first processor associated with the vehicle. Further, the first processor may be configured for determining one or more of the first user location and the first user orientation based on analysis of the first image…” & [0078]: “…the at least one first sensor 310 may be configured for sensing at least one first physical variable associated with the first vehicle 308…the at least one first physical variable may include…a first location, a first speed…”. See also [0121] regarding the first location including a first altitude.)
a data analysis module for determining aircraft relative position and velocity; (See at least [0151]: “…For example, a real pilot in a real aircraft flying over Nevada may be able to ‘see’ a second plane that is actually flying over Virginia as an augmented reality representation in close proximity to the real aircraft. The relative positioning of the representation of the second aircraft to the real aircraft may be programmed based on the training scenario. The scenario, for example, may begin by geospatially locating the two visually separated aircraft within 50 feet of one another in a common virtual airspace… If the either plane makes a real maneuver that affects the relative position of the two aircraft in the virtual environment, the result will be shown by changing the position of the representation of the other aircraft. If the second aircraft puts on its afterburners and the first does not, the pilot of the first aircraft will see the second aircraft pull away in the virtual airspace as the second aircraft flies faster in its real airspace.”)
a communication module for: transmitting and receiving data with the one or more computing devices in a second aircraft; and (See at least [0073]: “…the communication device 302 may be configured for transmitting at least one second presentation data to at least one second presentation device 324 associated with the second vehicle 318. Further, the at least one second presentation device 324 may include a second receiver 326 configured for receiving the at least one second presentation data over the second communication channel…”)
transmitting and receiving data with the operatively coupled augmented reality display device; and (See at least [0073]: “…the communication device 302 may be configured for transmitting at least one second presentation data to at least one second presentation device 324 associated with the second vehicle 318. Further, the at least one second presentation device 324 may include a second receiver 326 configured for receiving the at least one second presentation data over the second communication channel…” & [0081]: “…the first presentation device may include a first head mount display. Further, the second presentation device may include a second head mount display.”)
a virtual overlay program for displaying(See at least [0150-0151]: “…the common virtual airspace includes a computer generated training asset that is viewable by an operator of each vehicle as content overlaying a real airspace surrounding each of the respective vehicles. It is presented as augmented reality content. A system may identify a geospatial location for each of the two or more vehicles within the virtual airspace, which may be based on the vehicle's actual geospatial locations within their respective airspace and represented within the common virtual airspace…If the either plane makes a real maneuver that affects the relative position of the two aircraft in the virtual environment, the result will be shown by changing the position of the representation of the other aircraft. If the second aircraft puts on its afterburners and the first does not, the pilot of the first aircraft will see the second aircraft pull away in the virtual airspace as the second aircraft flies faster in its real airspace.”)
Robinson does not explicitly teach:
the one or more augmented reality display devices further comprising one or more outward-facing cameras mounted on a helmet wearable by the pilot collecting imagery from the vantage point of the pilot to detect and track other aircraft;
…based on the pilot’s vantage point from the collected imagery…
Dumas teaches:
the one or more augmented reality display devices further comprising one or more outward-facing cameras mounted on a helmet wearable by the pilot collecting imagery from the vantage point of the pilot to detect and track other aircraft; (See at least [0047]: “With reference to FIG. 1, the display system 10 of such a device according to the present invention, comprises at least one image sensor 12 that is capable of acquiring environmental images along the path traveled by an aircraft…” & [0058]: “According to a fourth variant, the image sensor 12 is mounted on a helmet worn by the pilot of the aircraft, for example the helmet of a head-up system capable of projecting a synthetic image, comprising information for assisting the piloting of the aircraft in order to be visually superimposed on the pilot's visual field of navigation.”)
…based on the pilot’s vantage point from the collected imagery… (See at least [0058]: “According to a fourth variant, the image sensor 12 is mounted on a helmet worn by the pilot of the aircraft, for example the helmet of a head-up system capable of projecting a synthetic image, comprising information for assisting the piloting of the aircraft in order to be visually superimposed on the pilot's visual field of navigation. According to this fourth variant, as detailed below, the image processing module 14 then comprises (an) image correction tool(s) capable of correcting the images acquired by the image sensor 12 through re-projection of these in a geographical horizontal plane (i.e. re-projection in a predetermined common reference for the processing of all acquired images). The implementation of such a fourth mounted variant of the image sensor 12 is advantageous because it takes advantage of the mobility possibilities of the driver's head which instinctively tends to look in the direction of external visual cues visible in actual flight conditions.”)
One having ordinary skill in the art, before the effective filing date of the claimed invention, would have found it obvious to combine Robinson’s system with Dumas’s augmented reality display devices comprising an outward-facing camera mounted on a helmet wearable by the pilot collecting imagery from the vantage point of the pilot to detect and track other aircraft. Doing so would be obvious “because it takes advantage of the mobility possibilities of the driver's head which instinctively tends to look in the direction of external visual cues visible in actual flight conditions” (See [0058] of Dumas).
Regarding claim 15, Robinson and Dumas in combination teach all the limitations of claim 14 as discussed above.
Robinson additionally teaches:
wherein the data capture means for obtaining aircraft position, orientation, altitude, and velocity data comprises one or more global positioning system (GPS) receivers for capturing aircraft position and altitude data. (See at least [0050]: “…a geospatial location system adapted to identify a current location of a vehicle (e.g. GPS…,” [0121]: “…the first location including a first altitude associated with the first vehicle 708…” & [0031]: “To maintain an accurate geometric understanding, a system and method may track information from sensors mounted within the vehicle, including a one or more sensors such as GPS, airspeed sensor, vertical airspeed sensor, stall sensor, IMU, G-Force sensor, avionics sensors, compass, altimeter, angle sensor, attitude heading and reference system sensors, angle of attack sensor, roll sensor, pitch sensor, yaw sensor, force sensors, vibration sensors, gyroscopes, engine sensors, tachometer, control surface sensors, etc.”)
Regarding claim 16, Robinson and Dumas in combination teach all the limitations of claim 14 as discussed above.
Robinson additionally teaches:
wherein the data capture means for obtaining aircraft position, orientation, altitude, and velocity data comprises one or more gyroscopes for capturing aircraft orientation data. (See at least [0031]: “…To maintain an accurate geometric understanding, a system and method may track information from sensors mounted within the vehicle, including a one or more sensors such as…gyroscopes…”)
Regarding claim 17, Robinson and Dumas in combination teach all the limitations of claim 14 as discussed above.
Robinson additionally teaches:
wherein the data capture means for obtaining aircraft position, orientation, altitude, and velocity data comprises one or more accelerometers for capturing aircraft velocity data. (See at least [0077-0078]: “…the at least one first sensor 310 may include one or more of…a first accelerometer…the at least one first sensor 310 may be configured for sensing at least one first physical variable associated with the first vehicle 308…the at least one first physical variable may include…a first speed…”)
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Robinson in view of Dumas and further in view of Dupont De Dinechin of US 20150054664 A1, filed 03/26/2013, hereinafter “Dupont”.
Regarding claim 18, Robinson and Dumas in combination teach all the limitations of claim 14 as discussed above.
Robinson does not explicitly teach:
wherein the data capture means for obtaining aircraft position, orientation, altitude, and velocity data comprises one or more pressure sensors for capturing aircraft altitude data.
Dupont teaches:
wherein the data capture means for obtaining aircraft position, orientation, altitude, and velocity data comprises one or more pressure sensors for capturing aircraft altitude data. (See at least [0020]: “a primary altitude of the aircraft determined from measurements by said static pressure sensor…” & [0080]: “The altimeter 6 is connected to the static pressure sensor 5a and the computer 3. It is capable of determining the pressure altitude Z.sub.P of the aircraft 1 relative to a reference level based on the measurement of the static pressure P.sub.S. To this end, the altimeter 6 uses for example a standard atmosphere table that includes tabulated values of static pressure as a function of altitude.”)
One having ordinary skill in the art, before the effective filing date of the claimed invention, would have found it obvious to combine Robinson’s method with Dupont’s technique of including a pressure sensor for obtaining aircraft altitude data. Doing so would be obvious to provide “reliable alternative data and information” (See [0013] of Dupont).
Claim(s) 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Robinson in view of Brinkman and further in view of Flaherty-Woods of US 20190244429 A1, filed 02/07/2018, hereinafter “Flaherty”.
Regarding claim 22, Robinson and Brinkman in combination teach all the limitations of claim 7 as discussed above.
Robinson and Brinkman in combination do not explicitly teach:
wherein the formation flight information is displayed on a virtual map that extends at least 180 degrees around the pilot's field of view on the trailing aircraft augmented reality display.
Flaherty teaches:
wherein the formation flight information is displayed on a virtual map that extends at least 180 degrees around the pilot's field of view on the trailing aircraft augmented reality display. (See at least [0103]: “As the operator scans the airspace, virtual symbols are displayed by the head wearable device 140 360° about the aircraft, overlaid on top of real aircraft locations.”)
One having ordinary skill in the art, before the effective filing date of the claimed invention, would have found it obvious to combine Robinson and Brinkman’s method with Flaherty’s head wearable device displaying virtual signals 360 degrees about the aircraft. Doing so would be obvious since “This allows the operator to visualize the location of battlespace assets, even if the assets are beyond the operator's natural line of sight” (See [0103] of Flaherty).
Claim(s) 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Robinson in view of Brinkman and further in view of Servantie of US 20180315322 A1, filed 04/23/2018, hereinafter “Servantie”.
Regarding claim 24, Robinson and Brinkman in combination teach all the limitations of claim 7 as discussed above.
Robinson and Brinkman in combination do not explicitly teach:
further comprising: defining a bounding box sufficient to enclose the leading aircraft where the leading aircraft is presumed in a position that takes up the most space on the trailing aircraft display.
Servantie teaches:
further comprising: defining a bounding box sufficient to enclose the leading aircraft where the leading aircraft is presumed in a position that takes up the most space on the trailing aircraft display. (See at least Fig. 3 & [0064-0073]: “In particular, during this step A), the acquisition module 12 acquires the actual position of the surrounding aircraft. As previously mentioned, this actual position is, for example, made up of three coordinates. Then, the acquisition module 12 sends this acquired position to the processing module 14. During the following step B), the processing module 14 delimits part of the airspace around the acquired actual position, and in particular around the surrounding aircraft, by a current three-dimensional contour…During the following step C), the processing module 14 commands the visualization module 16 to visualize the current three-dimensional contour in the compliant display zone 22 of the reference aircraft.”)
One having ordinary skill in the art, before the effective filing date of the claimed invention, would have found it obvious to combine Robinson and Brinkman’s method with Servantie’s technique of defining a bounding box sufficient to enclose the leading aircraft where the leading aircraft is presumed in a position that takes up the most space on the trailing aircraft display. Doing so would be obvious since it “provides a visualization of a state of traffic around a reference aircraft allowing the pilot to best perceive the evolution dynamics of the traffic” (See [0109] of Servantie).
Allowable Subject Matter
Claims 1-6 and 19-21 allowed.
The following is a statement of reasons for the indication of allowable subject matter:
The closest prior art of record is Byxbe (U.S. Pub. No. 2021/0350712 A1), hereinafter “Byxbe”, Robinson et al. (U.S. Pub. No. 2021/0049925 A1), hereinafter “Robinson”, and Dumas et al. (U.S. Pub. No. 2019/0094538 A1), hereinafter “Dumas”.
With respect to claim 1, Byxbe teaches a formation monitoring system that receives positioning signals from follower aircraft in a formation and determines the relative distance between two aircraft (See at least [0041] & [0055]). Byxbe additionally teaches an aircraft receiving a command from a user to move to another location within the formation (See at least [0045]), and commanding the aircraft to move to another location by sending a change-position signal to the aircraft (See at least [0056]). Robinson teaches a head-mounted augmented and virtual reality display device worn by multiple pilots of separate aircraft that displays common computer generated content from the unique perspective of each pilot based on their respective locations and aircraft’s attitude (See at least [0054] & [0063-0067]). Dumas teaches an image sensor mounted on a helmet worn by a pilot that acquires environmental images along a path traveled by an aircraft, where the helmet is part of a head-up system that projects a synthetic image comprising information for assisting the pilot of the aircraft in order to be visually superimposed on the pilot’s visual field of navigation (See at least [0047] & [0058]). However, neither Byxbe nor Robinson nor Dumas teach detecting and tracking other aircraft using images collected from the image sensor mounted on the helmet, determining a relative position in formation of each formation aircraft based on the collected images, or displaying procedural guidance on augmented reality displays in each formation aircraft in the subset of formation aircraft based on the pilot’s vantage point from the collected images.
Accordingly, none of the references, taken either alone, or in combination teach “…detecting and tracking other aircraft using the collected imagery…”, “…determining a relative position in formation of each formation aircraft based on the collected position data and imagery…” and “displaying, based on the pilot's vantage point from the collected imagery, procedural guidance to the pilot on a display screen of the operatively coupled augmented reality display device in each formation aircraft in the subset using the one or more computing devices” - in combination with the remaining elements and features of the claimed invention. It is for these reasons that the Applicant’s invention defines over the prior art of record.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NIKKI MARIE M MOLINA whose telephone number is (571)272-5180. The examiner can normally be reached M-F, 9am-6pm PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aniss Chad can be reached at 571-270-3832. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NIKKI MARIE M MOLINA/Examiner, Art Unit 3662
/ANISS CHAD/Supervisory Patent Examiner, Art Unit 3662
Prosecution Insights