DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s amendment filed on January 2nd, 2026 has been fully considered and entered.
In view of applicant’s amendments:
The objections to claims 5 and 13 are withdrawn.
The rejections to claims 4, 5 and 14 under 35 USC 112(b) are withdrawn.
The rejections to 4 and 12 are moot.
Response to Arguments
Applicant states: In other words, light from outside of the aircraft is captured by the optical couplers, the light is transmitted through optical waveguides (e.g., one or more fiber bundles or cables), then the light is processed by the processing unit to reconstruct the digital live image and by the optical beam expander to project the visual live image, and the digital live image and the visual live image, either simultaneously or in sequence, are displayed on the shared screen. None of the cited references teach or suggest this orientation of components or the method associated with said light processing and image displaying.
The examiner respectfully disagrees. Claim 1 recites a signal path. The light goes into a coupler, waveguide (fiber bundle in the amended claim), both a processing unit that displays a digital live image and a beam expander the projects the optical image, and out of a single display presenting both. This path is disclosed in Keurbitz, Figure 4. Scene radiation entering window 19 is focused by lens 33 onto the input image guide 35 (bundle fiber) at first intermediate image plane 47. The aircraft context, fuselage-mounted couplers and fiber routing, reconstruction of a single image from the light of several fibers, and the presentation of live camera video on the display are taught by Teo, Boucourt, Neutzler, and Minor respectively, as set out in the rejection.
Nonobviousness cannot be shown by pointing to the absence of the complete claimed arrangement in any single reference where the rejection rests on a combination.
Applicant states: In fact, the Office Action notes that Teo, Boucourt, and Neutzler do not teach or suggest the selective displaying on a shared screen of the digital live image and the visual live image. See Office Action at pg. 10, noting that the Office Action relies on Keurbitz and Minor to teach or suggest such a feature. However, Keurbitz and Minor fail to teach or suggest the structural orientation as set forth in claim 1.
The examiner respectfully disagrees. Keurbitz and Minor are not relied upon for the entire “structural orientation” of claim 1, they are relied upon for the hybrid display and split of the fiber carried light into digital and optical channels (Keurbitz, Figure 4, the light is directed to monitor 63, and CCD 61). This makes the claimed structure obvious to a skilled artisan, the question is not whether the combined teaching is directly disclosed, it is whether the combined teachings would have suggested the claimed device based on their teachings.
Applicant states: In particular, Keurbitz is directed to a periscope for viewing an outside environment of a vehicle. It is first noted that it would not be obvious to combine the Keurbitz reference to achieve the desired goal of the claims herein. Since a periscope must necessarily project through a structure of the vehicle (e.g., extend from an inner compartment to an outer environment), Keurbitz is designed to view the outside environment by weakening the structure of the vehicle with an aperture (e.g., a hole or aperture for the periscope to extend through), contrary to the teachings of the present Application. See Pub. App. at 46. Therefore, it would not be reasonable to combine Keurbitz with the remaining references since Keurbitz teaches away from claim 1.
The examiner respectfully disagrees. The prior art does not teach away from the claimed invention, as claim 1 itself necessarily requires penetration of the fuselage. The claim recites “optical couplers distributed across an outer surface of the fuselage,” but Figure 1 of the instant application’s disclosure makes clear that the fuselage must be penetrated. The couplers are described as “small holes or recesses” in the specification (paragraph 31). In both the prior art and in the instant application, the optical couplers and waveguiding components extend through the vehicle structure. The distinction described here is not present in claim 1, which is likely the aperture, diameter, and/or depth of the penetrating component.
Additionally, the rejection does not propose installing the periscope of Keurbitz in the aircraft. The aircraft is supplied by Teo, Keurbitz teaches the beam splitting practice which allows for both a digitized and pure analog optical signal to reach the display.
Keurbitz additionally supplies the core rationale: providing a purely optical channel has the advantage that “observation of the environment over the purely optical channel is still possible if electronics and/or the power supply fails.” (Paragraph 0007 of Keurbitz).
Applicant states: According to Keurbitz, a generating element 70 overlays a graphic onto a monitor 63, where a monitor image is also projected via a collimator optics 71 onto the monitor 63. See Keurbitz at FIG. 5 and 35-37. In other words, although two images are overlayed on the same screen, the graphic overlayed by the generating element 70 is not a view of the outside environment. There is no discussion clarifying what the generating element 70 creates apart from the general reference to a graphic, which can include anything from a predetermined still picture to telemetry data. To that extent, there is also no discussion on projecting a visual live image transmitted through an optical waveguide comprising at least one fiber bundle or cable from an optical beam, at least because the generating element 70 appears to generate a graphic overlay separate from the live image displayed.
The examiner respectfully disagrees. While the display of Keurbitz indeed is not explicitly for the live outside environment, Keurbitz is in fact relied upon for
a) fiber bundle transmission of the capture light (image guide 35)
b) the beam splitter 57 at the fiber output that supplies part of the fiber carried light to CCD 61 (digitization of the optical signal)
c) monitor 63 whose image is then combined with the direct optical image at the same beam splitter so that a viewer may see both from a single location.
Minor and Neutzler are what motivate an image capture by CCD 61 from the fiber output. Minor teaches that a display can be used to show video from a camera or FLIR, and that a combined digital and optical view can show sensor imagery with an outside view (Column 3, section 3. Objects and Advantages). Neutzler teaches a processor 140 that receives light from fiber bundles 525 and reconstructs a single digital image of the scene for a display 150. One of ordinary skill would have found it obvious to route the output of CCD 61 to a processing unit (Neutzler’s processor 140) and then to monitor 63 of Minor.
The structure of the combined invention does not need to explicitly recite the claimed function, it is naturally poised to perform the claimed function.
Applicant states: Minor fails to supply the missing features of Keurbitz. Notably, Minor is also a periscope which suffers the same drawbacks as Keurbitz in that the periscope must necessarily extend through the structure of the vehicle. It is also noted that it would not be obvious to combine Minor with the remaining references at least due to this significant teaching away as explained above.
The examiner respectfully disagrees. As explained above, there is no teaching away in the prior art which supports the combined invention of claim 1. Claim 1 requires couplers on the outer fuselage, so a through-structure optical path is not a point of distinction. Minor’s prisms 110/120 and housing 130 are not imported by the proposed modification; Minor merely teaches the architecture display (display, outside view, combined view, selectable live digital or optical view). Minor, Column 3, object (k) confirms the alignment in motivation: “…failure modes return the device to the optical view of the outside,” just as electronic failure reverts to the live optical view as claimed.
Applicant states: Minor also fails to teach or suggest the system of claim 1. Although Minor discusses an intermediate mode in which a display view and an outside view are presented to a user (e.g., a crewmember's eye 160), the display view and the outside view are different from the digital live image and visual live image of claim 1. See Minor at col. 6, lns. 54-59. In particular, the outside view of Minor is merely a reflection of light off a series of mirrors to reach the user indicative of a real-time, outside environment, and the display view is a simulated scene that does not represent a real-time, outside environment or displays an artificial scene (e.g., FLIR camera). Id. at col. 3, lns 5-9 and col. 14, lns 28-32.
The examiner respectfully disagrees. Minor’s objects in Column 3 states that the display view “can be used to show a simulated scene for training purposes” (object b), “can be used to show vehicle data” (object c), and “can be used to show video, e.g. from a daylight camera or FLIR” (object d). Both the camera and the FLIR imagery are live sensor images of the outside scene, not an artificial scene. There is no limitation to simulated content, simulated content is just one of many possibilities.
Minor is not relied upon for the means by which outside light is conveyed to the display. Boucourt, Neutzler, and Keurbitz are relied upon for fiber-bundle conveyance. Minor is relied upon for the teaching that, at the point where a direct optical view of the exterior and electronic display meet, the display is used to present live camera video of that same exterior, either alone or overlaid on the direct view, as selected by a user.
Applicant states: Further still, the combined view (which the Office Action alleges is equivalent to claim 1, a point which the Applicant does not agree), cannot show an image from a camera of the outside environment simultaneously with a reflected live view from the mirrors. Id. at col. 3, lns. 10-18 and col. 14, lns. 33-35, noting that the combined view is not configurable to show a live view from a camera of the outside environment in addition to projecting the live view reflected from the mirrors).
The examiner respectfully disagrees, Minors objects in column 3 conclude that the “combined view can be used to simultaneously show sensor imagery in, for example, low light, obscuring weather, or obscuring smoke viewing conditions” (object g). “Vehicle data” and “target indicators” are provided as examples of combined-mode content.
Claim 1 additionally recites that the selectively displaying is performed “either simultaneously or in sequence.” Minor’s switching between the outside view mode and the display view mode presents the direct optical image and the display image in sequence on the same viewing path, satisfying the claim.
Applicant States: Even if the above is not persuadable, the live view of Minor is based off of a reflection of the outside environment on a series of mirrors. Id. at FIG. 2 and supra. The system of Minor is purely analog using known mirror reflection methods. The live view of Minor is not based on transmitting received light through one or more fibers or cables in order to project the light onto a shared display, nor is there any discussion on reconstructing an image by a processing unit based on the light received.
The examiner respectfully disagrees.
Minor is not relied upon for fiber transmission or image reconstruction by a processing unit.
Boucourt teaches fibers Fe/Fr through the fuselage with processing unit 15;
Neutzler teaches fiber bundles 525, processor 140 reconstruction an image from the light of the bundles, and display 150;
Keurbitz teaches image guide 35, CCD 61 receiving the fiber output.
Minor also is not a purely analog teaching, the display view is fed by a projector 260 or flat panel display 1210 displaying digital imagery.
The rejection of claim 1 over Teo in view of Boucourt, Neutzler, Keurbitz and Minor is maintained as applied to the amended claim. The rejection of claim 11 over Teo, Boucourt, Minor and Keurbitz is maintained as applied to the amended claim.
Claim Objections
Claims 15 and 16 are objected to for the following informalities:
Claim 15 recites “The automatic adjusting of the digital live image” in line 1. Per the remarks, claim 5 was amended to recite that the hybrid display “automatically updates” the digital live image. There is no antecedent basis for the “automatic adjusting” as claimed.
Claim 16 recites, “is configured display.” The phrase should read, “…is configured to display…” Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 4-10 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Teo et al. (US 20160325836 A1) in view of Boucourt et al. (US 10969492 B2), and further in view of Keurbitz et al. (EP1467237B1), Neutzler et al. (US 8836848 B2), and Minor et al. (US 8264770 B2).
Regarding claim 1:
Teo et al. discloses a system for displaying a view of an external environment of an aircraft inside the aircraft (Abstract, “A system and method for displaying on a display inside of an aircraft a panoramic view of a view outside the aircraft”), the system comprising:
a plurality of cameras (Figure 1, cameras 20) distributed across an outer surface of a fuselage of the aircraft and configured to capture light from the external environment of the aircraft;
at least one display (Figure 4a, display 30) arranged on the inside of the fuselage;
wiring (wires 5) arranged through the fuselage to transmit the signal of the captured light through the fuselage (Figure 2), and at least;
a processing unit (processor 40) configured to receive the signal of the captured light from the optical waveguides (Figure 2 depicts this), to reconstruct a digital live image of the external environment (display 30 displays the digital live image) by combining electrical information contained in the electrical signals captured across the respective cameras and to display the digital live image on the at least one display (Figure 2)
Teo et al. does not disclose:
optical waveguides arranged through the fuselage to transmit the captured light from the optical couplers through the fuselage, the optical waveguides comprising at least one fiber bundle or cable;
that the processing units are for optical waveguides;
an optical beam expander configured to receive the captured light into a light beam carrying a visual live image of the external environment, and to project the visual live image on the at least one display.
Boucourt et al. discloses a system for aiding aircraft with collision avoidance and taxiing via the use of optical imaging and detection systems (Abstract), wherein:
optical waveguides (Figure 2a and 2b, Fibers ‘Fe’ and ‘Fr’) are arranged throughout the fuselage and wing (Description of the figures, ln 33-40 teach that the transmissions may be carried out in both the fuselage and wing) to transmit captured light from the optical couplers (convergent lenses L1)
processing units (15) upstream of a display system (17)
Neutzler et al. discloses the use of a waveguide-based image system (Figure 5a), in which:
a processing unit (Figure 1, processor 140) is configured to receive the captured light from the optical waveguides (optical fiber bundles 525), to reconstruct a digital live image of the external environment by combining optical information contained in the captured light across the respective optical couplers (lens array 115) and to display the digital live image on the at least one display (Figure 1, display 150; Field of the Invention and Background, paragraph 1, disclose this).
Keurbitz et al. discloses a periscope (Figure 1) with a purely optical channel used to direct an outside view to a display (Figure 4 shows that optical fiber bundles 35 convey the light to a monitor 63, ccd 61, and optical channel 65), further teaching:
An optical beam expander (the arrangement of lenses 53 and 59 in Figures 4 and 5 is exactly what is known in the art as a “Keplerian beam expander”, simply used in reverse for a smaller exit display) configured to receive the captured light from at least one optical waveguide, to expand the captured light into a beam carrying a visual live image of the external environment (Figure 4, optical unit 53 is depicted to be a lens which expands the light for the optical view 59), and to project the visual live image on at least one display (Figure 4, the light is directed to monitor 63, and also CCD 61).
Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to modify the invention of Teo et al. as follows:
The cameras are swapped with optical couplers by utilizing lenses as the optical endpoints, optical fibers to transmit captured light, and an optical signal processor distributed through the fuselage of the aircraft, as in Boucourt et al.; this swaps the electrical structure for an optical structure.
The wiring is replaced with optical waveguides which lead to an optical processing unit that is upstream of a display, as taught in Boucourt et al., wherein either:
the optical waveguides themselves, the processing unit, and the display are constructed in view of Neutzler et al. to process optical input from many viewpoints into a reconstructed image. This allows optical waveguides to be configured such that they convey visual information about a scene to a display, and not just general information about proximity for pilots; or
an optical beam expander is configured to receive the captured light from fiber bundles and then project it onto a display to construct the visual live image, as constructed in Keurbitz et al. Beam expansion and collimation is routine practice in the art, and it would be obvious to one of ordinary skill in the art to alter the ratio of lens sizes to meet any use case.
These changes may be accomplished using methods known to the art, and are motivated by existing measures to create sensors and display-based external view systems in aircraft (Teo et al.) using waveguides and fiber optics (Boucourt et al.), where the display system is motivated by optical displays as taught in Neutzler et al. and Keurbitz et al. This would predictably result in an external view system which relies on direct optical input rather than electrical components to read a scene, which comes with weight reduction and reduced aerodynamic drag (optical couplers can be recessed and have small aperture, optical routing systems are not as heavy as windows and many electrical components), low latency optical paths, benefits to the structure of the plane in absence of the space used by conventional viewing systems.
Regarding claim 2:
Teo et al. in view of Boucourt et al., Neutzler et al., and Keurbitz et al. discloses the system according to claim 1, wherein:
Neutzler et al. further teaches that at least one display comprises a digital display for displaying the digital live image provided by the processing unit (this is exactly demonstrated by the heuristics of Figure 7 – the processor 725 is upstream of the displayed image 730), or
Keurbitz et al. further teaches that a display unit with a screen (Figure 5, monitor 63 is interpreted to have a screen) projects the visual live image provided by the optical beam expander (Figure 5, the optical view 65 need merely be swapped with the monitor 63, or the expander lenses 53 and 59 may be added in front of the monitor 63).
Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to utilize the teachings of Neutzler et al. to further modify the system described in the rejection of claim 1 above, by ensuring that the processor is upstream of the display, or that a beam expander projects the image onto the display. This may be accomplished using methods known to the art (routine placement of system components), and would predictably result in a display which displays an image that is controlled by a processing unit, or which is directly coupled to the input light, which offers flexibility in modification of the image depending on use case as well as its ability to be displayed successfully.
Regarding claim 4:
Teo et al. in view of Boucourt et al., Neutzler et al., Keurbitz et al., and Minor et al. discloses the system according to claim 1, wherein:
The device of claim 1 includes the protective faceplate 150 of Minor et al., who further establishes the use of a non-opaque and non-transparent screen (In an embodiment, Figure 11, Electronically Switchable Shade [ESS] 1110 is a screen for the apparatus).
Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to modify the device described in the rejection of claim 1 above to include a semi-transparent screen under the teachings of Minor et al. This could be accomplished by using ESS 1110 to modify the screen in the device suggested by the teachings of claim 1 and would predictably result in a device which is adjustable in its transparency during display.
Regarding claim 5:
-Teo et al. in view of Boucourt et al., Neutzler et al., Keurbitz et al., and Minor et al. discloses the system according to claim 1, wherein:
Minor et al. discloses that the hybrid display is configured, via the processing unit to automatically adjust the visual live image provided by the optical beam (Summary discloses that application of electrical potential on ESS 1110 determines the degree to which the digital image is overlayed on the visual image).
Minor et al. does not expressly disclose that the hybrid display is configured to adjust the digital live image to match the visual. However, the means of adjustment is disclosed clearly by minor et al. – a layer of material sensitive to electric potentials changes in opacity in response to a change in potential. This material could be used anywhere along the optical path of the digital image to match the visual, as it is already disclosed to be used to alter the visual image instead of the digital (Figure 11 of Minor et al. shows this well – the ESS 1110 is upstream of the visual image, but could just as easily be configured in front of the digital source).
Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to modify the display of the system described in the rejection of claim 1 above to include the ESS 1110 of Minor et al., using the component as it is known to the art. This would predictably result in a display where the digital live image is dynamically adjustable such that it may match the visual image, as one would simply have to alter the electric potential to reach the desired visual parameters that define a ‘match’.
Regarding claim 6:
Teo et al. in view of Boucourt et al., Neutzler et al., and Keurbitz et al. discloses the system according to claim 1, wherein:
Teo at al. teaches that the at least one display comprises several displays arranged on an inner wall of the fuselage and shaped to represent aircraft windows (Figure 1 shows how the display 30 comprises several displays 30a-30c, Figure 3 illustrates how they are arranged on an inner wall of the fuselage and shaped to represent aircraft windows).
Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to further modify the system described in the rejection of claim 1 above to arrange the displays on an inner wall of the fuselage and shape them to represent aircraft windows, in view of the teachings of Teo et al. This could be accomplished by using the same configuration for displays as taught in Teo et al., in the device suggested by the teachings of claim 1, and would predictably impart the benefit of being suited for use in commercial aircraft while providing structural, weight, and cost related benefits to the airline.
Regarding claim 7:
Teo et al. in view of Boucourt et al., Neutzler et al., and Keurbitz et al. discloses the system according to claim 1, wherein:
Boucourt et al. teaches that the optical couplers are configured as fiber collimators; Figure 2a shows that the couplers are convergent lenses L1, which are known to the art as collimators for fibers (paragraph 37 of Keurbitz et al. explicitly teaches the use of collimators 71, but they are downstream of the coupler).
Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to further modify the optical couplers in the system described in the rejection of claim 1 above to be fiber collimators, which comprise a first lens that collimates light into a fiber. This replacement could be accomplished using materials and placement techniques known to the art, and would predictably result in optical couplers with a low profile with respect to the fuselage and high signal integrity with respect to the outside scene.
Regarding claim 8:
Teo et al. in view of Boucourt et al., Neutzler et al., and Keurbitz et al. discloses the system according to claim 1, wherein:
Boucourt et al. discloses that the convergent lenses are arranged recessed in the outer surface of the fuselage. (they are depicted in Figures 1, 2a, and 2b as being ‘inset’ within outer surface of the fuselage)
Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to modify the system described in the rejection of claim 1 above to reduce the external profile of the optical couplers by embedding them into the fuselage, using routing judgement when deciding how deeply to set the couplers. This would predictably have the benefit of reducing drag and aerodynamic disturbance, without modifying the function of the claimed invention.
Regarding claim 9:
Teo et al. in view of Boucourt et al., Neutzler et al., and Keurbitz et al. discloses the system according to claim 1, wherein:
Boucourt et al. further teaches that the processing unit is configured to employ one of:
Sensor fusion or computational imaging (Col. 5, ln. 61 – Col. 5, ln. 2 shows sensor fusion via the combination of laser telemeter 11 and camera 18; computational imaging is implied) to reconstruct the digital live image (Claim 3, “…the illumination of the field carried out by the laser telemetry is correlated with a visualization of the field of view…”)
Boucourt et al. do not teach compressed sensing.
Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to modify the system described in the rejection of claim 1 above under the teachings of Boucourt et al. to employ sensor fusion or computational imaging techniques to reconstruct a digital live image (‘visualization of the field of view’) using the methods and components taught therein, to combine sensor inputs or perform computational imaging at the processing stage. This would predictably result in a system that can reconstruct a digital live image of an external environment.
Regarding claim 10:
Teo et al. teaches an aircraft (Claim 8 teaches a system comprising an aircraft).
Teo et al. in view of Boucourt et al., Neutzler et al., and Keurbitz et al. discloses the system according to claim 1.
Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to modify the aircraft described in the rejection of claim 1 above to further comprise the system as taught by Teo et al., Boucourt et al., Neutzler et al., and Keurbitz et al. This may be accomplished using vehicles (aircraft), components, and methods known in the art, and would predictably result in an aircraft capable of supporting the display system of claim 1, permitting industrial use in avionics.
Regarding claim 16:
Teo, in view of Boucourt, Keurbitz, Neutzler and Minor discloses the system according to claim 1.
Teo does not explicitly state that the display maintains an image during power outage.
Keurbitz teaches that the hybrid display is configured to display the visual live image provided by the optical beam expander during a power outage (the purely optical channel is provided via Image guide 35, optical unit 53, and eyepiece 59). Crucially, this has the stated advantage that “observation of the environment over the purely optical channel is still possible if the electronics and/or the power supply fails,” (paragraph 0007).
Thus, a skilled artisan would find it obvious to ensure that the hybrid display is configured to display the visual live image provided by the optical beam expander during a power outage scenario.
Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to modify the invention described in the rejection of claim 1 above under the teachings of Keurbitz to ensure that the hybrid display is configured such that a visual live image is maintained during power outage. This requires no active components, merely the passive optical structure of the invention in claim 1, to be maintained. This would be an obvious consideration for a skilled artisan constructing displays for aircraft where external views are both commonplace and essential. This may be accomplished using methods, design oversight, and materials known in the art, and would predictably result in a system where device function is maintained even when electronics and power supplies are rendered inert.
Claim(s) 11, 13 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Teo et al. (US 20160325836 A1) in view of Boucourt et al. (US 10969492 B2), and further in view of Minor et al. (US 8264770 B2) and Keurbitz et al. (EP1467237B1).
Regarding claim 11:
Teo et al. discloses a method for displaying a view of an external environment of an aircraft inside the aircraft (Abstract, “A system and method for displaying on a display inside of an aircraft a panoramic view of a view outside the aircraft”),
Teo et al. further discloses that the displays are arranged on the inside of the fuselage (Figure 3).
Boucourt et al. discloses a system for aiding aircraft with collision avoidance and taxing via the use of optical imaging and detection systems (Abstract), wherein the system involves:
Capturing light from the external environment of the aircraft with a plurality of optical couplers (converging lenses L1) distributed across an outer surface of a fuselage of the aircraft (Description of the figures, ln 33-40 teach that the transmissions may be carried out in both the fuselage and wing);
Transmitting the captured light from the optical couplers through the fuselage with optical waveguides arranged through the fuselage, the optical waveguides comprising at least one fiber bundle or cable (Figure 2a, the couplers are upstream of bundled fibers Fe and Fr);
Reconstructing a digital live image of the external environment with a processing unit receiving the captured light from the optical waveguides (Claim 3, “…the illumination of the field carried out by the laser telemetry is correlated with a visualization of the field of view…”) by combining optical information contained in the captured light across the respective optical couplers (Col. 5, ln. 61 – Col. 5, ln. 2 shows sensor fusion via the combination of laser telemeter 11 and camera 18; computational imaging is implied) and displaying the digital live image on at least one display arranged inside the fuselage, wherein the at least one display comprises a hybrid display with a shared screen (Figure 2a, display system 17); and
Boucourt et al. does not teach:
expanding the captured light into a light beam carrying a visual live image of the external environment with an optical beam expander receiving the captured light from at least one of the optical waveguides and projecting the visual live image on the at least one display.
Selective displaying of a shared image as claimed.
Minor et al. teach a hybrid periscope system display (Title, “Multi-purpose periscope with display and overlay capabilities”) designed to switch between a digitally generated live image and a visual live image, or display both (Abstract teaches a periscope that can switch between or display both of an outside visual view, or display view, wherein the display view may be a digitized view of the outside – see Col. 3, Ln. 3-15).
The display (as viewed by an individual 160, through faceplate 150) comprises a hybrid display (switches between digital and live image via electrically switchable mirror 240) with a shared screen (150), on which the digital live image provided by the projector 260 and the visual image provided by the optical beam 180 are selectively displayed.
Minor et al. does not expressly teach that the projector is or is connected to a processing unit; or that an optical beam expander is downstream of the visual live image; however, they do teach projection lens 250, which expands the optical beam and is therefore an optical beam expander.
Keurbitz et al. discloses a periscope (Figure 1) with a purely optical channel used to direct an outside view to a display (Figure 4 shows that optical fiber bundles 35 convey the light to a monitor 63, ccd 61, and optical channel 65), further teaching:
An optical beam expander (the arrangement of lenses 53 and 59 in Figures 4 and 5 is exactly what is known in the art as a “Keplerian beam expander”, simply used in reverse for a smaller exit display) configured to receive the captured light from at least one optical waveguide, to expand the captured light into a beam carrying a visual live image of the external environment (Figure 4, optical unit 53 is depicted to be a lens which expands the light for the optical view 59), and to project the visual live image on at least one display (Figure 4, the light is directed to monitor 63, and also CCD 61).
This constitutes “expanding the captured light into a light beam carrying a visual live image of the external environment with an optical beam expander receiving the captured light from at least one of the optical waveguides and projecting the visual live image on the at least one display; and selectively displaying, via the hybrid display, either simultaneously or in sequence, on a shared screen the digital live image provided by the processing unit and the visual live image provided by the optical beam expander.”
Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to modify the method suggested by the rejection of claim 1 above, in view of the teachings of Boucourt et al. and Keurbitz, to include light capture from optical couplers and optical signal routing instead of electrical, while reconstructing a digital live image for display onto the displays of Teo et al, and to include a hybrid display system under the teachings of Minor et al. and Keurbitz et al. This could be accomplished by including a hybrid/switch system upstream of the display, such that an electrically switchable mirror (as in Minor et al.) is located at a point where the digital and visual live images meet prior to the display, and then to further modify it such that an optical beam expander (as in Keurbitz et al.) is downstream of the optical signal but prior to the display. This would allow a user to switch between the digital and visual live images at will, while ensuring that the visual live image amalgamated from the optical couplers is correctly scaled for viewing at the display.
Regarding claim 13:
-Teo et al. in view of Boucourt et al., and in further view of Minor et al. and Keurbitz et al. disclose the method according to claim 11, wherein:
Minor et al. discloses that the hybrid display is configured to dynamically adjust the visual live image provided by the optical beam (Summary discloses that application of electrical potential on ESS 1110 determines the degree to which the digital image is overlayed on the visual image).
Minor et al. does not expressly disclose that the hybrid display is configured to adjust the digital live image to match the visual. However, the means of adjustment is disclosed clearly by Minor et al. – a layer of material sensitive to electric potentials changes in opacity in response to a change in potential. This material could be used anywhere along the optical path of the digital image to match the visual, as it is already disclosed to be used to alter the visual image instead of the digital (Figure 11 of Minor et al. shows this well – the ESS 1110 is upstream of the visual image, but could just as easily be configured in front of the digital source).
Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to modify the method described in the rejection of claim 11 above to include the ESS 1110 of Minor et al., using the component as it is known to the art in the optical path of the digital live image, prior to the display. This would predictably result in a display where the digital live image is dynamically adjustable such that it may match the visual image, as one would simply have to alter the electric potential to reach the desired visual parameters that define a ‘match’.
Regarding claim 14:
-Teo et al. in view of Boucourt et al., and in further view of Minor et al. and Keurbitz et al. disclose the method according to claim 11, wherein:
Boucourt et al. further teaches that the processing unit is configured to employ one of:
Sensor fusion or computational imaging (Col. 5, ln. 61 – Col. 5, ln. 2 shows sensor fusion via the combination of laser telemeter 11 and camera 18; computational imaging is implied) to reconstruct the digital live image (Claim 3, “…the illumination of the field carried out by the laser telemetry is correlated with a visualization of the field of view…”)
Boucourt et al. do not teach the claimed alternative involving compressed sensing.
Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to further modify the method described in the rejection of claim 11 above under the teachings of Boucourt et al., to employ sensor fusion or computational imaging techniques to reconstruct a digital live image (‘visualization of the field of view’) using the methods and components taught therein, to combine sensor inputs or perform computational imaging at the processing stage. This would predictably result in a system that can reconstruct a digital live image of an external environment.
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Teo et al. (US 20160325836 A1) in view of Boucourt et al. (US 10969492 B2), and further in view of Keurbitz et al. (EP1467237B1), Neutzler et al. (US 8836848 B2), Minor et al. (US 8264770 B2), and Spencer (US 20200369361 A1).
Teo, in view of Boucourt, Keurbitz, Neutzler, and Minor discloses the system according to claim 5, wherein: the automatic adjusting of the digital live image provided by the processing unit to match the visual live image provided by the optical beam expander includes at least one of:
adjusting a contrast of the digital live image, adjusting a resolution of the digital live image, adjusting a brightness of the digital live image, or any combination thereof.
Keurbitz teaches the updating of the electronic image to match the optical scene image including adjustment of a brightness and a contract of the electronically generated image. Where the monitor 63 image and the scene image are superimposed by beam splitter 27, “the brightness/contrast setting of the monitor 63” with the variable aperture 73 is used to set “the ratio of the brightness of monitor graphic to scene” (Figure 5).
Minor et al. teaches equalizing the two components of the combined view: “In combined mode, one layer of upper ESM 270 can be made transparent to reduce the brightness of the outside portion of the view to equalize the brightness of these two components of the combined view,” and that the ESM state is set by the applied electrical potential (Figures 7A-7C, Figure 8).
Neither reference teaches automatic brightness adjustment.
Spencer teaches a transparent display member 120 placed in an aircraft cabin window stack such that displayed content and the outside view are presented through the same panel, with a controllable circuit 22 and a pair of light-receiving sensors 26, 28. The circuit 22 will “calculate a different in the amount of light between the outside and inside of the cabin and adjust either the darkness or color of pixels… or the amount of light transmitted through a background,” and such changes “can be automated to maximize visibility” (Figures 3, 5).
Before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to modify the invention described in the rejection of claim 5 above under the teachings of Keurbitz, Minor, and Spencer to include automated brightness and/or contrast adjustment in the hybrid display system. This could be accomplished by utilizing light sensors and a control system in the manner of Spencer to control the brightness/contrast settings of the display, using routine design oversight known to a skilled artisan. Predictably, this would result in a device that maintains a displayed live image that maximizes visibility by adjusting brightness and contrast in response to internal and external information.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PREET B PATEL whose telephone number is (571)272-2579. The examiner can normally be reached Mon-Thu: 8:30 am - 6:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THOMAS A HOLLWEG can be reached at 571-270-1739. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PREET B PATEL/Examiner, Art Unit 2874
/THOMAS A HOLLWEG/Supervisory Patent Examiner, Art Unit 2874
Prosecution Insights