Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
METHOD OF GENERATING IMAGES REPRESENTABLE BY A VISOR WEARABLE BY A PILOT OF AN AIRCRAFT
Document Type and Number:
WIPO Patent Application WO/2022/078907
Kind Code:
A1
Abstract:
It is an object of the invention to provide a method for generating images representable by a visor (20) wearable by a pilot of an aircraft (10), wherein said method comprises at least the following steps: - real time recording of at least the aircraft's roll, pitch and yaw data (10); - at least the roll, pitch and yaw data of the visor (20) worn in real time; - real time detection of at least the geographical position of the aircraft (10); - using the roll, pitch and yaw data from the aircraft (10) and the worn visor (20) to generate images viewable by said visor (20), wherein said images represent the environment outside said aircraft (10) at said geographic location.

Inventors:
VINATI FELICE (IT)
VINATI SAMUELE (IT)
VINATI MATTEO (IT)
VINATI GIACOMO (IT)
VINATI MARIACHIARA (IT)
Application Number:
PCT/EP2021/077903
Publication Date:
April 21, 2022
Filing Date:
October 08, 2021
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
IEROM S R L (IT)
International Classes:
G01C23/00
Foreign References:
US5072218A1991-12-10
JPH06265627A1994-09-22
US20130120164A12013-05-16
US20180075762A12018-03-15
US20160189426A12016-06-30
EP0911647A21999-04-28
US7266446B12007-09-04
Attorney, Agent or Firm:
PARISI, Luigi et al. (IT)
Download PDF:
Claims:
CLAIMS

1. A method of generating images that can be represented by a visor (20) that can be worn by an aircraft pilot (10) , wherein said method provides for the use of an electronic control unit (450) , equipped with a memory (460) , said electronic control unit (450) being configured to perform the phases of the method, wherein said method comprises at least the following phases :

- real time detection of roll, pitch, yaw and geographic orientation data of the aircraft (10) ;

- real time detection of roll, pitch, yaw and geographical orientation data of the worn visor (20) ;

- real time detection of the geographic position of the aircraft (10) ;

- use of the roll, pitch, yaw and geographic orientation of the visor (10) data as a function of the roll, pitch, yaw and geographic orientation of the aircraft (10) data to calculate the current spatial orientation of the visor (20) with respect to the outside environment the aircraft (10) , in order to allow said electronic control unit (450) to generate in real time images that can be viewed from the above visor (20) , wherein said images represent the outside environment with respect to said aircraft (10) in the current geographic position of the aircraft (10) and according to the current spatial orientation of the visor (20) ; and wherein said images are associated with visual and/or acoustic alarms in case said aircraft (10) is in the proximity of or on a collision course with obstacles present on the territory, and wherein the position of said obstacles is stored in the memory (460) associated with said electronic control unit (450) on the basis of pre-made maps .

2. A method according to claim 1, wherein the phase of real time detection of the roll, pitch and yaw data and of the geographic orientation data of the airplane (10) is performed with the aid of an inertial platform (30) placed on board of said airplane (10) .

3. A method according to claim 2, wherein a calibration phase of the roll, pitch and yaw angles of the airplane (10) with respect to a reference system integral to the external environment and a calibration phase of the orientation of the airplane (10) with respect to a known geographic direction is provided.

4. A method according to claim 1, wherein the real time detection phase of the roll, pitch and yaw data and of the geographic orientation data of the visor (20) worn is carried out with the aid of an inertial platform (40) placed on board the above visor (20) .

5. A method according to claim 4, wherein in which it is provided a phase of calibration of the spatial orientation of the visor (20) with respect to a reference system integral to the aircraft (10) , so that at the start of the phases of the method, the spatial orientations of the axes of yaw (Y' ) , roll (R' ) and pitch (P* ) of the visor (20) substantially coincide with the spatial orientations of the axes of yaw (Y) , roll (R) and pitch (P) of said aircraft (10) .

6. A method according to claim 5, wherein the calibration phase of the visor (20) spatial orientation with respect to a reference system integral to the aircraft (10) , is performed by a pilot of the aircraft by observing a fixed reference (70) placed on the cockpit of the aircraft (10) .

7. A method according to claim 5, wherein it is provided a phase of calculation of the differences between the spatial orientations of the yaw (Y' ) , roll (R* ) , pitch (P' ) axes of the visor (20) and the spatial orientations of the yaw (Y) , roll (R) , pitch (P) axes of said aircraft (10) , said calculation phase being finalized to determine instant by instant the spatial orientation of the visor (20) with respect to the external environment, so that said visor (20) displays images representing the external environment according to a current spatial orientation of the visor (20) .

8. A method according to claim 1, wherein the phase of real time detection of the geographic position of the aircraft (10) is carried out with the aid of a geolocation detector (50) placed on board of said aircraft (10) and wherein it is provided a phase of calibration of the geolocation detector (50) placed on board of said aircraft (10) with the coordinates of the geographic point where the aircraft (10) is located at the time of the above calibration.

9. A method according to claim 1, wherein the phases of detection of the roll, pitch, yaw and geographic orientation data of the airplane (10) and of the roll, pitch, yaw and geographic orientation data of the visor (20) worn and of the geographic position of the airplane (10) are performed simultaneously.

10. A method according to claim 1, wherein said electronic control unit (450) is placed on board of said aircraft (10) .

11. A method according to claim 1, wherein said images are associated with visual and/or acoustic alarms in case said aircraft (10) is in the proximity of or on a collision course with telephone and/or power line cables, or other obstacles, and wherein the position of said cables is stored in the memory (460) associated with said electronic control unit (450) based on maps issued by telephone and/or power companies .

12. A method according to claim 1, wherein the height of obstacles shown on said pre-made maps is determined on the basis of software analysis of satellite photos of the mapped area, wherein such photos have an indication of the time and day on which they have been taken, so that the height of the said obstacles is determined on the basis of the shadows generated by the said obstacles and detectable from said satellite photos and on the basis of the information of the time and day on which the said photos have been

15 taken from the satellite .

13. A method according to claim 1, wherein in case of an unexpected change of the external environment in the vicinity of a landing area (10) , said electronic control unit (450) is configured to receive a warning signal (95) or, as appropriate, a go-ahead signal coming from an external source connected by radio to said electronic control unit (450) in order to update at the same time the representation of the virtual environment corresponding to the area in which an unexpected change of landing conditions has occurred. 14. A method according to claim 13, wherein the transmission of data relating to the safety of the aircraft sent from external sources and received by said electronic control unit (450) is performed via a data link operating with 5G technology.

16

Description:
"METHOD OF GENERATING IMAGES REPRESENTABLE BY A VISOR WEARABLE BY A PILOT OF AN AIRCRAFT"

FIELD OF THE INVENTION

It is an object of the invention to provide a method for generating images representable by a visor wearable by a pilot of an aircraft . PRIOR ART

It is well known that one of the main problems of flying, both for aircrafts such as planes and helicopters as well as for other known aircrafts, is the poor or absent visibility that usually occurs in conjunction with adverse weather conditions .

Bad weather conditions such as fog or rain cause significant problems for flying, as does darkness in the night hours .

In particular, especially for aircrafts operating under the conditions established by the Visual Flight Rules (VFR) , such problems require aircraft to be kept on the ground.

In the aeronautical field, flight simulators are also well known, i . e . devices that use the computational power of current computers to recreate a virtual world closer to the real one than one can imagine .

The simulator and flight controls are then associated through actuators that measure exactly the movements of the flight controls, to the computer of the simulator that in turn synchronizes the movements of the flight controls with movements in the space of the simulator cabin that in turn are synchronized with the three- dimensional cartography that represents the virtual world in which the aircraft moves, in order to generate a feeling of maximal realism aimed at training new pilots or to maintain their professional condition.

In synthesis, and in a simplified manner, in the traditional simulators it is not the aircraft that moves (and this is obvious being the simulator hosted in special structures) , but it is the virtual world that moves around the fixed point represented by the seat of the pilot, and moves with speed and in the spatial directions latitude, longitude, height, yaw, pitch and roll, which vary with the variation of the movements of the flight controls .

Also known are glasses for three-dimensional (3D) visualization, wearable by the pilot in training, where said 3D glasses are generally equipped with an inertial platform applied to them, which transmits instantaneous orientation data in space of the pilot ' s head, modifying the image of the virtual world created by the simulator on the basis of such data.

Such 3D glasses are already being used today to replace large, expensive simulators with closed cabin and screens on which 3D cartography is displayed.

In general therefore the principle of operation of the simulators of flight is the virtual representation of the reality, which in the context of the simulation can be assimilated to a cube (the virtual world) with a point in the center (the aircraft) , where it is the same cube that is moved as long as the software of the simulator calculates the new values of position and the other parameters attributed to the aircraft from the simulation.

Of course, while flight simulators are extremely useful in pilot training, they certainly do not solve the problem of actual flying in bad weather and/or low visibility conditions .

An object of the present invention is to provide a means for the pilot to overcome adverse weather conditions or poor visibility on board any aircraft in particular an airplane, helicopter or other aircraft .

A further purpose of the present invention is to achieve the above results in a simple and economical manner.

BRIEF SUMMARY OF THE INVENTION The present invention achieves the above-described purposes by means of a method of generating images that can be represented by a visor that can be worn by an aircraft pilot, wherein said method provides for the use of an electronic control unit, equipped with a memory, said electronic control unit being configured to perform the phases of the method, wherein said method comprises at least the following phases :

- real time detection of roll, pitch, yaw and geographic orientation data of the aircraft;

- real time detection of roll, pitch, yaw and geographical orientation data of the worn visor;

- real time detection of the geographic position of the aircraft;

- use of the roll, pitch, yaw and geographic orientation of the visor data as a function of the roll, pitch, yaw and geographic orientation of the aircraft (10) data to calculate the current spatial orientation of the visor (20) with respect to the outside environment the aircraft, in order to allow said electronic control unit to generate in real time images that can be viewed from the above visor, wherein said images represent the outside environment with respect to said aircraft in the current geographic position of the aircraft and according to the current spatial orientation of the visor.

An advantage of this embodiment is that the pilot, when wearing the visor, sees a virtual environment displayed in the same way he would have seen the same environment in which he is flying, but without undesirable atmospheric effects (fog, darkness, rain, etc. . ) .

The invention is also capable of ensuring that there is no delay in data processing, no connection disturbance, and that the dynamic graphic representation is always correct .

According to an embodiment of the invention, said images are associated with visual and/or acoustic alarms in case said aircraft is in the proximity of or on a collision course with obstacles present on the territory, and wherein the position of said obstacles is stored in the memory associated with said electronic control unit on the basis of pre-made maps .

This realization increases flight safety especially at low altitudes .

Further features of the invention can be inferred from the dependent claims .

BRIEF DESCRIPTION OF THE FIGURES

Further features and advantages of the invention will become more apparent in the light of the detailed description which follows with the aid of the accompanying drawing plates in which:

- Figure 1 schematically illustrates the main components which enable the method of the invention to be implemented;

- Figure 2 illustrates a block diagram of the main steps of the method of the invention; and

- Figure 3 schematically illustrates a calibration step of the spatial orientation of a viewer used in the method of the invention.

DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS OF THE PRESENT INVENTION

The invention will now be described with initial reference to the block diagram in Figure 1 .

In the following of the present description, the expression inertial platform will be understood as meaning, as is it known, a device comprising a set of sensors allowing the determination of the orientation in space of a given object to which said inertial platform is applied.

In particular, the inertial platforms may each comprise, for example, a three-axis accelerometer and a three-axis gyroscope and a magnetometer, wherein all said components may be managed by an electronic control unit, provided with a microprocessor, and by an appropriate software .

In figure 1, an aircraft is first represented, globally indicated by the numerical reference 10,

In the example of figure 1, a helicopter is concerned (but the invention is applicable to any other types of aircraft such as an airplane or other) , wherein said aircraft 10 is also represented with its three axes of yaw Y, roll R and pitch P.

On board of the aircraft 10 there are one or more inertial platforms 30, which have the task of determining, instant by instant, the orientation in space of the aircraft 10 and its geographical orientation with respect to a fixed reference, for example North. In particular, the inertial platform 30 may comprise, for example, a three-axis accelerometer 32 and a three-axis gyroscope 34 as well as a magnetometer 36, wherein all said components may be managed by an electronic control unit 450, equipped with a microprocessor.

The electronic control unit 450 is configured to perform the various steps of the method of the invention and makes use of a memory 460 that contains software and operating data.

In particular, the electronic control unit 450 may be part of a computer or other electronic computing system and is located on board the aircraft 10, as is the memory 460 that contains the aeronautical simulation software .

In turn, the pilot C is provided with a three-dimensional type of visor 20, which can be constituted by a helmet or simply by 3D glasses; in any case, the visor 10 is provided with screens placed in correspondence of the eyes of the pilot C, said screens being suitable to represent a virtual environment .

For example, the visor 20 may be affixed to the helmet of the pilot C or, alternatively, may be affixed to a system that also includes a headset for radio communications .

The viewer 20 is also shown in figure 1 with its own yaw Y' , roll R' and pitch P' axes and geographic direction or orientation, for example with respect to North.

The viewer 20 allows for graphical visualization of the external virtual world from the pilot ' s point of view, as further discussed below.

An inertial platform 40 is also attached to the viewer 20, which has the task of determining instant by instant the orientation in space of the viewer 20 and its geographical orientation with respect to a fixed reference, for example North.

In particular, the inertial platform 40 may comprise, for example, a three-axis accelerometer 42 and a three-axis gyroscope 44, as well as a magnetometer 46, wherein all said components may also be managed by the electronic control unit 450.

The connection between the viewer 20, the inertial platform 40 and the electronic control unit 450 may be wireless or may be made by traditional means or by cable.

In addition, a camera 97 is attached to the viewer 20, which allows the scene outside the viewer 20 to be filmed, in particular the aircraft instrumentation 10 and the flight controls .

The invention also provides for the use of the GPS (Global Positioning System) system placed on board the aircraft 10 in order to determine the spatial coordinates of latitude and longitude, as well as of altitude, of the aircraft 10, coordinates provided precisely by the GPS system 50.

The input of the spatial coordinates of latitude, longitude and altitude of the aircraft 10 is an information that allows the computer system representing the virtual world, to generate a representation of the virtual world derived from the position, at that moment, consequent to the GPS - Global Positioning System - coordinates of the aircraft 10 in flight .

In other words, the method of the invention provides a step of real- time detection of the geographical position of the aircraft that is carried out with the help of a GPS detector placed on board said aircraft .

However, it is quite clear that the GPS system could be replaced or combined with other known geolocation systems, for example GLONASS, BEIDOU, GALILEO, A-GPS, QZSS or others without going beyond the scope of the invention decriminated and claimed.

Further, the speed of the aircraft 10 that gives rise to the realtime change in the virtual world may be inferred or determined by the change in geographic coordinates measured in real-time by the geolocation detector 50.

The geolocation detector 50 may also detect the altitude of the aircraft 10 and its variation over time, consequently determining the current flight levels of the aircraft 10 and communicating them to the control unit 450 that manages the simulation of the environment outside the aircraft 10.

In order for the method to work, some preliminary system calibration steps are necessary or appropriate, preferably - but not exclusively - to be carried out when the aircraft 10 is on the ground during pre-flight checks .

First, it is necessary to calibrate the orientation of the aircraft 10 relative to a known geographic direction, such as North or relative to the known orientation of a runway,

This calibration can be done using a compass 60 or a magnetometer or using the information given by the known orientation of the runway or by other known means .

It is also necessary to calibrate the pitch and yaw angles of the aircraft 10 with respect to a reference system integral to the external environment .

Secondly, it is necessary to calibrate the spatial orientation of the visor 20 with respect to a reference system integral with the aircraft 10 itself, so that at the beginning of the operation of the system, the spatial orientation of the yaw axes Y' , roll R' and pitch P' of the visor 20 coincides substantially with the spatial orientation of the respective yaw axes Y, roll R and pitch P of the aircraft 10.

This calibration can be performed by having a straight-headed pilot C observe a fixed reference 70 placed on the cockpit of the aircraft 10, for example with the aid of the camera 97. Other methods of calibration are however possible (figure 3) .

Such a calibration may, alternatively, also be performed in flight, after having worn the visor 20, since it serves solely to calibrate the spatial orientation of the visor 20 with respect to the reference system integral with the aircraft 10, regardless of the geographical position and spatial orientation of the aircraft 10 itself .

Finally, a calibration step, which may for example be performed when the aircraft 10 is on the ground, is performed between the geographical point where the aircraft 10 is initially located and the coordinates measured by the geolocation detector 50 placed on board said aircraft 10 at said position.

Figure 2 illustrates a block diagram of the main steps of the method of the invention.

In particular, according to the method of the invention, first of all, real time detection of roll, pitch and yaw data of the aircraft 10, as well as geographic orientation with respect to a fixed reference (block 100) is performed.

As illustrated above, the phase of real-time detection of roll, pitch and yaw data of the aircraft 10, as well as of geographic orientation, is carried out with the aid of the inertial platform 30 placed on board said aircraft 10 and of the instruments proper to said inertial platform 30, namely the three-axis accelerometer 32, the three-axis gyroscope 34 and the magnetometer 36. The method also includes collecting real-time roll, pitch and yaw data, as well as geographic orientation, from the visor 20 worn by the pilot C (block 110) .

In particular, the real time detection phase of the roll, pitch and yaw data, as well as of the geographic orientation, of the visor 20 worn by the pilot C is performed with the help of the inertial platform 40 placed on board said visor 20, and of the instruments belonging to said inertial platform 40, i . e . , the three-axis accelerometer 42, the triaxial gyroscope 44 and the magnetometer 46.

The method further comprises detecting in real time the geographic position of the aircraft 10 using a geolocation detector 50 placed on board said aircraft 10 (block 120) .

During the execution of the method according to the invention, the detected data relating to the orientation in space of the aircraft 10 in flight is associated with the current geographical position data of the aircraft itself, determined by means of the geolocation device 50.

In order to compensate for variations in the orientation of the head of the pilot C and thus consequently to compensate for variations in the orientation of the visor 20 worn by the pilot C with respect to a reference system integral with the aircraft C, a further calculation step is carried out .

This calculation step is aimed at determining exactly the orientation of the viewer 20 with respect to the external environment (block 140) .

In particular, there is provided a step of calculating any differences between the orientations of the yaw axes Y' , roll R' and pitch P' of the visor 20 and the respective orientations of the yaw axes Y, roll R and pitch P of said aircraft 10, said calculation step being aimed at determining instant by instant the spatial orientation of the visor 20 with respect to the external environment, so that the visor 20 displays in real time images representing the external environment according to a current spatial orientation of the visor 20.

In fact, once the spatial orientation of the visor 20 with respect to the external environment is known, the method according to one aspect of the present invention coirprises representing on the visor 20 virtual images corresponding to the external environment that would be visible to the pilot C under normal visibility conditions (block 150) at the current geographical position and with the current spatial orientation of the aircraft 10.

In essence, the roll, pitch and yaw data of the aircraft 10 and the worn visor 20 are used to generate images viewable by the visor 20, wherein said images represent the environment outside the aircraft 10 at the current geographic location, taking into account the current orientations of the aircraft 10 and the visor 20.

Thus, according to an embodiment of the present invention, all the inputs described above and coordinated with each other allow the system to reproduce the external world in a virtual but extremely precise manner.

This takes into account that, unlike traditional flight simulators, it is the aircraft that moves, and the virtual world created by the method of the invention is stationary exactly as in reality.

When the pilot wears the visor 20, he/she sees the virtual world displayed in the same way he used to see the same space and the same world but without undesired atmospheric effects (fog, darkness, rain, etc. ) .

For example, in the case of a situation that all pilots fear, even those equipped with IFR - Instrument Flight Rules : fog or darkness, or in the case of a situation where it is necessary to rescue someone, but the fog prevents it . In cases such as these, by wearing the visor 20 and using the method the invention, possibly by calibrating the spatial orientation of the visor 20 with respect to the reference system of the aircraft 10 (even if necessary on board the aircraft in flight) , the effect is obtained that the world appearing on the visor 20 is the same as the real one but without fog, without darkness : a world of colors and light .

The pilot will be able to "see" where he/she couldn' t before, he/she will be able to save that life that he/she couldn' t before .

It should also be noted that the system according to the invention allows for an increase in flight safety in conditions of fog or poor visibility, allowing aircraft flying VFR and also for those equipped with IFR instrumentation to fly in conditions of absolute visibility.

It is also suitable for military aircraft for missions in prohibitive weather conditions or of a covert nature .

The system of the invention makes any aircraft an instrument aircraft at an absolutely affordable cost.

Anew acronym VIFR - Visual Instrument Flight Rules - can be considered i .e . the condition made possible by the present invention to be able to practice visual flight with visual flight rules but with assisted or autonomous flight systems equal to instrument flight.

In addition to containing sufficient data to represent the external environment, the system can also have - in addition - a mapping of the obstacles present at low altitude .

In particular, it is possible to foresee the visualization of the presence of the aforementioned cables on the basis of maps released by telephone and/or electricity companies .

In fact, it is well known that many low-flying accidents occur precisely because of the presence of such cables . Alternatively, it is possible to envisage the construction of maps of obstacles present at low altitude, where such maps are created on the basis of software analysis of satellite photos of the area to be mapped, where such photos present an indication of the time and day on which they were taken by the satellite .

On the basis of the shadows of said obstacles detectable by such satellite photos and on the basis of the information of the time and day when they were taken by the satellite, it is possible to derive the height of such obstacles and to construct the corresponding map.

According to the method of the invention, it is contemplated to generate a visual and/or acoustic alarm in case the aircraft 10 is in the proximity of to or on a collision course with said cables belonging to telephone lines or power lines .

In the event that a rescue intervention is to be carried out or there is a supervening danger situation (generically indicated with the numerical reference 95) right in the area where an aircraft 10, for example a helicopter, is to land, the electronic control unit 450 may receive a danger signal or a green light ("landing allowed" ) , as the case may be, signal sent, for example, by rescuers who have arrived on the scene before the helicopter and update the representation of the virtual environment provided to the pilot C accordingly, all to the advantage of flight safety.

In particular, the transmission of data related to the safety of the aircraft from external sources to the electronic control unit 450 takes place via a data link operating with 5G technology.

Obviously, modifications or improvements may be made to the invention as described without departing from the scope of the invention as claimed below.