The article was prepared specifically for the UNIGINE blog by the press service of Daedalean AG
Today we would like to introduce you to Imaginator, a simulator project aimed at training and testing visual-based flight control systems. It is aimed to be a developer tool and simulates the output of onboard visual sensors (cameras). Imaginator was created by Daedalean, a Swiss technological company, and is powered by the UNIGINE rendering engine.
What is Daedalean? A Zurich-based start-up founded in 2016, Daedalean develops flight control systems based on сomputer vision and machine learning. They will be eventually used to power AI-based piloting systems capable of outperforming human pilots in all their functions. But even today they already have their use: the systems that Daedalean is currently preparing to release serve as pilot aid, providing data for air traffic detection, GPS-independent navigation and landing. Such ‘AI co-pilots’ work through real-time processing of the visual data from cameras installed on an aircraft by neural networks working in the onboard computer. These systems, as well as the eventual full autonomy, are eagerly expected by the aviation industry. But on top of that, they will play a key role in the development of the new sector which is not yet started: advanced air mobility—pilotless electric aircraft serving as an air taxi, cargo delivery, ambulance fleet, etc.
How did the idea arise? Daedalean is training its neural networks to recognize images received from cameras:
- Visual navigation—identify key points on the landscape and compare them to a map to identify their own position.
- Landing advisory—spot and identify a helipad or runway, determine distance and the angle of approach, recognize safe spots and obstacles for emergency landings (for example, when a car moves into the spot marked as safe, it’s not safe anymore);
- Traffic detection—detect and classify hazards (other aircraft as well as birds, clouds, buildings, trees, wires, etc.).
Training and testing, then adjusting the parameters and testing again, require hundreds of thousands of flight hours. The cost for real flight testing would be prohibitive. This is why Daedalean began developing an alternative solution of its own: a simulator able to provide the synthetic input to the algorithms, that is, to duplicate as closely as possible how cameras onboard an aircraft see the environment under real-world flying conditions.
The architecture of Imaginator is innovative for professional flight simulators: it’s cloud-based, fully scalable, and all the ‘heavy lifting’ is done on the server-side. This architectural approach made it the first visual simulator of its class provided as software-as-a-service, and a welcome and friendly tool for computer vision engineers to develop, train, and evaluate their visual algorithms, set up their own simulations, and explore edge cases.
What does the UNIGINE engine do? UNIGINE is capable of rendering immersive 3D landscapes of any place on Earth in real-time, with superior photorealism. According to Peter de Lange, Daedalean’s Team Lead of Simulation: “For the purpose of machine learning, developers need to have the base data used as ‘ground truth’ (the gold standard, to which they compare the output generated by neural networks which are being trained), and this is where UNIGINE proves indispensable—its images of the environment provide a perfect synthetic ground truth.”
However, paradoxically, the simulated imaginary should not be too perfect: it should strive to duplicate realistic weather conditions—visibility should change over the entire spectrum of possibilities, be it mist, rain, clouds, etc. The actual imaging from an airborne camera will not be stable, nor will its resolution be very high, and the lenses will be exposed to moisture, debris, and dirt. Moving objects can be perceived by sensors as pixelated, lagged or distorted. A visual simulator must reflect all these variables to be considered as a feasible alternative to flight tests.
Imaginator’s motto is “Bridge the gap between SIM and REAL.” Thanks to UNIGINE’s features, Imaginator provides images of a broad range of available weather conditions (including sunshine, up to three layers of clouds, rain, snow, fog, and haze) that can be ‘realified’: adapted for low resolution, vibration, rotating cameras, etc. Imaginator gives its users full control over weather conditions, flying scenarios, and image quality settings thus allowing them to push their algorithms’ limits during testing. The system needs to detect and classify various objects as well as estimate the distance to them—so the engine easily renders any number of other objects in the air, including various models of airplanes, helicopters, drones, or whatever air obstacles developers want to include in their scenarios.
Why UNIGINE? UNIGINE’s trademark game engine technology ensures that the images are easily customizable for the Daedalean needs, provides scalability, and allows to overcome real-time constraints. Daedalean considered alternative solutions, but none of them provided a similar level of precision, photorealism, or scriptability. Unlike UNIGINE, their APIs do not allow for full control over rendering or accessing the internal rendering buffers; and simulators designed for the automotive industry are optimized for a very different use case: ground view.
Imaginator infrastructure. The main innovative feature of the architecture is moving the UNIGINE-based services to the server-side in the cloud and employing the simulation as a set of microservices. The service can be used immediately, without any installation or overhead, and can be accessed from anywhere. The API allows a developer to generate the first scenario within minutes. Any degree of customization within a client’s request (e.g. locations, weather, image settings, configurations of cameras, the density of air traffic, various malfunctions, etc.) is available smoothly and instantaneously. Load balancing ensures that client demands are distributed equally among the Imaginator servers, and allows for pooling, which is extremely useful for generating large datasets.
Imaginator, though, is not a finished product, it is still under development. The creators have a lot of ideas for its roadmap. To identify the business requirements and possible use cases, Daedalean partners with aircraft and eVTOL manufacturers, as well as avionics manufacturers, so that they can test Imaginator, apply it to their tasks at hand, and offer their feedback. Daedalean plans to launch Imaginator to the market as a SaaS application—Simulation As A Service.