UEFN: The power of UE for Fortnite

The safety and effectiveness of autonomous vehicles relies on rigorous training through simulation. To ensure such training reflects realistic terrain, weather conditions, and scenarios, many companies turn to real-time technology like Unreal Engine.

Case in point: Microsoft Project AirSim uses Unreal Engine to safely build, train, and test autonomous vehicles through high-fidelity simulation. In these realistic environments, AI models are capable of running through millions of flights, faster than in real time, where they learn how to react to countless variables much like they would in the physical world.

Using Unreal Engine for rendering simulation environments, Project AirSim can be deployed on Azure or desktops; this enables it to generate massive amounts of data for training AI models for every action during each phase of flight, from takeoff to cruising to landing. In the safety of Project AirSim’s high-fidelity environments, AI models learn through trial and error how to navigate the full range of terrain, weather, and man-made obstacles to ensure safe flight.

“One of the many applications Unreal Engine offers in this field is computer vision on digital ground truth for training autonomous vehicles of all kinds,” says Sébastien Lozé, Unreal Engine Simulation Division Business Director. “Realistic simulated worlds with embedded effects of weather and other scenarios provide an adequate diversity of environmental conditions for the AI model to learn and become operational for the real world.”

Project AirSim lifts off with Unreal assets and support

To support true-to-life flight, Project AirSim offers access to Unreal Engine and Azure deployment of simulated 3D environments representing diverse urban and rural landscapes, together with a suite of sophisticated pretrained AI models and other design assets to help accelerate autonomy in aerial infrastructure inspection, last-mile delivery, and urban air mobility.

“Our vision is to create an AI-ready simulation platform that enables autonomy, during all phases of flight,” says Ganesh Rao, General Manager, Autonomous Systems at Microsoft.

“Building autonomy requires AI, which requires large amounts of AI-ready data. Project AirSim brings together the power of cloud and AI at a flexible scale, and the creation of new simulation capabilities in combination with existing ones to generate AI-ready data at scale, and build custom machine learning models using MLOps. The modularity and open aspect of Unreal Engine helped us with these goals.”

The entire simulation industry is entering a new era where emerging technologies are increasing the capabilities of artificial intelligence and autonomous vehicles. Being able to ingest very large data-rich environments in real time in Unreal Engine enables companies to test these capabilities in realistic scenarios so that autonomous vehicles of all kinds are fully trained for real-word interaction.

The Unreal Engine community of developers and its partner ecosystem create an extended team to assist companies who use the platform for their simulated training grounds. “Having an ecosystem of designers, developers, and game creators helped us tap into the knowledge and create a wide variety of scenarios, from warehouses to the African Savanna, with various vehicle form factors,” says Ade Famoti, Senior Director, Strategic Partnerships - Project AirSim.

Microsoft says the potential use cases for Project AirSim are almost limitless: next-gen transportation such as eVTOL (electric vertical take-off and landing), inspecting powerlines and ports, ferrying packages and people in crowded cities, and operating deep inside cramped mines or high above farmlands are just some examples.

Leveraging Unreal Engine Project AirSim is enabling the industry surrounding autonomous training to move closer toward scalable commercial operation.

“We are at the cusp of a transformation in aerial mobility as we know it,” says Ganesh. “Project AirSim is a critical tool that lets businesses build, test, and enhance solutions in a virtual environment before safely deploying in the real world.”.

Visit the Unreal Engine Simulation page to learn more about how Unreal Engine is transforming and enhancing the training of autonomous vehicles.