MindMaker: Deep Reinforcement Learning

Aaron Krumins - Blueprints - Jun 3, 2019

Design Self-learning NPCs using Deep Reinforcement Learning (A2C, DDPG, PPO, TD3, DQN, SAC)

  • Supported Platforms
  • Supported Engine Versions
    4.27, 5.0 - 5.3
  • Download Type
    Complete Project
    This product contains a full Unreal Engine project folder, complete with Config files, Content files and .uproject file, which can be used as a template to create a new project.


Introduction to MindMaker: https://www.youtube.com/watch?v=ERm_pZhAPIA

Blueprints Overview: https://youtu.be/Tuo423NujEk

Discord Group: https://discord.gg/shxFwtmsHa

Add life like realism to video game characters using Deep Reinforcement Learning – a machine learning approach that combines neural networks with a learning model to sculpt agent behavior. Using the DRL Learning Engine and the MindMaker AI Plugin, an NPC can be made to perform any combination of actions involving objects in it's environment simply by changing a reward function.

With these tools, games and simulations can function as OpenAI Gym environments for training autonomous machine learning agents. Possible use cases include robotic simulation, autonomous driving, generative architecture, procedural graphics and much more. For game developers, the use cases for self-optimizing agents include controlling NPC behavior (in a variety of settings such as multi-agent and adversarial), prototyping game design decisions, and automated testing of game builds.

The MindMaker DRL Learning Engine *: A functioning version of the DRL Learning Engine is included with project. Algorithms presently supported in MindMaker DRL for UE 5.1 include Stable Baselines3 : Actor Critic ( A2C ), Deep Deterministic Policy Gradient (DDPG) , Deep Q Network ( DQN ), Proximal Policy Optimization ( PPO ), Soft Actor Critic ( SAC ), Twin Delayed DDPG ( TD3 ). Includes Python source code for the learning engine is included in Content\MindMaker\MindMakerSource

* The DRL Learning Engine is a standalone executable contained within the project files that is launched at the commence of play by the MindMaker AI Plugin

Technical Details


·        Deep Learning for Large Observation and Action Spaces

·        Support for OpenAI Gym custom environments

·        Modular Design to scale across multiple projects

·        Save and Load Previously Trained Behaviors

·        Support for Multiple Rewards For Complex Behaviors

·        Direct Access to Neural Network via Blueprints

·        Detailed Explanations and Tutorial

·        No Expertise with Neural Networks Required

· Includes Python Source Code for Modifications

· Includes the required MindMaker AI Plugin

Number of Blueprints: 7

Input: NA

Network Replicated: No

Supported Development Platforms: Win 64

Supported Target Build Platforms: Win 64

Documentation: https://github.com/krumiaa/MindMaker

Tutorials and Examples:

Cart Pole: Creating a Custom Deep Reinforcement Learning Environment in UE4

Further Resources:

Intro to Reinforcement Learning for Video Game AI