April 28, 2016

FMX Update: Put Your Digital Game Face On

By Dana Cowley

FMX 2016, set to the theme of “Blending Realities,” is in full swing in Stuttgart, Germany. Epic’s sessions and more on that are detailed here

fxguide’s Mike Seymour interviewed members from Ninja Theory, Epic Games, Cubic Motion, 3Lateral and IKinema over the weeks following GDC, where the performance debuted. Mike has published the FMX primer, “Put Your (Digital) Game Face On” which provides a highly detailed record of the steps taken to make Hellblade: Senua’s Sacrifice the most realistic live performance inside of a game engine to date.

Here's an excerpt:

This remarkable project came together in just 8 weeks. To achieve this the team used the most cutting-edge facial rigs, latest advances in eye modeling, rendering and deep learning algorithms.

Melina Juergens, Ninja Theory’s video editor, was the model for Senua and the performance artist who drove the rig at GDC. To summarise the process, her face was filmed with a mono head camera, and her body tracked via an Xsens rig. The face data fed the 3Lateral rig via the Cubic Motion's solver. The body data fed the IKinema body IK system. The face and the body solutions were then combined and rendered in real time in the Unreal Engine with some custom modifications.

UE4 render (left) compared to EXR from photoshoot - internal skin test


Anyone interested in mixed reality live performance, advanced facial capture, real-time motion capture and animation, and computer graphics rendering will find this piece of interest. Thanks to Mike for documenting this historic collaboration.

In addition, Brian Karis has posted his Digital Human slides from GDC 2016, and you can watch all of our presentations that took place in West 3001 at the GDC 2016 YouTube playlist.

On a side note, the inaugural FMX VR Jam, hosted by Epic, wraps tomorrow. We've brought together five student teams from across Europe to test their creative and technical talents as they explore the theme, "What Moves You?" Each member on the winning team will receive a Vive Pre developer kit thanks to Valve's support of our Unreal Dev Grants program. VRFocus has the full story.