18 avril 2018

Siren at FMX 2018: crossing the uncanny valley in real time

Par Dana Cowley

At GDC 2018, Epic Games teamed with 3Lateral, Cubic Motion, Tencent and Vicon to take live-captured digital humans to the next level. Now, attendees coming to FMX in Stuttgart, Germany can meet “Siren,” the high-fidelity, real-time digital character based on the likeness of Chinese actress Bingjie Jiang. Alexa Lee, the UK-based actress driving Siren's performance, will be live at the Vicon booth 2.1 during the show, which runs April 24-27.


Siren was created as a proof-of-concept demonstration of what is possible to achieve in Unreal Engine. The collaborative project included contributions from several leaders in digital human technology:

3Lateral completed 3D and 4D scanning of actress Bingjie Jiang; data was acquired in the 3Lateral scanning system. Their team produced the model and texture data, built a digital double facial rig with the underlying controls and applied their Rig Logic technology that enabled facial animation to be driven in real-time. 

Cubic Motion handled all of the live and offline facial performance capture, tracking, solving and animation. Cubic Motion’s software analyzed the video from the performer’s head-mounted cameras and then applied that animated performance to the facial rig that 3Lateral created. Cubic Motion also provided a live solver, using the live video feeds of the actress’ performance to drive the CG character in real time. 

Epic Games led look development, which included improvements to rendering techniques and skin shaders, and built the CG character. Epic’s Unreal Engine 4 pulled performance data from Vicon, and facial data from Cubic Motion, applied that data to the character and rendered the scene in real time. Epic also provided engineering resources to support the entire team of artists and creatives.

Tencent worked in conjunction with Epic Games on the character’s look development, and provided high-level creative and technical guidance.

Vicon handled all of the body and finger motion capture for Siren. The Vicon system feeds performance data right into Unreal Engine from its Shogun software system. The capture session for this demo was completed at Vicon headquarters in Oxford, England.

Digital characters are becoming more and more important as new forms of media emerge. Eventually we may all have digital likenesses of ourselves, and having compelling characters that can emote and that we can react to is the next step in creating truly believable characters. 

This demo shows what is possible, and also provides a window into some of the tools that Epic Games will be building into Unreal Engine so that other people will be able to achieve this level of character work as well.

Join the conversation by following @UnrealEngine and the #MeetSiren hashtag on Twitter.