April 18, 2018
Siren at FMX 2018: crossing the uncanny valley in real time
Siren was created as a proof-of-concept demonstration of what is possible to achieve in Unreal Engine. The collaborative project included contributions from several leaders in digital human technology:
3Lateral completed 3D and 4D scanning of actress Bingjie Jiang; data was acquired in the 3Lateral scanning system. Their team produced the model and texture data, built a digital double facial rig with the underlying controls and applied their Rig Logic technology that enabled facial animation to be driven in real-time.
Cubic Motion handled all of the live and offline facial performance capture, tracking, solving and animation. Cubic Motion’s software analyzed the video from the performer’s head-mounted cameras and then applied that animated performance to the facial rig that 3Lateral created. Cubic Motion also provided a live solver, using the live video feeds of the actress’ performance to drive the CG character in real time.
Epic Games led look development, which included improvements to rendering techniques and skin shaders, and built the CG character. Epic’s Unreal Engine 4 pulled performance data from Vicon, and facial data from Cubic Motion, applied that data to the character and rendered the scene in real time. Epic also provided engineering resources to support the entire team of artists and creatives.
Tencent worked in conjunction with Epic Games on the character’s look development, and provided high-level creative and technical guidance.
Vicon handled all of the body and finger motion capture for Siren. The Vicon system feeds performance data right into Unreal Engine from its Shogun software system. The capture session for this demo was completed at Vicon headquarters in Oxford, England.
Digital characters are becoming more and more important as new forms of media emerge. Eventually we may all have digital likenesses of ourselves, and having compelling characters that can emote and that we can react to is the next step in creating truly believable characters.
This demo shows what is possible, and also provides a window into some of the tools that Epic Games will be building into Unreal Engine so that other people will be able to achieve this level of character work as well.
Join the conversation by following @UnrealEngine and the #MeetSiren hashtag on Twitter.