August 14, 2018

3Lateral reveals digital human and character performance advancements in UE4

By Dana Cowley

Earlier this year at the Game Developers conference, we partnered with 3Lateral to reveal a stunning development in real-time, human-driven digital characters, featuring Andy Serkis. Powered by Unreal Engine and 3Lateral’s Meta Human Framework, we were able to create both a digital Andy and a digital character Osiris Black using volumetric data capture of Serkis’ "Macbeth" performance, achieving unprecedented levels of fidelity and realism. These demos show how one set of performance capture data can drive the creation of two vastly different digital characters.

Today 3Lateral has unveiled the next iteration in this story. Now, you can watch as Osiris Black seamlessly morphs into digital Andy while being driven by the same Macbeth performance. 3Lateral’s comprehensive technology framework uses Unreal Engine to power 4D volumetric capture, and empowers much more than just extraction of data for animation – it provides detailed information of the facial deformations and motions to build fully valid animation curves, a next-generation process compared to 2D and 3D facial tracking and solving.
This new demo illustrates 3Lateral's Rig Logic and Gene Splicer technologies, components of Meta Human Framework, that enable transition between alien and human digital DNAs. Furthermore, 3Lateral has developed a unique file format for comprehensive and memory-light digital representation of distinct characters. The morph video also shows how multiple different characters along the transition from alien to human are driven with the same animation curves. This is one more proof point of the empowering nature of 3Lateral technologies. 

This gathering of annotated data and level of photorealism have wider implications for machine learning, and can be applicable in entertainment, biometrics, medicine and many other fields. We’re blown away by 3Lateral’s capabilities, and look forward to seeing what they're able to achieve next.