IKinema LiveAction and Unreal Engine: A Powerful Duo for Live Virtual ExperiencesSiobhán Hofma, IKinema on | Tags news
To successfully convince and engage your audience during their virtual experience is the ultimate high-five for creators in the virtual field. Achieved by means of compelling storytelling, dynamic environments, and importantly – characters that behave with infinite realism and believability – essential elements to emotionally connect with characters inside their story, film and TV is the perfect modern-day example.
So, when the project brief for denizens of mediums such as TV broadcast, virtual production, and the arts was to deliver realistic and credible characters, many have relied on our VR and virtual production tech, LiveAction, in UE4 to produce a series of diverse and cutting-edge live virtual shows.
What is LiveAction
At the core of IKinema’s technology, is a powerful full-body IK solver which delivers the most natural and believable motion for 3D animated characters. With LiveAction software, the actor's performance is live captured using motion capture cameras, the data is instantaneously streamed to Unreal Engine through LiveAction, data is auto-cleaned using ACP filters, then perfectly retargeted to the virtual character (with zero latency). The final output is continuous, uninterrupted natural motion from actor to character, and a powerfully efficient real-time pipeline saving animation teams an immeasurable amount of time and frustration.
By leveraging LiveAction with Unreal Engine, they kicked out some of the world’s most interactive and realistic virtual experiences to date.
Hasbro, Facebook Live Broadcast: Announcement launch of Monopoly Ultimate Banking game, LiveAction acted as the bridge between motion captured from an actor performing in studio, and retargeting the MoCap data directly to their 3D animated Mr. Monopoly moving within a cartoon stage. Running parallel with Unreal Engine and several other technologies, LiveAction helped bring Mr. Monopoly to life producing realistic mannerisms for a convincing performance, and bolstered the team with an unbreakable pipeline during the live Q&A session with their fans.
Globo TV, 2016 Rio Olympics: Here, LiveAction and UE4 provided the Globo TV team with a fully reliable real-time pipeline to stream and retarget athletes captured data to their animated avatars, for live broadcasts to Brazil’s nation throughout the Rio Olympics. Real presenters could analyse match replays, walk around and interact with virtual athletes and their virtual stadium environments during real-time sequences.
Ninja Theory, Hellblade: Senua’s Sacrifice: LiveAction played a role in seamlessly delivered perfectly matched actor behaviour (on-set), resulting in convincing humanoid motion for the virtual Senua character (on-screen) with cutting-edge and first-ever-seen VFX techniques. The show unfolded directly in front of a live audience, winning Best Real-Time Graphics & Interactivity Award at SIGGRAPH 2016.
The New York Metropolitan Museum, The Return: An interactive art installation featuring a digital recreation of Adam. Tullio Lombardo's profound scultpure. combined digital puppetry, motion capture and live action. Visitors could ask the onscreen Digital Adam questions, with the actor performing and answering in a separate volume onstage to onlookers.
Under the hood
LiveAction Main features
- Live motion capture streaming and solving with zero latency on multiple characters (example, up to 12 on a medium spec CPU, and 13 plus on more powerful processors)
- Supports OptiTrack, Vicon, and Xsens motion capture systems
- Supports joint retargeting and rigid body solving (Facility and Studio versions only)
- Animation Cleaning Pipeline (ACP) – auto real-time correction of MoCap data such as noise, jitter, feet sliding, floor penetration
- Dedicated UI for full pipeline setup in Unreal
- Solve on similar or disproportionate characters
New to LiveAction 3.0
- Automatic bone mapping
- Template editor – custom name matching
- Skeleton matching on target rig
- Improved work flow for ACP
- Supports custom MoCap systems through the MoCap Streaming Protocol (VRPN based)
- Fast setup – see results within a few minutes
So, when your next brief reads something like this:
PROJECT: Virtual Experience
DELIVERY DATE: Ahh… yesterday
BRIEF: Mind blowing character motion. On budget. On time. MSL (Minimal Stress Levels)
NOTE TO SELF: Download Unreal Engine and contact Support@IKinema so we can help!!
IKinema LiveAction 3.0 is available for Windows platform and comes in both node-locked and floating licenses. Three license levels on offer – Binary, Facility, and Source. Visit the IKinema for more info.