April 29, 2019

Virtual production in Unreal Engine 4.22: livestream recap

Virtual Production

Film & Television

News

By Jimmy Thang
As you may have noticed in the release notes, the recent launch of Unreal Engine 4.22 delivered several virtual production improvements. To highlight some of them, Epic Games Senior Cinematic Designer Grayson Edge recently conducted a livestream where he showcases improvements to Sequencer, explains how Take Recorder works, and shows viewers how to use Vcam with an iPad. He demonstrates how these numerous improvements have dramatically increased efficiency to a point where what used to be a week's worth of work can now be trimmed down to a single day. He also shares his production workflow. You can check out the stream in its entirety below.  
Edge begins the stream by outlining what virtual production means within the context of Unreal Engine. Specifically, he defines it as the cross section between physical and digital worlds. Elaborating further, the cinematic designer says that virtual production enables directors and creatives to work on a physical stage, but view and interact with virtual environments and characters at the same time. 

In the stream, Edge showcases a variety of tools that his team uses to produce cinematics. On the hardware front, these tools include Blackmagic Studio cameras and an iPad. When you couple these devices with actors wearing mocap suits, you've got the foundation to start puppeteering in-game characters in real time.

Spicing up the set, Edge elaborates on the physical props that his team uses, which include swords, guns, and axes that have mocap markers attached to them. A relatively new tool that the team has been leveraging is a Bluetooth device that syncs up to mocapped prop guns, enabling actors to fire the weapon for a corresponding in-engine effect. Finally, iPhones attached to a head mount enable facial mocap data to be applied on an in-game skeleton through the Live Link plugin, in real time.

Introduced in 4.22, Take Recorder, which is an extension of Sequence Recorder, combines these elements together. While Sequence Recorder has been used to record actors' performances for many projects, including Epic’s VR game Robo Recall, it wasn't streamlined for virtual production. On the other hand, Take Recorder, with its simpler UI, was tailor-made for it. It's also much better at managing data, which is helpful in a virtual production environment. 

In the video stream embedded above, Edge shows viewers how to enable and use Take Recorder. He provides several tips, which include:
  • Adding different actors into Take Recorder
  • Adding and recording microphone audio
  • Adding Live Link as a source
  • Enabling recording
  • Naming slates
  • Reviewing the recording
  • Assigning colors to different tracks to easily classify elements such as cameras, actors, and more
  • Having an actor record a base performance to perform against himself/herself on subsequent takes
  • Matching a virtual in-engine camera with a real physical camera to get 1:1 representations of the virtual and real world

Edge also shows how to set up and use the Virtual Camera plugin with an iPad. With this tool, directors, cinematographers, and creatives can achieve numerous powerful things in real time. In the stream, he shows how users can record with the iPad. As well as sharing best practices, Edge gives practical advice on: 

  • Adding motion stabilization
  • Zooming in and out
  • Adjusting focal length
  • Reducing keyframes to get smoother camera movements
  • Optimizing tablet performance

Edge wraps up the stream by showcasing several other new virtual production features. Animation blending, for instance, enables users to smoothly blend between two different animations, which was not something that was easy to achieve in the past. He also highlights the fact that users can copy and paste animation tracks; by making slight variations on these tracks and then duplicating them, users can create simple, yet believable crowds. Finally, Edge shows off the new object-binding track where users can manually change materials, meshes, and more.

To read about how famed director Robert Zemeckis used virtual production on the set of the movie Welcome to Marwen,  check out our Spotlight video. For more news, articles, and insights, visit our virtual production page and sign up for our newsletter.

If you would like to play around with these new virtual production features yourself, download Unreal Engine 4.22 for free today.