7. August 2019

Virtual production: putting the latest UE4 tools through their paces

Virtual Production

SIGGRAPH 2019

Film & Television

Spotlight

Von Ryan Mayeda

At our Unreal Engine User Group at SIGGRAPH 2019, we revealed our collaborative project with Lux Machina, Magnopus, Profile Studios, Quixel, ARRI, and Matt Workman, which put our latest virtual production tools through their paces and demonstrated how this rapidly evolving technology can radically change the way films are made.

Offering up their studios in Los Angeles, Lux Machina constructed an LED volume consisting of four LED panels (three walls and a ceiling) powered by nDisplay, together with engine-controlled ARRI SkyPanels. The volume displays a seamless environment running in Unreal Engine that surrounds the actor and props, who are also lit by it, and receive reflections from it. When you look through camera, you see the real-world elements perfectly integrated into the environment in real time, without the need for green screens.

Matt Workman, founder of Cinematography Database, acted as DP and Director. “When I first walked on the set and the wall was up and we started to look through the camera, it really started to feel like I was just filming in this actual location,” he says.

As the DP moves the camera, the Unreal Engine camera moves with it in the 3D world, producing realistic parallax through the lens. “We can track a camera’s position in space in real time and render its perspective so that we can compellingly convince a camera that something else is happening in front of it that really isn’t there,” explains Kris Murray, VP of Technology at Lux Machina.

Since the scene is CG, it can be edited in ways that would be extremely difficult and time-consuming—if not impossible—in a real-world location; because it’s running in Unreal Engine, changes can be made live on set. For example, if a rock is located in an inconvenient position for a particular camera angle, you can simply pick it up and move it, virtually.

With the ability to remotely control Unreal Engine via a new plugin, lighting can be adjusted from a device such as an iPad. Cinematographers and gaffers can dynamically control the lighting of a scene on set, both within the CG environment and on the real-world actors and props, with just a fingertip gesture—something that Workman finds very exciting. “This opens up kind of like a virtual playground to shoot in,” he says.

Whether you’re using the output from the virtual set as final pixel or as an intermediate stage in the film production, the advantages of the high-fidelity rendering are clear.

Philip Galler, Chief Technology Officer at Lux Machina, elaborates. “No matter what the project is, the creatives always want to see the closest representation to the final product as early on in the creative process [as possible],” he says.

“In this new frontier of virtual production, the filmmaker is more grounded into the scene that they're shooting, into the story they're trying to tell,” says AJ Sciutto, Virtual Production Producer at Magnopus.

From the team’s experience of working on this project, it’s clear that virtual production fosters collaboration among the entire crew. “Because you can interactively change the world, it brings all of those departments together because each one of them has a role in how this world is portrayed at some point along in production,” says Matt Madden, Senior Virtual Production Supervisor at Profile Studios.

Kim Libreri, Chief Technology Officer at Epic Games, agrees. “We've tried to expand the engine to be a very collaborative platform,” he says. “So the director of photography, the director, they can all go into the scene and make modifications as they wish, as opposed to waiting for months and months.

“It's been my dream to get computer graphics to the point that they're totally photoreal,” he continues. “I’ve loved video games and movies all my life, and this is bringing the best of them together.”

Want to find out more about this project? Stay tuned for some extended footage and more in-depth explanations coming soon. Meanwhile, why not explore the new frontier of virtual production yourself? All of these features are available in the latest preview version of UE4. Download Unreal Engine today.