Taking Unreal Engine's latest in‑camera VFX toolset for a spin

July 21, 2021
Recently, Epic Games and filmmakers’ collective Bullitt assembled a team to test out the latest in-camera VFX toolset, part of the extensive suite of virtual production tools in the upcoming Unreal Engine 4.27 release. To put each of the new tools through their paces, they created a short test piece to mimic a production workflow.

Shot entirely on NantStudios’ LED stage in Los Angeles, the production test capitalized on new workflows that enable dynamic lighting and background changes on set with minimal downtime, together with multi-camera and traveling vehicle shoots. It also harnessed the power of twin NVIDIA Quadro A6000 GPUs. The team was able to produce final pixels in just four days—a demonstration of the efficiency and creative freedom the toolset offers. 
 
The team also took advantage of new Level Snapshots that enable directors to easily match previous takes when returning to an earlier set or lighting configuration; sophisticated remote control UI building tools that offer creatives direct control of the Unreal Engine-powered elements of the shoot from the convenience of a tablet or laptop; and OpenColorIO color management that ensures artists creating Unreal Engine assets can see what the final captured output will look like.

Over the coming weeks, we’ll go behind the scenes to explore how these new tools were used to finish the test piece in record time, and show you how you can use the techniques on your own real productions. We'll also talk to the team and get their take on how these advancements in on-set workflows are transforming the future of filmmaking. And, we’ll share the assets as a sample project on Marketplace. To make sure you don’t miss any of this content, sign up below to receive the latest updates in your inbox as they happen.
 

Interested in finding out more?

Sign up below and we’ll send you links to every article we post, event we organize, and announcement we make regarding virtual production and in-camera VFX.


In the first video in the series, the filmmakers talk about the huge benefits they witnessed when working with these latest updates to Unreal Engine’s in-camera VFX toolset. The entire team, including the director, producer, production designer, director of photography, and first assistant director, all agreed that what they experienced represented a quantum leap for on-set filmmaking.
 
The ability to find new shots on the stage is just incredible. In the same amount of time that it would take me to change a light in the physical world, I'm able to have new lighting on the wall. As a director, that's a game changer. 
- Ryan McNeely, Director, Visual Creatures
Recently, some intrepid filmmakers have been reaping the benefits of using in-camera VFX within LED volumes in productions such as The Mandalorian. With existing tools, however, digital set changes could require hours or even days of time for light baking, losing much of the spontaneity in the process. The new toolset in Unreal Engine 4.27 includes GPU Lightmass, which takes advantage of light baking on multiple GPUs to radically accelerate set changes, making them as much as an order of magnitude quicker. Instead of breaking for the day, teams can now just take a coffee break.
Ryan McNeely, who acted as director on the production test, explains why this is such a huge step forward. “As a director, time is your biggest enemy, and the thing about this, the speed at which we're able to move the amount of setups, that's honestly the dream,” he says. “The ability to find new shots on the stage is just incredible. In the same amount of time that it would take me to change a light in the physical world, I'm able to have new lighting on the wall. As a director, that's a game changer.”
For renowned filmmaker Anthony Russo, it’s the ability of in-camera VFX to bring everything together and enable the team to see the final result right on set. “One of the things that excites us most, of course, about this is the fact that we can do in‑camera choreography, where all the elements of the frame are actually in concert with one another and organically working on one another to create a more visceral experience,” he says.

Director of Photography Matthew Jensen agrees. “It's so much easier now to be able to make the changes in real time, see their effects, and then make adjustments from there,” he says. “It allows for a happy accident. It allows for more spontaneity in the creative process.”
What’s more, in-camera VFX has the ability to remove the silos that can cause creative intent to get lost in translation. 

“It's not separated into components of pre-production, shoot, and post-production,” says Diane Castrup, Producer. “Usually you just have your physical set that you're standing on and the green screen is behind you and you just kind of give it over after you're done shooting and you hope that it all works out.”

Production Designer Quito Cooksey echoes these sentiments. “Once principal photography ends, post takes over,” he says. “And if you're not part of that, everything that you really envisioned and designed goes through a filter and doesn't always come out the other end in the way in which you envisioned. With this, you have the virtual art department with control of the concept designs all the way through to final pixel.”
Sean Harner, First Assistant Director, admits to being skeptical when he first arrived on set. He’s now a convert. “I am totally blown away with how you can move things around, how we can change sun direction, how I can physically change sets,” he says. “We’re taskmasters, and it's all about time. If I lose an hour on a commercial or a movie, I can't get that time back and that could be $100,000 or that could be $500,000.”
It was photoreal and it was ready to reel. That means that you could go from shooting it to projection in the same day. That's technology at its finest. 
- Sean Harner, First Assistant Director, 2H Productions
Another huge advantage of the new toolset is the ability to quickly and easily recall a set that you’d shot earlier—perfect for those additional pickups, as Harner discovered when working on the project. “And just in between the sets is 20, 30 minutes’ setup,” he says. “It was photoreal and it was ready to reel. That means that you could go from shooting it to projection in the same day. That's technology at its finest.”

The last word goes to Joe Russo. “It feels like we're moving from filmmaking 1.0, jumping to filmmaking 5.0—and I don't see any other path forward that is nearly as exciting, compelling, or freeing for artists as this path,” he says.
 
At Epic, we have an extensive virtual production team who are dedicated to finding better ways for filmmakers to work. Here, we chat to some of them about the Unreal Engine 4.27 features used in this production test.
 
Unreal Engine 4.27 is currently available in Preview; you can download it from your Epic Games launcher. The exciting new workflows and tools it offers are influenced by input from leading filmmakers who are pushing the limits of virtual production today. As the Bullitt team discovered, the toolset represents an evolutionary leap forward in quality, efficiency, and ease of use—delivering whole new levels of creative freedom. 

UPDATE: Unreal Engine 4.27 is now available, and you can download the full sample project from this ICVFX production test to explore for yourself. 

    Interested in finding out more?

    Sign up below and we’ll send you links to every article we post, event we organize, and announcement we make regarding virtual production and in-camera VFX. 
    News
    August 19

    Unreal Engine 4.27 released!

    Creators across all industries have something to celebrate with this release: In‑camera VFX goes next-level with a slew of efficiency, quality, and ease-of-use improvements, while other highlights include path tracing for stunning final images, out-of-the-box access to Oodle and Bink, production-ready Pixel Streaming, and much more.
    News

    Unreal Engine 4.27 released!

    Creators across all industries have something to celebrate with this release: In‑camera VFX goes next-level with a slew of efficiency, quality, and ease-of-use improvements, while other highlights include path tracing for stunning final images, out-of-the-box access to Oodle and Bink, production-ready Pixel Streaming, and much more.
    Event
    October 5

    Build: Architecture

    Build: Architecture 2021 is coming! Join us on November 2 for architectural configurators, digital twins, open worlds, and more. Registration now open. 
    Event

    Build: Architecture

    Build: Architecture 2021 is coming! Join us on November 2 for architectural configurators, digital twins, open worlds, and more. Registration now open. 
    Event
    September 28

    Join us at Autodesk University 2021

    Our activities at this year’s Autodesk University, October 5-7, will be all about connectivity—connecting data, technology, and people to make better experiences through real-time technology. Join us at our virtual booth, at the AU Media and Entertainment Keynote, or at one of our many product demonstrations and sessions.
    Event

    Join us at Autodesk University 2021

    Our activities at this year’s Autodesk University, October 5-7, will be all about connectivity—connecting data, technology, and people to make better experiences through real-time technology. Join us at our virtual booth, at the AU Media and Entertainment Keynote, or at one of our many product demonstrations and sessions.