Image courtesy of Aaron Sims Creative

THE EYE: CALANTHEK explores real-time filmmaking in Unreal Engine 5 Early Access

Created in just six weeks with Unreal Engine 5 Early Access, MetaHuman Creator, and a small team of artists, THE EYE: CALANTHEK is an enigmatic and suspenseful short film by Aaron Sims Creative (ASC). It premiered on October 28 in Shortnitemares, a virtual shorts festival curated for Fortnite and its Halloween-themed Fortnitemares event.

The film tells the story of intergalactic maintenance worker Valentina (named for Valentina Tereshkova, the first woman in space) who is called to repair a drone on a distant planet, where she encounters a hostile alien creature and a strange force that won’t let her leave.
 
As part of the project, Aaron Sims Creative is making the alien, whom they named Teuthisan—derived from the Ancient Greek for ‘squid’—available to download for use in Unreal Engine 5 Early Access free of charge.
Image courtesy of Aaron Sims Creative

Pivoting from live-action to full-CG

With the film initially conceived as a live-action piece to be shot using virtual production techniques on an LED stage, its creators were forced to pivot to a full-CG production due to unforeseen circumstances. This shift also compressed an already ambitious timeline, requiring the team to make careful choices about how to proceed.

The condensed schedule meant that there was limited time to create brand-new assets; the alien creature, which had already been created for the live-action project, would have to be incorporated into the new plot.

The film’s director Aaron Sims, renowned creature designer and founder of ASC, joined forces with writer Vivian Yoon, who also voiced Valentina, to develop the concept into something that could be achieved under these constraints. By the time Vivian came on board, the spaceship and environment had also already been established, using assets from TurboSquid together with Quixel Megascans, which are included with Unreal Engine. In Unreal Engine 5, Quixel Bridge is integrated into the Unreal Editor, meaning you can drag and drop Megascans directly into your projects.
Image courtesy of Aaron Sims Creative
“All of our story ideas were based around using these assets and creating a story around that,” says Vivian. “Aaron and I talked about how cool it was to sort of paint yourself into a corner and then have to figure it out, and spark new and creative ideas. We were starting from a place of ‘How do we have an impactful ending?’, and once we came up with that, we reworked the script to have one cohesive story.”

Crafting the film’s leading lady in MetaHuman Creator

But one character had not yet been created: Valentina was originally intended to be a real actor. Until recently, creating a convincing digital human could take weeks or months. But Aaron and the team were able to turn to MetaHuman Creator, which enables anyone to make one in about an hour or less.

“Without this technology, I wouldn’t have gone with a human character on this project—it made bringing her to life so much easier,” says Aaron. “Before MetaHuman Creator, creating a character like this from scratch would have been impossible, given the timeframe.”

Valentina’s spacesuit and helmet were crafted in Maya. By obscuring her mouth, the team was able to avoid spending time on lip-syncing, something that wouldn’t have fitted into the extremely tight schedule.
Image courtesy of Aaron Sims Creative

High-quality previs informs decision-making on visuals and story right from the start

To sketch out the character animation, Aaron and motion capture actor Christopher Graham used an Xsens suit and StretchSense gloves to capture motion for both Valentina and the alien from the comfort of their home. Using Live Link, they were able to preview the shots in real time.

“It meant we didn’t have to spend days doing keyframe animation to block out the characters’ performance, so that sped up the process,” says Aaron. “It got us to something as proof of concept very quickly.”

To further validate their ideas, the team created high-quality previsualization very early on in the production. “We were blown away because it looked like a finished product,” says Vivian. “The story took shape from there.”
Aaron explains the importance of this. “Part of creating in Unreal Engine is knowing the limitations,” he says. “I like taking risks, I always think I can do it no matter what, and if I hit a wall, I deviate—and the software enabled me to deviate so quickly. A lot of people do previs as gray renders or it’s very rudimentary; ours is very close to the final product, so we answer those hard creative decision-making questions on day one and then refine and massage our story with more flexibility.”

Producing final pixels from Unreal Engine 5 Early Access

Much of the early motion capture data ended up being used in the final film. This data was transferred onto the characters in Unreal Engine 5, before a customized Control Rig was added to enable keyframed animation to be layered on top. To help with these tasks, as well as some shot setups and effects, the team brought in Shape Shifters Creative.

The facial animation was keyframed using the MetaHuman facial rig, which is included with every MetaHuman for use in Unreal Engine. With the mouth hidden, a lot of the emotion needed to be conveyed through the expressiveness of the eyes.

The final frames were entirely rendered in Unreal Engine 5, with assets from the Unreal Engine Marketplace for ambient effects—such as dust and particles; sky and clouds; and smoke—adding the finishing touches.
Image courtesy of Aaron Sims Creative
“There was no post-production done outside of Unreal aside from the final editing, for which we used Adobe Premiere Pro 2021,” says Aaron. “All renders as you see them were created in Unreal Engine 5.” 

Real-time filmmaking leads to nonlinear decision-making and creative iteration

One of things everyone involved on the project appreciates is how working in Unreal Engine enables them to quickly iterate and try out new ideas, even late on in the production.

“What was cool was being able to work concurrently rather than linearly,” says one of the show’s producers, Jill Gilbert. “Historically, you shoot and go back to beginning and redo, and we were constantly refining everything, working on a concurrent path; we were able to make creative changes till the last minute.”

Vivian even took a novel approach to script writing because of this flexibility. “The creative process evolved toward the end to where we weren’t even using a script,” she says. “I ended up just recording voice-over files in lieu of written dialogue and sending it over to Aaron. This is unheard of in traditional animation, which is so slow-moving—but working in Unreal, I could send over batches of voice-over files that could be inserted into rapid iterations of the film to explore different story avenues.”

Unreal Engine 5 Early Access promises new potential for filmmakers

Working in Unreal Engine 5 was not a new experience for the team at ASC, despite the fact that it’s still in Early Access and not considered production-ready. They were heavily involved in the Valley of the Ancient sample project that accompanied the launch of the program, creating a giant 50-million-polygon character that they were able to animate entirely in engine, thanks to Nanite, UE5’s virtualized micropolygon geometry system.
Although Epic doesn’t recommend using Unreal Engine 5 in production while it’s still in Early Access, Aaron is taking it in his stride.

“It’s still being built and refined, so it’s changed, and even the version I’m working in now is different, but I’m very comfortable with the interface, which is different from UE4’s. I prefer the interface—the look, the navigation, and the tools,” he says.

Another new feature he’s enjoying is Lumen, Unreal Engine 5’s new dynamic global illumination and reflections system that eliminates the need for lightmap baking, as well as the new tools for animating in engine.
Image courtesy of Aaron Sims Creative
“Lighting is so important to me as a filmmaker, so Lumen has become a big part of my workflow,” he says. “I’ve been pushing my team at ASC to focus on animating in engine for some time; there were amazing things we could do in UE4, but we could really take advantage of UE5’s new tools and Control Rig with this project.”

The future of filmmaking with Unreal Engine

Aaron shows no sign of stopping on his creative journey with Unreal Engine. “The most exciting thing about Unreal for me as a filmmaker is that, as one person, I can create my film and tell my story in real time,” he says. “This has never been possible to the same level of detail and creative freedom till now. I feel this is just the beginning for filmmakers and as we see the software evolve and educational resources increase, this will only make it easier for anyone to tell any story they desire.”
Image courtesy of Aaron Sims Creative
Aaron’s career in the film industry spans more than three decades, and he’s seen many changes over that time. “We’ve come all the way from practical effects, makeup effects, puppets, and animatronics, through VFX, to now, where I feel Unreal has been bringing back the real-time on-set experience to filmmakers that I feel has been missing with the age of green- and blue-screen VFX—the ‘fix it in post’ mentality,” he says. 

The experienced veteran is still dreaming up new ways to use the technology to enhance the filmmaking process.

“With the use of on-set LED screens and Unreal Engine to create set extensions, filmmakers can bring the cast and crew anywhere in the world and beyond, and that has changed everything,” he continues. “Because I have a more practical makeup effects background, I feel the next step is bringing CG creatures onto the set on an LED screen, so that actors and filmmakers can actually see and work with them.

“Before we did THE EYE, we were about to test this process on a live-action short. The timing and some other aspects did not align for that project, but it is something that I know will be the next thing someone will do, and this will change creature-based VFX.” 

In the meantime, Aaron has no shortage of new ideas up his sleeve to realize in Unreal Engine. 

“After my experience with THE EYE, I have personally fallen in love with creating my stories entirely in engine, and I really enjoy that process,” he says. “I have so many stories I can’t wait to tell in this medium.”

    Want to try Unreal Engine 5 Early Access?

    While UE5 is expected to be production-ready in early 2022, you can download Early Access now to explore the new features, along with the nine-foot alien Teuthisan, complete with his Control Rig.