September 18, 2018
UE4 helps AnotherWorld VR bring cinematic realism to the KOBOLD VR Experience
Inspired by pagan mythology, ancient superstitions, and old Germanic fairy tales that don’t always have a happy ending, KOBOLD—named for the terrifying creature players ultimately encounter—is a spine-chilling transmedia experience that blurs the lines between cinema and VR gaming. The VR experience is accompanied by a short film that viewers can watch before stepping into the virtual world; the film introduces the characters they’ll meet, and provides the backstory of a missing boy and his family.
KOBOLD recently premiered at the 75th Venice International Film Festival - where it was nominated for the Best Virtual Reality Experience for Interactive Content—and is due to compete at several more prestigious festivals, including the Raindance International Film Festival in the UK, and the 47th Festival Du Nouveau Cinéma in Montréal. By immersing its players in a highly realistic and frightening world where they can freely move about, find and solve clues, and make choices which can affect the outcome of the narrative, KOBOLD aims to appeal to both gamers and to horror-movie fans willing to try out a new medium. Judging by the screams of its early adopters, it’s succeeding. “That was pretty spectacular,” says one of the first to try out the VR experience. “That last moment, when you turn around, and you observe that…that dark entity…that thing! It comes at you, and your body just reacts naturally.”
Berlin-based AnotherWorld VR developed the KOBOLD VR Experience over a period of one and a half years. The team of ten dedicated VR developers handled VR storytelling, programming, level and character design, photogrammetry, motion capture, facial animation, spatial audio design, and “tweaking the invisible magic that happens in the Unreal Engine,” as the company’s Creative Director Max Sacker tells us.
Cinematic Realism in VR
AnotherWorld VR specializes in creating VR experiences with an emphasis on storytelling, cinematic aesthetics, and immersive gaming interactivity, drawing on its background in filmmaking. For KOBOLD, the aim was to achieve cinematic realism in VR.“That’s a tall order for any game studio, not to mention an independent one,” says Sacker. He feels that in the gaming industry, the promise of ‘cinematic realism’ has often been broken, with the final product failing to live up to the pre-rendered trailers. Sacker and his team were determined not to disappoint with KOBOLD.
Early in the process, the team experimented with photogrammetry, and their tests confirmed their initial hopes that scanning real-world characters, locations, and objects could help create a feeling of true immersion. Using RealityCapture, they were able to quickly process large volumes of photographs and turn them into high-quality 3D assets.
To further add to the realism, the team employed Faceware - advanced, markerless facial-capture technology that enabled them to film facial gestures of real actors and characters with a video camera and transfer them onto their digital doubles with very high fidelity. “Reading faces is an intrinsic skill that we learn from the moment we are born, and continue to develop in our daily lives. The slightest twitch, smirk, or wrinkle can tell an entire story,” explains Sacker. “In virtual reality, as in real life, every detail counts, and the realism of facial animations is paramount to maintaining the immersion of the experience.”
To capture the actors’ performances, the team employed a Perception Neuron motion-capture suit. Unlike traditional systems, the Perception Neuron does not rely on specialized cameras to track motion; instead, it uses sensors about the size of a penny that can be easily fastened to the head, arms, torso, legs, feet, and individual fingers of the performer, and can send a data signal to a laptop via Wi-Fi - effectively giving you the freedom to set up a motion-capture studio anywhere you can get a Wi-Fi signal.
Putting it all together in Unreal Engine
AnotherWorld VR now needed to put all this real-world data together into an immersive interactive experience that retained the highly realistic qualities they’d worked so hard to capture. For the team, there was only one engine that could match the vision they were working towards. “We felt that the graphics, shaders, lights, particles, and cinematic tools in UE4 were of an incredibly high quality, and closely resembled the toolset we knew from moviemaking,” says Sacker.There’s one aspect of moviemaking that Lead Artist Tom Kis is glad is different in Unreal Engine: “Coming from movies where viewing a scene would mean waiting for hours - sometimes days or weeks - to see the final rendered result, the UE4 real-time previewer gives us a near-final look, which is a huge time-saver,” he says. “A big plus is that with every new engine version, we are able to push our visuals further and further.”
Both Lead Programmer Sandro Forster and Spatial Sound Designer Takuro Sakomoto are fans of Unreal Engine’s visual scripting system, Blueprint. “One of the features I value most in UE4 is the time-based flow control available in Blueprints,” says Forster. “[With it, I] can easily and very quickly write code that depends on time and blends in perfectly with scripted sequences.” Sakomoto, meanwhile, uses Blueprints to easily create different sound occlusion systems by manipulating the attenuation algorithm and audio volumes.
For Technical Artist Jasper Stutterheim, it was one of Unreal Engine’s less high-profile features that stood out: “Having editable curves in 3D space and many different ways to manipulate or shape them to your needs adds another level of possibility to designing content in UE4,” he explains. “This feature may not immediately seem special, but its versatility makes it very powerful for developers.”
Sacker is able to sum up the team’s experience in a nutshell - “Unreal enabled us to create visuals and interactions that we previously thought were impossible,” he says.
Interested in creating your own VR experiences? Download and explore Unreal Engine 4.20 today.