Image courtesy of Warner Bros.
Accio Unreal Engine: The making of New York’s Harry Potter VR experiences
April 12, 2022
Imagine waking up to a letter from Hogwarts. Soon, you’ll be swapping your normal life for a world where you’ll cast spells, ride brooms, and meet magical creatures galore.
It’s an exciting thought. The only problem? There hasn’t been a wand in the world that could actually cast a spell or broom that would actually help you fly. Until now.
At Harry Potter store New York, anyone can feel what it’s like to wield their own magic firsthand thanks to two virtual reality (VR) experiences—no Hogwarts letter required.
It’s an exciting thought. The only problem? There hasn’t been a wand in the world that could actually cast a spell or broom that would actually help you fly. Until now.
At Harry Potter store New York, anyone can feel what it’s like to wield their own magic firsthand thanks to two virtual reality (VR) experiences—no Hogwarts letter required.
The spark of a great idea
For the team at Wevr, the studio commissioned to produce these experiences, the journey began when their previous VR projects—which had been featured at the Sundance and Tribeca film festivals—caught the attention of Warner Bros.“Warner Bros. asked us to help with the creative development of a number of VR initiatives,” begins Luis Blackaller, Wevr Creative Director/Co-Director of the Harry Potter experiences. “Our CEO, Neville Spiteri, decided to build an interactive demo, which immediately won over the studio. That led us to develop and produce these two new interactive VR stories for the Harry Potter franchise. Our goal was to create a doorway into the Wizarding World of Harry Potter, where audiences could transform into Hogwarts students and try magic for themselves.”
Image courtesy of Warner Bros.
Both interactive experiences are extensively based on the books and films, with impressive details that will make every minute feel authentic and true to the original canon, rather than added as an afterthought. For the first experience, which is called Chaos at Hogwarts, you start by waiting in a replica of King’s Cross station. After selecting a character avatar and suiting up with VR gear (including a voice-activated wand with a signature vibration for every spell), it’s time to transition from the King’s Cross replica into a virtual Platform 9 and 3/4s to chase down and miss the Hogwarts Express. Luckily, Dobby apparates in to shepherd you to Hogwarts, where you can explore the castle and cast spells while facing everything from pixies to a dragon. Best of all, the experience has multiple endings: Whether you defeat the creatures will depend on how good your group is at casting spells.
Meanwhile, for Wizards Take Flight, participants are given brooms that they can control by leaning left or right, much like the students in the books and films. In this experience, you help deliver a package for Hagrid by traveling through London and defending him against Death Eaters—experiencing the VR all while the physical stage mimics London’s weather by sprinkling rain or blowing wind that’s perfectly timed with the virtual world.
Meanwhile, for Wizards Take Flight, participants are given brooms that they can control by leaning left or right, much like the students in the books and films. In this experience, you help deliver a package for Hagrid by traveling through London and defending him against Death Eaters—experiencing the VR all while the physical stage mimics London’s weather by sprinkling rain or blowing wind that’s perfectly timed with the virtual world.
Image courtesy of Warner Bros.
“We needed to achieve the right balance between the integrity of the details, fan wish fulfillment, and an amazing interactive story,” adds Anthony Batt, Wevr Co-Founder/Co-Director of the Harry Potter experiences. “On the visual end of things, we treated the films as canon. We recreated most digital assets from scratch, including a section of London around the Thames River; environments like the Grand Staircase with talking paintings; and creatures like Death Eaters, Dobby, and Hagrid. We know fans are extremely familiar with these characters, and we wanted them to feel as lifelike as possible. In the end, we even created a dialog database for every line ever said by Dobby or Hagrid, to make sure they stayed in character.”
Image courtesy of Warner Bros.
A pinch of Unreal Engine
In order to create such extensively detailed characters together with interactive stories that multiple people could experience in real time, Wevr used Unreal Engine from the start. “This project touched every aspect of the engine, and then some,” reveals Blackaller. He explains the team first used Unreal Engine to test how layout models of every set—which were based on detailed floor plans, photographs, and footage from the movies—would perform in VR.“Thanks to Unreal Engine’s flexible runtime environment, we were able to prototype ideas on elements like scale, then iterate quickly,” Blackaller continues. “Once most of the layout was ready, we then began adding realism, recreating every material and texture in-engine while optimizing for performance in our very demanding multiplayer VR setup. It was important that every set piece felt alive, and we accomplished this using a mix of interactive positional sound design, dynamic lighting and shadows, and other elements like clouds, fog, fire, and water.”
Image courtesy of Wevr
Animation on the project was also moved to Unreal Engine from the early development and layout stages. Every creative choice, whether it was a character’s Hogwarts robe fluttering in the wind or a roaring dragon, was backed by a VR walkthrough. This enabled the team to study tempo and action blocking for every scene, as well as make crucial adjustments in real time on stage.
“When you have an interactive narrative experience where six people are each given freedom to roam around at will, and you have to get them through the experience in about ten minutes, it becomes extremely important that you’re able to explore many variations of a sequence to deliver the most effective use of the stage from the perspective of all six players,” adds Blackaller.
“To do this, we developed our own live motion capture system using Unreal Engine’s powerful Live Link features and began bringing in actors to run capture sessions using Xsens and Rokoko for the body and iPhones for the face. Once a take was locked, our animation team would then embellish the performances. The end result effectively combines cinematic-quality character performances with interactive features, so characters make realistic eye contact with every human player in the room, no matter where they are.”
“When you have an interactive narrative experience where six people are each given freedom to roam around at will, and you have to get them through the experience in about ten minutes, it becomes extremely important that you’re able to explore many variations of a sequence to deliver the most effective use of the stage from the perspective of all six players,” adds Blackaller.
“To do this, we developed our own live motion capture system using Unreal Engine’s powerful Live Link features and began bringing in actors to run capture sessions using Xsens and Rokoko for the body and iPhones for the face. Once a take was locked, our animation team would then embellish the performances. The end result effectively combines cinematic-quality character performances with interactive features, so characters make realistic eye contact with every human player in the room, no matter where they are.”
Image courtesy of Warner Bros.
A dash of interactivity
In addition to all the art and animation, Unreal Engine also enabled the Wevr team to develop a series of interactive systems for both the VR experiences, including a spell-casting system, a broom flight simulator, and a comprehensive haptic feedback system made in collaboration with Dreamscape Immersive.“A particular challenge in multiplayer VR is ensuring players and physical objects are in sync with their virtual counterparts. Each system needs to remain on-frame at every moment. Otherwise, players would experience an object in a virtual location when it would turn out to be somewhere else in the physical world,” Batt continues. In order to maintain the illusion, real-time choreography would be needed. This would keep participants fully immersed in the story while channeling their movements in a way that didn’t feel artificial.
For the flight simulator in Wizards Take Flight, the team built a guided flight system where participants were constrained around a tridimensional track. There was also an intelligent battle system that kept spawning Death Eaters and having them attack players with different levels of intensity based on their flight and combat ability. In other words, the more active a player was, the more intense the battle would get.
Image courtesy of Warner Bros.
For the Chaos at Hogwarts experience, the team accomplished a similar illusion with a walking redirect system. In real life, players were wandering in circles around a small square stage. Meanwhile, in VR, Unreal Engine orchestrated a choreography that seamlessly transformed the castle around the players as they moved from room to room, hiding and loading new sections of the castle on the fly.
“Whether it was for our realistic character animation, cloth simulation, or a flight system; pretty much everything we did in the project benefited directly from an Unreal Engine feature,” says Blackaller. “We believe virtual reality and other screen-based interactive media powered by Unreal Engine are going to transform everything from theme parks to retail, enabling audiences to have meaningful interactions with the stories, worlds, and characters they love rather than watching passively. We are excited to be at the forefront of this movement, and we are eager to push the bar even higher.”
Wizards Take Flight and Chaos at Hogwarts are now playing at Harry Potter New York. To learn more about the production of these amazing experiences, visit Wevr’s Harry Potter page.
“Whether it was for our realistic character animation, cloth simulation, or a flight system; pretty much everything we did in the project benefited directly from an Unreal Engine feature,” says Blackaller. “We believe virtual reality and other screen-based interactive media powered by Unreal Engine are going to transform everything from theme parks to retail, enabling audiences to have meaningful interactions with the stories, worlds, and characters they love rather than watching passively. We are excited to be at the forefront of this movement, and we are eager to push the bar even higher.”
Wizards Take Flight and Chaos at Hogwarts are now playing at Harry Potter New York. To learn more about the production of these amazing experiences, visit Wevr’s Harry Potter page.
Get Unreal Engine today!
Get the world’s most open and advanced creation tool.
With every feature and full source code access included, Unreal Engine comes fully loaded out of the box.
With every feature and full source code access included, Unreal Engine comes fully loaded out of the box.