Unreal Engine 5 helps Aya Nakamura stun music fans in the Fortnite Soundwave Series

MetaHuman for realistic avatars

“When I heard about this project, I immediately realized what a breakthrough it would be,” says Louis de Castro, Founder, CEO, and Creative Director of MADO XR.

For De Castro, Aya Nakamura’s Soundwave Series performance perfectly encapsulates how the digital world is becoming a canvas for music artists. “I picture it as an enormous playground that boosts your imagination and also allows you to look at your performance from a different angle,” he explains. “Being part of a huge project that demonstrates all the power of the metaverse is like being a pioneer. What can be more exciting?”

The Aya Nakamura virtual experience represents a milestone both for Aya and for French music—a point not lost on Aya herself. “The collaboration is an opportunity to perform my biggest show ever to fans old and new and from all around the world,” she said before the experience. “I can’t wait to share my music with you all and be the first French artist to collaborate with Fortnite!”

Alain Veille, President of Warner Music France, explains that music experiences hosted in virtual worlds like Fortnite provide a unique opportunity for artists to meet fans where they are already spending their time. “Bringing artists to a game like Fortnite, which players love, is a way to get them to connect more directly with their fans who share a common interest,” he says. “ ‘I had no idea they were a Fortnite fan’ is a common positive refrain we hear from fans about their favorite artists.”

As well as providing new and exciting ways to connect with fans, virtual experiences can offer a far more practical way for busy music artists to reach millions of people at once. “We often have more flexibility to work around the artist’s schedule to get things done, compared to a live tour passing through many cities and venues,” says Veille.

The first step of the process was the scanning. The team took more than 150 pictures of Aya’s face and body to create a high-fidelity scan that was transferred into MetaHuman Creator, a process that is now accessible to anyone through the Mesh to MetaHuman feature of the new MetaHuman Plugin for Unreal Engine. “The most challenging part of the process was the details that make her human—adjusting the color of the lips with the skin tone and face shadows, so we could replicate that in MetaHuman Creator,” says De Castro. “The next step was to reassemble the facial rig to make her emotions appear. After that, we proceeded to animation through motion capture.”
 

As an early adopter, De Castro has seen how much digital human technology has evolved from the traditional pipeline in which you had to redo the facial rigging, animation, and bones to achieve a facial expression that would mimic a real human. “In that sense, MetaHuman Creator is truly a one-shot tool,” he says. 

The best litmus test of the team’s success is the reaction of the artist herself. “I was really surprised when I discovered my avatar,” says Aya. “It looks so real—especially the movements and gestures. I instantly recognized myself, it’s really impressive!”

The team at Atlas worked hard to recreate the look and feel of the CG visuals in the music video for the virtual world that the players would run through. With the digital Aya and UE5-built environments of the music video resulting in a more photorealistic look that you would ordinarily expect to see in Fortnite,  the team had to dive into Fortnite’s Creative toolset to achieve a good match. 

The future of music experiences 

Ruxandra says that Warner Music believes the future will bring hyperreality metaverse experiences overlaid on ultra-thin headset devices. “These metaverse experiences will be persistent, synchronous, and live,” she says. “They will span the digital and physical worlds, offer unprecedented interoperability, be populated by content from a wide range of contributors, and have their own fully functioning economies.”

Within these metaverse experiences, Ruxandra continues, music will be a dominant form of interaction, communication, and consumption. “The possibilities are endless for our artists to engage with their fans in immersive, real-time, and customizable ways,” she explains. “We are incredibly excited to dig in here and now, at the outset of these future experiences, so that we can ensure our artists have the tools needed to become experts within these spaces.”

For De Castro, Fortnite is the natural destination for virtual music experiences like the ones Ruxandra describes. “Let's face it, no one puts on a better virtual show than the Fortnite team,” he says. “There ain't no party like a Fortnite party, and this has helped Epic Games set itself up as a forward-thinking host when it comes to making the best in-game concerts. 

“The platform recommended itself to us as a number one destination for those who want to bring together gamers, music artists, artists in general, and designers. With the metaverse buzzing, it's the best example of how digital reality works.”