Image courtesy of ROTU Entertainment

Rhythm of the Universe blends music-based gameplay and an important message to create an epic VR series

By Epic Games Content Producer Jimmy Thang |
November 13, 2020

ROTU Entertainment is an immersive-entertainment studio based in Boston, Massachusetts, where art and technology meet in harmony. Our ever-evolving network of artists, entrepreneurs, and industry leaders, based worldwide, provides us with unparalleled access to talent, creativity, and innovation. Inspiration is the key to our success, and collaboration is our lifestyle. Learn more at www.ROTU.com.
Comprised of many Berklee College of Music students, ROTU Entertainment has produced music videos, documentaries, and collaborative live shows that highlight real-world issues. For the company’s first game, it’s developing Rhythm of the Universe, a seven-part VR fantasy-adventure series that emphasizes a message about respecting the planet. The first title in the saga, Ionia, releases later this year and focuses on habitat conservation with a percentage of the game’s sales going to the Wildlife Warriors organization. 

With team members who worked at Valve and Capcom, ROTU Entertainment believes that VR can be a tool that’s used to raise awareness, particularly around climate change, in their case. With Rhythm of the Universe, the developer will incorporate real music theory into the gameplay. All of this is wrapped up in a gorgeous world with top-notch VR graphics. To see how ROTU Entertainment is pulling this off, we interviewed Game Director Emir Cerman, Art Director Charles Logan, Senior Producer Jason Parks, Technical Director Jamie Gibson, and Technical Director Michael Hoag.
 

Where did the idea to make a music-based adventure game stem from?

Game Director Emir Cerman:
Most of the ROTU team are Berklee College of Music graduates; our entire lives have been heavily influenced by the power of music. After working with thousands of musicians from over 100 countries, it was clear music is humanity's universal language. When we set out to create a new fantasy world that could connect with people from all walks of life, we knew that our unique music-mythology concept would need to be woven into every game element. We believe that everything in the universe has a rhythm, and that comes to life [in the game] as music-inspired creatures and ecosystems, rich cultures, and powerful music-based magic we call Modus, and so much more. We're excited to bring this world to life as an immersive story living experience that’s only possible in VR.

Did Rhythm of the Universe have any influences from other games or other works of art?

Art Director Charles Logan:
There have been so many franchises out there that we've grown up loving. It is hard to deny the impact stories like Star Wars, Harry Potter, the Uncharted series, or the art of Roger Dean had on us. It has been an incredible, awe-inspiring journey to be able to put our first foot forward on a similar platform. We wanted to create a world of our own. In the end, it would very likely not be possible to conceive of Rhythm of the Universe without the inspiration we've garnered growing up with these fantastic works of art.
Image courtesy of ROTU Entertainment
One of the game's central themes revolves around wildlife conservation, with a percentage of the game's proceeds going to nonprofit group Wildlife Warriors to combat climate change. Can you elaborate on the significance of this initiative?

Senior Producer Jason Parks:
ROTU creates games that foster empathy for environmental and social issues. Early studies show that VR has a powerful conversion rate when it comes to taking action. We want that action to be in support of global nonprofits. That is why every game under the umbrella of the Rhythm of the Universe franchise will tie directly to a reputable nonprofit. For Ionia, humanity has wiped out an inconceivable percentage of animal life since 1970. At ROTU, we don't want our grandchildren thinking of rhinoceroses as we do dinosaurs. We will be donating over two percent of every copy of our VR game sold to the Wildlife Warriors Foundation. ROTU is finalizing a public partnership that will showcase Bindi and Rob Irwin, the late children of Steve Irwin, who founded Wildlife Warriors and are some of the most popular conservationists in the world. We sent Wildlife Warriors a VR headset and a copy of Ionia. They loved the game. 

For perspective, Activision recently featured Rob Irwin in Crash Bandicoot’s marketing campaign. We are excited to be associated with such well-known influencers, but the ROTU team chose Wildlife Warriors because 100% of its proceeds goes to protecting wildlife! The Australian zoo covers the overhead for Wildlife Warriors, so ROTU knows that every copy of our game will directly support wildlife as it should. I am ecstatic about this partnership because I grew up watching Steve Irwin and gained much of my love of nature from him. Having his organization support our game is a dream come true. In terms of nonprofit support, Wildlife Warriors is just the start. We are planning seven games that each takes place in our lore on the supercontinent of Pangea. Each land's storyline and message aligns with a social or environmental issue. 

Vividly brought to life in VR, Rhythm of the Universe: Ionia features amazing graphics with extremely detailed environments coupled with immersive lighting. How did the team deliver here?

Technical Director Jamie Gibson:
The vast majority of the game's lighting is based on a variable density set of lightmaps built with UE’s Lightmass. Objects along the player's path, which will be viewed from up close, get denser lightmap textures. We leveraged features such as specific emissive contributions to make our glowing plants light the room when areas got too dark. We manually tuned all of the light bounce strengths on just about every surface before finding the depth was starting to pop. On top of that, we also have liberal use of cheap dynamic lights. These are movable lights that do not cast shadows and tend to have a minimal attenuation radius. These can help fill in dark areas or be used as direct light sources for torches or interactive objects and plants. Another crucial element we implemented was heavy use of post-process and varying degrees of height-based fog, which changes every time you encounter a new scene. The options available to designers within post-process and fog go way beyond just how the game is lit and can make the difference between photo-realistic, cinematic, or stylized [scenes].

To get the detailed environments, we've had to implore a whole suite of optimization and performance-saving operations, but first, we must address the biggest hurdle. Ionia is actually a rather large area, with each area having a lot of detail. It was apparent, very early on, that we couldn't manage all of that on the screen at one time, so we opted to use map streaming. As a VR comfort feature, we already darken the headset whenever the player teleports, so we use these moments when the world is entirely black to stream in and out of various map areas. This way, players never see themselves standing over an empty abyss as things all pop into view, and they're only processing what they're actually seeing.

For the optimization of assets, we've had to keep our shader complexity within a reasonable tolerance. We had to void overdraw, where too many translucent or masked surfaces overlap with large areas of see-through pixels. We've implemented UE's Hierarchal LOD system to combine parts of the map comprised of many pieces into single assets when viewed from afar, often baking all of their materials down into one atlas. We merged meshes and adjusted the LOD distances on all of them, and leveraged distance-based culling to efficiently have things remove themselves when simply out of visible range. The nice thing about this is that it's faster to set up and more natural for the engine to cull objects this way than using level streaming, so the two in combination are saving us a considerable amount of overhead both down in the trenches and across the map as a whole.
Image courtesy of ROTU Entertainment
The assets in Rhythm of the Universe: Ionia are highly detailed. Did the studio leverage any Quixel Megascans for the game?

Cerman:
Quixel Megascans were extremely beneficial for level-design previsualization. Once we had our vision locked, we decided which elements to create from scratch and adapt and augment our world. We used individual meshes like rocks and cliffs, etc., as base meshes and modified those to meet our needs. As an independent studio, access to this library helped push our creativity and capabilities. We were able to blend textures with our custom shaders to create depth that enhanced our world.

Rhythm of the Universe features a world full of beautiful ancient architecture and interactive plant life. Did the team look at real-world cultures and environments to design the game's levels?

Logan: We spent quite a lot of time during R&D working with our concept artists from all around the world. For the temples in Ionia, we've looked into the architecture of the Aztecs and Mayans and deconstructed how their artwork had evolved over the years. While creating this brand new mythology, we asked questions such as, "How would they live here?" and "What was the purpose of this?" to hone in on the authenticity of the world. It was vital for us to have a uniform musical motif inspired by countless instruments, mostly unheard of. We wanted to create a sense of a lived-in environment that has now been emptied and ruined as the players go on their journey.
Image courtesy of ROTU Entertainment
The gameplay involves solving music puzzles, foraging materials, climbing around the rocky terrain, and more. How did the team come up with and iterate on Rhythm of the Universe's gameplay loop?

Technical Director Michael Hoag:
It is an impossible challenge to sit down and develop a game from start to finish as it appears in its final form, especially when frequent scope changes and rewrites are going on from the team's creative side. So, building upon lessons learned in previous development efforts, almost every game loop function is developed in isolation on its own test map, which we internally call gyms. For example, we have a teleportation gym where all the systems for creating the arcs and detecting new playspaces and blocking arcs on hidden geometry, is figured out. We have a climbing gym where we practice placing grips and making climbable vine splines, which allow us to learn which situations should be avoided. We have an audio gym where we develop footstep cues for different ground materials and see how sound occlusion and other effects play out. Similarly, each of the puzzles and instruments first gets introduced to the team in its own unique gym. Frequently, the technical team is asked by the art team something like, "Hey, can we put the player on a zipline and fly down through a forest canopy in VR?" Then we panic and say, "Absolutely not. That's crazy talk." Then curiosity takes over, and we think, okay, well, let's try it in a gym first and prove that it can't be done. Eventually, we find out that nothing is truly impossible in UE4. In any case, building up a library of small testing environments separate from all the eye and ear candy is a critical first step in our process.
Eventually, we find out that nothing is truly impossible in UE4. "
- Technical Director Michael Hoag
The next step is grey boxing out the levels, literally with primitive shapes and checkerboard textures. Before the COVID pandemic hit, we considered what our game would look like as an arcade experience. And with grey-boxed levels, we started to plan out the flow of sections based on a set amount of time through key segments. We knew we wanted to have a lot of vertical travel on open-faced elevators and ziplines, etc. This is because it feels like you're traveling further than you are in VR, and because it makes it a lot easier to stream sections in and out of view when the location you came from is not laterally adjacent to you. Making adjustments so that these moves aren’t too long for the dialog to fill or too fast that it creates motion sickness is far easier to iterate on within a grey box than a polished level. The minute you start placing rocks, plants, or other decoration around, you start burning unnecessary hours once a change comes in. It's at this point we begin to layer in the required game loops, making sure we don't introduce several new features to a player in one spot.

Before the final decorating process, we put a lot of thought into what can be seen from any area the player is confined to. It's likely very similar to how things are done in film, as we aim to create multiple "sets" consisting of interesting assets up close and faking everything further away as much as possible. We started with billboards, simple planes textured with clips from our concept art showing a vista seen from here, a swamp goes there, and even what creatures and plants should be found nearby. Breaking up sightlines is critical, as one trick we use frequently is setting occlusion distances on heavy assets to disappear entirely once you're a certain number of meters away from them. Not being able to see that far ahead adds a bit of mystery your first time through. Not being able to see that far behind you as you ride along a twisty underground river is just as important for keeping performance in check. It's only in the final stages that we start to replace temporary assets with better-looking ones, fitting game loops into place along with our AI systems, and dealing with lighting, fog, and post-processing last.
Image courtesy of ROTU Entertainment
The studio has previously stated that the series aims to gamify music theory. Can you explain how you're planning to achieve this?

Parks:
Every piece of content created by ROTU utilizes humanity's universal language, which is music. We do it in a way no one else has done before, through the gamification of music theory. A concept defined by the experiences of thousands of international and indigenous musicians working together over the past ten years. We call it Music Mythology in our games. 

The key is to gamify learning in a way that does not feel like learning. We make sure that everything stays fun and intuitive. In Ionia alone, users learn and perform real-world music theory, including arpeggiated scales, modes, and intervals performed on a Pangean-inspired Hang drum and Marimba. Another example, at the end of Ionia, users will repeatedly perform a 4/4 conducting pattern to progress. The interactive environment is also inspired by orchestral instruments from the bell-shaped nostrils of the Brassasurus to the castanet plants that follow, chatter, and try to bite users. Clefs and other music theory symbols made it into Ionia, inspiring architecture, tools, and clues to Ionia's mysterious past. All of these passively familiarizes users with common musical elements and instruments.

Each game in the Rhythm of the Universe VR series will have completely different music theory inspired challenges that collectively provide many of the skills taught at the Berklee College of Music. Many of our core team members graduated from Berklee, so it is not surprising why our shared love for the universal language made it into our games.

After launch, we plan to pursue research studies to identify our games' retentive learning and empathetic value. Being based in Boston, we have access to top quality medical universities to pursue such research.

Rhythm of the Universe features many inventive, often larger-than-life creatures themed after musical instruments. How did the team approach designing them? 

Logan:
The core trick was coming up with something that resonated with the world's musical theme and created beings that sprout to life around the core idea of rhythm or vibration, which is this magical frequency that envelopes the universe. Many creatures have some connection to the rhythm, and the designs needed to reflect that through a visual motif of musical undertone. A lot of the design process started from rough sketches, jumping to paintovers, and renders visualize what these creatures essentially looked like and how they behaved. We've resorted to jumping to 3D in the concept phase as early as possible to "fail early" on many designs to make sure proportions, and everything worked in our favor. The hardest parts were perhaps the anatomies of such beings, which required us to break down their skeletons and construct their musculature over this shell, providing believable movement, and more. We've made quite a few iterations and even left a whole bunch of them on the cutting room floor at the moment, but who knows? Perhaps we will see them in the future!

 
Image courtesy of ROTU Entertainment
Rhythm of the Universe: Ionia, which is set to come out later this year, is the first game in a planned seven-part series. What can you tell us about subsequent titles?

Parks:
ROTU Entertainment has plans for seven VR games based on lands inspired by modes of music. Each game takes place in our lore on the supercontinent of Pangea. Each land's storyline and message aligns with a social or environmental message like marine conservation, air preservation, climate change, and more. Each game will have dramatically different gameplay and storylines tied together through the lore but hold up as standalone games. 

Our second game, Eolia, is already in development. Broken into three chapters, Eolia will challenge the player to master new musically-inspired Modus powers, discover new characters, and learn how to attune to the Great Harmony, all the while facing the ongoing threat of climate change head-on. In the world of Pangea, there are forces of harmony and forces of discord. Without balance, the world struggles to survive. In this game, Eolia's land is on the brink of extreme climate change due to this imbalance. Droughts, severe weather, and water riots are growing threats the player will face in this ongoing VR series. Should you choose harmony over discord, you will be paired with an animal companion to help you on your journey to bring all seven lands of Pangea into harmony. If you fail to make real change, discord will take over, leaving despair in its wake. 

Why was Unreal Engine a good fit for Rhythm of the Universe?

Hoag:
Roughly half the development team had already been working with UDK for many years, long before Unreal Engine 4 had released. The other half had never worked with a game engine before. Given that nobody on the team knows how to write code in C++ or C# with confidence, Blueprints visual scripting made UE4 an obvious choice over any other offering out there, especially having worked with Kismet and with similar pipelines for mesh importing and material shaders.

Starting directly with version 4.1 on paid subscriptions, we've had a front-row seat to each new innovation added to UE4 over time. That has allowed us to learn the systems deeply and be able to carefully consider how best to use each new feature as UE4 has evolved. We were also early adopters of VR, starting with the old DK1 units. We now own multiple types of headsets to make sure we get the right feel in each, as we customize the experience for each different hand controller. It's essential to get the proper grip and wrist alignment when holding something like a xylophone mallet, for example.

Our first technical demo in VR was developed between 4.14 and 4.16, and our final game will be packaged in 4.25. Given how supremely powerful and impressive the Animation Blueprints and the Niagara particle systems are, finding a better fit than UE4 for our team would be inconceivable. Given we are planning to release multiple experiences, the only open question will be finding the right moment to migrate over to UE5.
Given how supremely powerful and impressive the Animation Blueprints and the Niagara particle systems are, finding a better fit than UE4 for our team would be inconceivable. "
- Technical Director Michael Hoag
Image courtesy of ROTU Entertainment
With Rhythm of the Universe: Ionia being the studio's first VR game, what have you learned about developing for the medium thus far?

Parks:
Thus far, the most important lesson is defining and sticking to game-design principles that balance the playability and integrity of storytelling. After trying The Blu in 2015, I ripped off the newly acquired HTC Vive and yelled at Emir Cerman, our creative director, "This is what I am doing with the rest of my life!" Soon after, we started our VR game journey by finding incredible development and production talent within our network. Quickly, our efforts led to understanding our strengths and where we need to improve. Next, we researched every possible game within our niche, learning from the pros and cons of VR games that came before us. Quickly, our focus included game-design guidelines that considered locomotion comfort most important but did not restrict the player to a single location. We combined the benefits of free roam with the comfort of a stationary simulation. Next, we considered the sophistication of the average VR users. Since VR is so new, it is reasonable to assume that most users will not be familiar with the controllers and how they may be able to explore and interact with the world around them. The ROTU team decided to build a game anyone can play, making gameplay intuitive. For example, before COVID, we shared Ionia with a 101-year-old woman, who was able to play through our vertical slice herself! This is done by utilizing a few core game mechanics, but the context of those core mechanics keeps evolving throughout the story. These core mechanics are taught gradually and early in the game by sneaking them into a five-minute interactive tutorial that provides backstory. We purposefully used limited text hints and instead focused on the organic discovery of the core game mechanics. Lastly, our team developed an iterative design process that led to new ideas being easily implemented and tested, allowing for ideas that pop up in development to be considered for implementation into the game. This process led to some of the most iconic aspects of our game. 

In a nutshell, the ROTU team learned that VR has the potential to rewrite the future of entertainment. We attempt to combine the best aspects of traditional storytelling with the interactivity of games. We feel Ionia offers a glimpse into this exciting future.

Thanks for your time. Where can people learn more about Rhythm of the Universe?

Parks:
Thank you for the thoughtful questions. Make sure to wishlist Ionia on Steam and make sure you check us out on Twitter, Facebook, Discord, and our website

    Get Unreal Engine today!

    Get the world’s most open and advanced creation tool. 
    With every feature and full source code access included, Unreal Engine comes fully loaded out of the box. 
    Event
    Unreal Build: Virtual Production 2020
    Coming to you from the comfort of your own home on Tuesday, November 10, Unreal Build: Virtual Production is a free half-day virtual event that showcases incredible projects and innovations from film and television industry trailblazers.
    Event
    News
    Connect with the Unreal Engine community online
    Although many physical events around the world are currently on hold, there are plenty of places to connect with the Unreal Engine community online.
    News
    Interview
    11-person studio Neon Giant delivering next-gen passion project The Ascent
    Comprised of industry veterans who worked on the Wolfenstein, DOOM, and Gears of War franchises, Neon Giant is a small indie studio based in Sweden that’s developing a heavy hitter for Xbox Series X. 
    Interview