Gnomes & Goblins
September 12, 2016

Immersion and Interaction Merge in Gnomes & Goblins

By Daniel Kayser

For many, the joy of fantasy lies in the exploration of their own imagination. To escape their present reality and connect with a story, a set of characters or a particular cause that resonates with their individual ideals. But what if those characters connected back? What if those fantasy worlds felt more like memories we actually experienced? What if we truly interacted with our own imaginations?

With the advent of virtual reality comes the opportunity for creative story tellers to immerse their audience in all-new, richly-crafted worlds like never before. These worlds and the creatures that inhabit them combine the best elements of linear story telling with real-time interactivity to deliver something truly unique.

One such VR experience aims to ignite interest in the possibilities that the medium provides. Entitled Gnomes & Goblins, the recently-released interactive experience isn't quite a game, but it isn't exactly a movie either. Discovering what it is just might be completely up to each individual who experiences it, which turns out to be the creator's intent.

To find out more about Gnomes & Goblins, which was developed using Unreal Eninge 4, I spoke to Wevr Creative Director Jake Rowell to get some insight into the (unlikely) origin of the project, its ambitous creative goals and what fans might expect in the future from this potential-packed franchise.

Where did the original concept for Gnomes & Goblins come from?

JR: t is an interesting set of events to be honest. The whole thing began over a year ago with an unexpected trip (and visitor) to the Wevr office. Jon Favreau was working with animation director Andy Jones on the film The Jungle Book when Andy informed him that he was heading over to Wevr to check out a piece I directed, called theBlu: Whale Encounter. Interested in VR, Jon asked if he could tag along and check it out. What we did with theBlu seemed to really resonate with him, because the next morning he woke up at 4am and started drawing images of the characters and worlds he wanted to explore in VR. Shortly after, a conversation started between Jon and Wevr about collaborating on a project together. Less than a month later, I signed on as Jon’s creative director to help bring his vision of Gnomes & Goblins to life. 

Please tell our audience what they will actually experience in this preview of Gnomes & Goblins.

JR: One of the first things the audience will notice is an immersive, comfortable VR environment.  Jon wanted the goblins’ forest to be a familiar place, like something from a dream where your imagination is invited to experience the world without fear and discomfort.  We spent a lot of time discussing the ideas of lucid dreaming, enchanted moments, having a storybook feel, and how can we capture these moments in VR.

The scale of the forest, the leaves blowing just the right way with the right sound, all help to create the correct mood for the player. It’s a heightened sense of things, enough to feel like you’re in a dream.

Another area of focus is the ability to connect with a character in a meaningful way in the VR space.  Just like with people, the first level of connection with our character are the eyes.  The base foundation for our AI is eye contact and eye tracking to the headset and controllers – so whenever he’s in your space, he’s tracking your head and hands.  If you pick up an object, he sees it and it has meaning. If you move toward him, he will move back.  If you move back, he might move forward if he trusts you.  All the while, his eyes are connected to your head and hands.  We wanted to create the beginning of a relationship, the start of a personal attachment and someone you want to go back and visit.   

Establishing a personal relationship with the character found in this preview is obviously an essential part of the experience. From an AI perspective, how difficult was it to achieve the level of believability that you managed to incorporate into this preview?

JR: This was a team effort across many areas of development. Andy Jones (Animation Director), Mauricio Hoffman (Lead Animator) and Chris Christensen (Lead A.I. Engineer) spent many iterations of work on the animations and the way they are layered together. They also spent many cycles on the code that is driving all of those animations and how the character reacts to the player’s movement. John Bernhelm (Lead Designer) also played a pivotal role in this development by crafting different emotional states so you can see it subtly go from nervous to happy or sad based on your actions. For example, we wanted to reward a thoughtful approach from the player, so the character gets more nervous if you're standing up tall or moving too quickly. All of these aspects required careful tuning and many play tests to find the right balance in the VR space. 

The setting of Gnomes & Goblins is both beautiful and immersive. Did you land on the art style for the project right away?

JR: Jon and I spent a lot of time talking about his vision for Gnomes & Goblins and what he gravitated toward visually. With film, animation, games, and fine art as inspiration, we explored connections that could offer up hints as to the mental image of his world. Traditional Disney films and classic Nintendo games kept coming up as references that helped point us in the right direction early on. We also spent some time talking about black and white films and how DP’s would light various sets to achieve a specific look or mood for the story. 

From there, I took to the concept phase and started presenting Jon various visual ideas to see if we could narrow in on a visual language that we all could get behind. He also did these great charcoal sketches of the gnomes and goblins that we kept going back to for a source of inspiration. After multiple iterations and discussions, we presented an idea that was a mix of classic Disney animation in terms of color palette with the look and feel of claymation with respect to lighting and materials. The combination of the two helped define the look of our world and the goblins that live there.

Once we had our visual language in place and some key concepts approved, it was time to start working on the CG aspect of the product. This was a bit of a difficult road since the majority of interactive products stem from a more hard/opaque surface look. Therefore it was a challenge getting the supple nature of the tones correct and balanced while running at 90 FPS. Our Lead Artist, Nghia Lam, started building an über material that was based on a claymation foundation, and supplemented it with features that we would need such as metal, moss, Fresnel Effect, etc. Our Graphics Programmer, Julian Kantor, did a tremendous job on his custom material functions that ran through the levels. These helped drive the look of the lighting and interaction through properties in the levels so we could set parameters separately between the night and day scenes. It really was a collaborative experience for everyone on the team as we shaped this world together. 

It’s pretty clear from the announcement that this preview is just scratching the surface of what’s to come. What can you tell us about the additional realms and denizens we might experience via future content for Gnomes & Goblins?

JR: Jon has a pretty detailed and expansive vision for this VR world. Unfortunately, I can’t say too much more about future location and denizens beyond the title. We are being introduced to the Goblins in this preview, so it is safe to say we will be introduced to the Gnomes and their realm in the future. 

The Gnomes & Goblins preview is releasing for free. Why was it important to pursue this route and what type of audience are you hoping to attract?

JR: The Gnomes & Goblins preview being released for free was something that Jon, Wevr and the entire development team felt passionate about. The VR medium is just getting started and it is important to try and help move the medium forward one step at a time. If theBlu: Whale Encounter (also free) served as a great introduction to VR, it is our hope that the Gnomes & Goblins preview hints at the potential future of what VR has to offer. 

How has Unreal Engine 4 helped bring the creative vision for Gnomes & Goblins to life?

JR: Early in the project I spoke with our Lead Artist, Nghia Lam, and Lead A.I Engineer, Chris Christensen, about the challenges we faced with VR and room-scale development. After much discussion about the various options we could use, the biggest contributing factor in choosing Unreal was the immense suite of tools and the power to edit continually throughout the production. It gave us the ability to pivot as things evolved during production while improving the look of the entire product over time. 

Like many productions, we started with just an empty level and a lot of test maps built on individual features. We slowly integrated them into the main scene as features came online. The Blueprint system was also a big advantage, in that it let us build and test features/systems quickly and we were able to integrate in a way that was fast and flexible.  The Swarm tools were also beneficial in being able to cut the render times of light-maps down to a fraction of the time it would take on a single processor. It is important to note that Nghia ran point on a lot of these tasks and did an amazing job leading our visual effort with his knowledge and experience with Unreal. 

Are there any aspects of Unreal Engine that were particularly useful to you and the team throughout development?

JR: Absolutely, we had many areas that Unreal helped the vision and quality of the final product. To help express this, I reached out to some of the crew and they each offered up some in depth perspective on Unreal and the various use cases they enjoyed during production.

Chris Christensen (Lead A.I Engineer): 
I found the animation Blueprint system to be very nice because of how easy it is to iterate on how the animations are layered. Also the preview mode for animation Blueprints is great for debugging them.

Nghia Lam (Lead Artist): 
My favorite feature was the material editor and the instancing process that was built around it.  Because of that I was able to write new features and push to the rest of the team without it disrupting the workflow.  It just magically spawned new features for everyone weekly.  At a traditional studio, you'd have to put in a tools request and wait in line for the code team to look at your feature and make a call on whether it would fit their planned engine development. In Unreal, I would just do it with just a few people and cut a process that would normally take months down to days or even hours.
Jon Bernhelm (Lead Designer):
On design side, we benefited greatly by the tight coupling between Blueprint and C++ code. Often, we'd prototype with Blueprints and then re-implement in code, or Chris would develop a C++ class and expose a lot of parameters and functions to Blueprint. As a designer, it is amazing to be empowered with Blueprint to extend and build upon C++ classes. I don't have to go bug a programmer when I just need one more variable to make something work! Chris built an impressive Animation Blueprint for our character with all of the knobs and variables exposed, which let us show the animators exactly what the character would look like with various layered animations turned on. We used this functionality to build an "emotion gallery" where differently-tuned versions of our character were placed side by side so we could easily test how our layered animation sets were reading for it's various emotional states. We also benefited greatly from the easy-to-use built in spline system. Our interactive bridges are spline meshes, which we let us easily tweak their placement and length without rebuilding, and we utilized splines to control much of our character's movement.

Julian Kantor (Visual Engineer/Animator): 
The material system had several features that I found particularly useful. I created several material functions, which enabled me to re-use modular pieces of logic in multiple materials. I found that material parameter collections, which store global variables that can be accessed from any material, were also helpful in several situations. When used in combination, these features allow some pretty powerful behavior - for example, with a single Blueprint node we can set "wind intensity," which in turn feeds into multiple material function calls that are embedded in a variety of materials, including grass, leaves, branches and candle flames.

It is evident that Gnomes & Goblins isn’t exactly a movie or a video game by either of their traditional definitions. What type of experience do you consider it to be?

JR: This is something new that VR is offering the content creators a chance to discover and define. You are the star, and the experience is your exploration of the world and interaction with its inhabitants. Personally, I consider it more like a memory of an experience you lived rather than something you played or watched. It isn’t a passive cinematic experience where you sit and enjoy the ride, but it isn’t a game where you need a certain skill set to advance. You do have agency and exist in this world, but the goals are not explicitly stated and everyone has freedom to drive their own story. John Bernhelm (Lead Designer) would often say that there isn’t a right or wrong way to “play”, but there is a clear cause and effect element to our world. 

Considering the scope of your career, which spans a variety of roles across high-profile gaming and film franchises, what excites you the most about VR and the potential experiences it can provide?

JR: In my opinion, VR takes the best parts of feature animation and visual storytelling and combines it with the best parts of interactive development and world building in a real-time environment. It is important to have a good understanding of both to push the boundaries of the medium and find new ways to engage the audience. With theBlu and Gnomes & Goblins, scale (large and small) and room-scale presence are huge factors in what we are trying to explore. This is year one of VR development and we are just scratching the surface of what is possible in the medium. I’m extremely excited about its future and the potential of AR and what it can do to entertainment and everyday life as a whole. 

Thanks for your time, Jake! How can people experience Gnomes & Goblins for themselves and where should they go to learn more about the project?

JR: My pleasure – I’m happy to talk about our process and share some of the experiences we faced while making Gnomes & Goblins. For more info, please visit the Wevr website or the Transport website. You can also visit Steam as well as Viveport for more details – Thank you.