March 8, 2019

Blacksad breakdown: Howest student creates a playable 3D comic in UE4

By Nic Belliard

Hey! My name is Nic Belliard and I am a third-year graduate student at Howest (Kortrijk/Belgium) in the Digital Arts and Entertainment program majoring in Game Graphics Production.

While attending DAE, I worked on a variety of projects from illustrations to detailed environments and characters. Most of my work can be found on my Artstation portfolio: https://www.artstation.com/aipapi.

In this blog, I will discuss how to translate a 2D graphic novel into an interactive 3D experience with techniques that involve shading, geometry, composition, lighting, mood, and more. The primary focus will be on re-creating the atmosphere and setting from a comic strip. I found that once I've achieved the general effect, I could apply them to multiple environments. 

This work is based on my graduate project “Blacksad,” a short, immersive game created in Unreal Engine.
I hope you have a fun read!

First steps

The first step was to select a graphic novel to re-create in 3D. I chose “Blacksad,” by Juan Diaz Canales (writer) and Juanjo Guarnido (artist), because the story has a lot of character. The style of this novel is so captivating, from the lighting to the setting, that it was hard to resist not working on it. Once the novel was chosen, it needed to be analyzed thoroughly so I could understand the style and details. My main questions were:
  • What materials did they work with? 
  • How is their linework? 
  • How are their objects/characters drawn? 
  • How is the lighting? 
  • What is the general mood? 
  • What are the general colors? 
  • Where is the focus every time in the scene? 
  • How is the scale difference? 
For this analysis, I’ve created a style guide to help me better understand the comic and its style. 

General analysis

Blacksad-image-4.jpg

Environment 1 analysis

Blacksad-image-5.jpg

Environment 2 analysis

Blacksad-image-6.jpg

Environment 3 analysis

Blacksad-image-7.jpg As seen in the above images, the general details include:
  • Harsh black outlines
  • An overall grain effect 
  • Canvas texture background
  • Water paint effect/stains 
  • Warm colors + blue tints
  • Similar color grades
  • Raw painted shadows
  • Sketchy lines 
General: The comic has a watercolor and grungy effect layered on top combined with a nice warm and moody tone for each image (to achieve the detective style mood of the comic). Each object has a black outline, depending on the shot, some outlines are more "sketchy" than the others. To get this effect, I needed to keep it generally raw. 

In choosing scenes to recreate, I looked at ones that were open and told a story. In order to find the perfect shots, I had to ask myself: 
  • What happened?
  • Is this interesting to look at?
  • Can I expand on what is currently being shown?
  • What are my options for re-creation?
Blacksad-image-8-gif.gif

The objects

To start this project, I first focused on the objects. This was a good starting point since they are the base of every scene.
The scenes I chose were very detailed and crowded, and even though most of them had a lot of copying and pasting, there was still a large variation of objects in each scene. There were around 30-40 objects, many of which were re-usable. Deciding what had to be done first and what models were needed was my top priority from the beginning. Analyzing each scene can take time, but once you understand the scene, you can get fast results.
Blacksad-image-9.jpg
For example, in the images above, you can see that there are way too many objects that would need to be re-created, so many that it would make your head spin out of control. By simply splitting up this scene and thinking about what needs to be copy-pasted, you can move along much faster.

Eventually, it’s also necessary to analyze the size differences and stylization of each object. When you look at the style guide, you can see that the objects may not seem to be that stylized, but they do have a slight exaggeration in the silhouette. For example: The legs of a chair are a bit shorter in comparison to the main body, some objects have curves that shouldn’t have curves (like a bookcase), etc. Generally, the objects were comparatively realistic with a subtle touch of stylization.
Blacksad-image-10.jpg
Blacksad-image-11.jpg
When I originally started this project, the idea was to simply recreate one environment. Since I moved much faster than I anticipated, I ended up creating more environments and eventually turned the project into a short immersive game. More environments led to more objects. Once I had all of my objects ready, it was time to build my scenes.
Blacksad-image-12-gif.gif

The setting

Blacksad-image-13.jpg The first two weeks (of the total six) were mainly focused on getting all of the shapes, proportions, and feeling right. Recreating the first environment, The Office (image above) was quite difficult at first. Starting off a new scene can always be annoying because you need to calculate the correct room size. If you mess this up then everything in the scene will not feel right later on.

Once you finish general scaling, you can start dressing the scene with all of the details.Throughout this process you will still need to adjust proportions of your objects and the room in the engine itself.

Environment 1

Due to the fact that this was the first environment, it took a bit longer to set everything up. Setup included Blockout — BSP Shapes, basic lighting, important objects, getting the right settings, proportion research, etc. This environment was also the base for the other environments, so getting this right was crucial. Blacksad-image-14-gif.gif  

Environment 2 & 3

The second and third environments went a lot faster because it was based off the first one. In fact, a lot of the elements were re-used. By simply re-adjusting the objects and the layout of room,  you can easily create a totally new environment.
Blacksad-image-15-gif.gif
Blacksad-image-16-gif.gif

Connectors

Blacksad-image-17-gif.gif
Since there were multiple environments, I also needed to create a connector to switch between the different rooms. The first scene (The Office) was placed in an apartment, as seen in the references, and that’s how I came up with the idea of connecting the room with a hallway. I chose to style this hallway like one you would see in an old apartment because it fit with the moody scenes. Switching between scenes felt like switching between apartment rooms thus connecting the pieces of the story.
Blacksad-image-18.jpg

End result

Notice that each room translates its own story and mood. 
Blacksad-image-19-gif.gif
Blacksad-image-20-gif.gif
Blacksad-image-21-gif.gif

The look (shaders & materials) 

Main outline shader

The shaders were tricky. For the project, I needed something general that would work with each scene. Luckily, most of the images in the graphic novel had the same “feel.” Eventually, to get the right graphic style that can be easily adjusted, you would need to use different kinds of shaders. Instead of one big shader, you would actually use multiple. Think of a painting: you work layer by layer, bit by bit to get the desired outcome.
 
It’s eventually built up like this:
  • Main outline shader
  • Image overlays to get the painted look (these are just textures you place on top of each other on your screen)
  • Custom real-time shadows 
Blacksad-image-22.jpg
Once I had a rough blockout of my scene, I started playing around with the outline shader to achieve the right result. Everything moved fluidly, but eventually came to a halt due to an issue. An outline of the object was shown, but it was only a silhouette and there were no inner detail lines shown (details inside the object). That meant that if I put an object on top of another, it wouldn’t show the outline of the smaller object. For example, if a big carpet was placed under a chair, only the outline of the carpet would be shown.

First draft 

Notice that outlines are only shown on BSP walls and all the detail inside of an object is ignored:
Blacksad-image-23.jpg

Second draft 

All the lines show better now. The solution is discussed down below:
Blacksad-image-24.jpg
To fix this, I just had to adjust one simple thing. Blacksad-image-25.jpg
This is the main post process material that creates the outline shader. Due to the distance mask that was placed in the material (red outline) to create outlines closer to the camera, it broke the material. By simply removing this, everything was fixed. Blacksad-image-26.jpg
To get the outline, you need to find the depth of the scene to portray the outline and the vectors from the silhouette of the objects in the scene.

If you want more information about this, you can find it here: 

UE4 Tutorial – Toon Shader, June 4 – 2018
How to Outline Objects – UE4’, October 22 – 2017 

Without outline

Notice the painted feel. Blacksad-image-27.jpg

With outline

Notice that it brings everything increasingly together. Blacksad-image-28.jpg
Eventually, I was able to achieve a visually strong and supporting shader that lifted and translated the scene. The images above show the purpose of the shader and how important it is for this project.

Notice that without the shader, the image looks painted (which is good for the base) and all of the objects in the scene blend together. This makes it difficult to read the scene. The shader brings out more of the details in the scene and makes it more readable.

Outline shader for interactable objects

Since I made this project into a small game, I had to implement many details in order to immerse the player into the story, which include interactable environment pieces, audio, VFX, animations, story elements, etc. 

The player is able to click on certain objects in the scene to interact with, some of these will slowly move closer to the camera, and you will be able to rotate it around its axis to look at it more closely. These objects get an outline. When clicked, it moves closer to the camera and you are able to rotate it until you press the same button again.
Blacksad-image-29.jpg
Blacksad-image-30.jpg
I will not go into detail as to how this works since it is mainly based on existing tutorials that you can check out here: 

How to create an outline effect through Post Process:
http://www.michalorzelek.com/blog/tutorial-creating-outline-effect-around-objects/

Unreal Engine 4 – Zoom and Rotate Object
https://answers.unrealengine.com/questions/543371/rotating-an-object-relative-to-the-first-person-ca.html
Blacksad-image-31-gif.gif

Custom Shadows 

The custom shadow shader is subtle and really powerful at the same time. Without it, the scenes would not feel as special. It supports the scenes in a balanced way. It basically changes the shadows of the objects with a custom mask where light hits and adds a raw-edged visually appealing shadow. It also makes the AO in the scene a bit more dramatic/extreme, increases the range of it, and builds around the objects. Finally, the custom shadow adds more life to the scene and the movable lights/objects by increasing and decreasing the size depending on the light, character location, etc.

A surprisingly easy post-process material: Blacksad-image-32.jpg
Notice the cracked/uneven shadows where the light hits (in picture 1 after the GIF below) and the AO around the objects in the next picture. When you walk around in the scene, you can also see that the shadows kind of liquify together as if they were water. Funnily enough, I didn’t intend to produce this effect; it just happened as a happy accident and added a uniqueness to the scenes.
Blacksad-image-33.gif
Blacksad-image-34.jpg
Blacksad-image-35.jpg

Image overlays

To get the painted effect in the scenes, a combination of subtle image overlays had to be used to get a more grungy effect. This combined with the materials, post-processing, shadows, and lighting delivered the desired result. 
Blacksad-image-36.jpg
Here, simple grungy images were literally placed over the screen. However, you can’t just place one image and expect to be done. It needs to be built up subtly, otherwise it would be really obvious that you simply placed an image on top of the screen. This works well in combination with a dirt mask.
Blacksad-image-37.jpg
The image overlays make the whole scene look a bit rougher. If you look at the walls or specific objects in the next images, you may notice the effect. 
Blacksad-image-38.jpg
Blacksad-image-39.jpg
As said before, all of these are small, subtle changes that work together perfectly and create a strong end result (see the demonstration video). 

Materials

The materials are done fairly simple. Each object has an instance from a master material. These instances are just plain simple colors that you can adjust with a tileable texture if necessary. This technique was used instead of making hand-painted textures for each object because it would take an eternity to finish (due to so many objects, individual texturing would be impractical). Of course, you would get a more specific result by hand-painting, but using the colors and post-processing gives you faster results in a more efficient way.

Main Material

Blacksad-image-40.jpg
Blacksad-image-41.jpg
Difference with and without materials (colors)
Blacksad-image-42.jpg
Blacksad-image-43.jpg

Lighting and post-processing

Getting the lighting right is a real hassle since it requires a lot of effort to set everything from the basic directional light to smaller detail lighting (like lamps, candles, etc.).

If you look back to the references, you can see that the main tone of the environments is always mysterious, dark/moody, washed out by smoke, and features strong contrasting lights. The first environment needed to be more mysterious and somewhat inviting so it has been given a warmer but washed-out tone.

This also needs to be built up slowly.
Blacksad-image-44.jpg
Blacksad-image-45.jpg
Blacksad-image-46.jpg
To make the scenes more interesting, a GodRay material has been made with a strong orange color that accents the scenes. This combined with sub-lights (like lamps, candles, etc.) that also receive a stronger orange color make the scenes a lot more visually appealing,. 
Blacksad-image-47.jpg
The GodRay itself is built up with a mesh (cone) with a panning (an image that moves from one side to another) material on top that blends with the scene. It also reacts towards the custom real-time shadows to make them move.

Each room also receives a specific post-process volume to change the mood in the scene. This means that when you enter a room, the lighting and color may change since they have a certain range. Once you enter this range, the environment will be changed.

The main lighting is movable together with the sub-lights since everything is real-time.
Blacksad-image-48.jpg
In the images below, you can see that with just some lighting and post-processing, you can take your project to the next level.
Blacksad-image-49.jpg
Blacksad-image-50.jpg
Blacksad-image-51.jpg

The immersion

Sound 

Most of the audio has been downloaded from FreeSounds.org. A lot of sounds were needed to build up the ambiance for each scene because they all have their own “personality,” which include:

Environment 1 (Office): Moody, dark, mysterious.
Environment 2 (Blacksad’s Appartment murder scene): Bright, distressed, ominous.
Environment 3 (Ivoc Statoc’s Office): Misleading, worried, dark.

I chose the audio based off my own interpretation of the comics.

To achieve the mood, you need to work with the audio cues. Whenever you enter a room and walk over a collision box, it will trigger a specific sound. Some sounds are only played from a certain distance. For example, the closer you move towards windows, the louder the street sounds get.

Some yellow boxes trigger specific sounds when the player walks over them, others allow the user to interact with them (such as turning on the lamps).
Blacksad-image-52.jpg
To experience this yourself, watch the full demonstration video or play the game (which can be found on my portfolio)! 

Portfolio: https://www.artstation.com/artwork/Bm1yQ8 
Video: https://www.youtube.com/watch?v=_VZfcK9Rp1A 

The story

The story was an extra component of the project that complimented the environments. I implemented text portions of the graphic novel into the scenes to immerse the viewer in the environment and to make them curious about the story. The dialogue boxes (similar to what you see in a comic book) provide extra information about the specific room. They encourage the player to continue looking for clues, to further explore the environment, and leave them wanting to know more about what happened in each room.

The implementation was similar to the audio cues. Trigger boxes placed in the scenes would display the text blocks on the screen like a comic strip (see the images below).  
Blacksad-image-53.jpg
Blacksad-image-54.jpg

The VFX

VFX inside the scene itself is not that extreme. The main ones are smoke (cigarette, mist, steam etc.), fire, and some detailing.

The stylized smoke was made out of a cloud-looking mesh that uses the GodRay Material to make it transparent and misty; then it was placed in a particle system.
Blacksad-image-55-gif.gif
The smoke used for the mist is also a particle system using a flipbook image (an animation portrayed on a single image) to portray mist that you can view from afar (for example in the hallways of the scenes). 
Blacksad-image-56.jpg
Blacksad-image-57.jpg
The fire in the scene is just an image that gets transformed in a material by panning noise textures (distortion) over the specific image to make it seem like it moves. See Blueprints below: 
Blacksad-image-58.jpg
Blacksad-image-59.jpg

The conclusion

By working with only post-processing shaders, image overlays, and lighting you can create desired stylistic environments much faster. Of course, it may not be 100 percent representative of your reference images because you can only do so much with shaders. To get near perfect results, you would need to hand-paint scenes just like a graphic novel artist, but this would take much longer.

If your goal is to get the general effect with shaders, you can work roughly three times faster. As proven by my graduation work, you can make a small game that resembles a graphic novel in a rather short period of time (it took me just six weeks). This effect gives a look that is easily adjustable for each environment. With this, you can create a lot of variations by simply changing the images in the masked materials. 

Another piece of advice is that you need to take your time and analyze the project thoroughly. Build up subtle layers like a painting; create smaller pieces that compliment each other (shaders, lighting, colors, VFX, grunge, gameplay, interaction, etc)  instead of focusing on one big element. Piece by piece, you will create what has been portrayed in your reference; which, in my case, was to reach the goal of building a 3D graphic novel in UE4.

This was such a fun project and I sincerely hope this blog was a helpful read.

If you are interested to see more of my work or just want to message me, you can reach me at: 

ArtStation
LinkedIn
Facebook

Cheers!