8 de marzo de 2019
Blacksad breakdown: Howest student creates a playable 3D comic in UE4
While attending DAE, I worked on a variety of projects from illustrations to detailed environments and characters. Most of my work can be found on my Artstation portfolio: https://www.artstation.com/aipapi.
In this blog, I will discuss how to translate a 2D graphic novel into an interactive 3D experience with techniques that involve shading, geometry, composition, lighting, mood, and more. The primary focus will be on re-creating the atmosphere and setting from a comic strip. I found that once I've achieved the general effect, I could apply them to multiple environments.
This work is based on my graduate project “Blacksad,” a short, immersive game created in Unreal Engine. I hope you have a fun read!
First steps
The first step was to select a graphic novel to re-create in 3D. I chose “Blacksad,” by Juan Diaz Canales (writer) and Juanjo Guarnido (artist), because the story has a lot of character. The style of this novel is so captivating, from the lighting to the setting, that it was hard to resist not working on it. Once the novel was chosen, it needed to be analyzed thoroughly so I could understand the style and details. My main questions were:- What materials did they work with?
- How is their linework?
- How are their objects/characters drawn?
- How is the lighting?
- What is the general mood?
- What are the general colors?
- Where is the focus every time in the scene?
- How is the scale difference?
General analysis
Environment 1 analysis
Environment 2 analysis
Environment 3 analysis
As seen in the above images, the general details include:
- Harsh black outlines
- An overall grain effect
- Canvas texture background
- Water paint effect/stains
- Warm colors + blue tints
- Similar color grades
- Raw painted shadows
- Sketchy lines
In choosing scenes to recreate, I looked at ones that were open and told a story. In order to find the perfect shots, I had to ask myself:
- What happened?
- Is this interesting to look at?
- Can I expand on what is currently being shown?
- What are my options for re-creation?
The objects
To start this project, I first focused on the objects. This was a good starting point since they are the base of every scene.The scenes I chose were very detailed and crowded, and even though most of them had a lot of copying and pasting, there was still a large variation of objects in each scene. There were around 30-40 objects, many of which were re-usable. Deciding what had to be done first and what models were needed was my top priority from the beginning. Analyzing each scene can take time, but once you understand the scene, you can get fast results.
For example, in the images above, you can see that there are way too many objects that would need to be re-created, so many that it would make your head spin out of control. By simply splitting up this scene and thinking about what needs to be copy-pasted, you can move along much faster.
Eventually, it’s also necessary to analyze the size differences and stylization of each object. When you look at the style guide, you can see that the objects may not seem to be that stylized, but they do have a slight exaggeration in the silhouette. For example: The legs of a chair are a bit shorter in comparison to the main body, some objects have curves that shouldn’t have curves (like a bookcase), etc. Generally, the objects were comparatively realistic with a subtle touch of stylization.
When I originally started this project, the idea was to simply recreate one environment. Since I moved much faster than I anticipated, I ended up creating more environments and eventually turned the project into a short immersive game. More environments led to more objects. Once I had all of my objects ready, it was time to build my scenes.
The setting
The first two weeks (of the total six) were mainly focused on getting all of the shapes, proportions, and feeling right. Recreating the first environment, The Office (image above) was quite difficult at first. Starting off a new scene can always be annoying because you need to calculate the correct room size. If you mess this up then everything in the scene will not feel right later on.Once you finish general scaling, you can start dressing the scene with all of the details.Throughout this process you will still need to adjust proportions of your objects and the room in the engine itself.
Environment 1
Due to the fact that this was the first environment, it took a bit longer to set everything up. Setup included Blockout — BSP Shapes, basic lighting, important objects, getting the right settings, proportion research, etc. This environment was also the base for the other environments, so getting this right was crucial.
Environment 2 & 3
The second and third environments went a lot faster because it was based off the first one. In fact, a lot of the elements were re-used. By simply re-adjusting the objects and the layout of room, you can easily create a totally new environment.
Connectors
Since there were multiple environments, I also needed to create a connector to switch between the different rooms. The first scene (The Office) was placed in an apartment, as seen in the references, and that’s how I came up with the idea of connecting the room with a hallway. I chose to style this hallway like one you would see in an old apartment because it fit with the moody scenes. Switching between scenes felt like switching between apartment rooms thus connecting the pieces of the story.
End result
Notice that each room translates its own story and mood.
The look (shaders & materials)
Main outline shader
The shaders were tricky. For the project, I needed something general that would work with each scene. Luckily, most of the images in the graphic novel had the same “feel.” Eventually, to get the right graphic style that can be easily adjusted, you would need to use different kinds of shaders. Instead of one big shader, you would actually use multiple. Think of a painting: you work layer by layer, bit by bit to get the desired outcome.It’s eventually built up like this:
- Main outline shader
- Image overlays to get the painted look (these are just textures you place on top of each other on your screen)
- Custom real-time shadows
Once I had a rough blockout of my scene, I started playing around with the outline shader to achieve the right result. Everything moved fluidly, but eventually came to a halt due to an issue. An outline of the object was shown, but it was only a silhouette and there were no inner detail lines shown (details inside the object). That meant that if I put an object on top of another, it wouldn’t show the outline of the smaller object. For example, if a big carpet was placed under a chair, only the outline of the carpet would be shown.
First draft
Notice that outlines are only shown on BSP walls and all the detail inside of an object is ignored:
Second draft
All the lines show better now. The solution is discussed down below:
To fix this, I just had to adjust one simple thing.
This is the main post process material that creates the outline shader. Due to the distance mask that was placed in the material (red outline) to create outlines closer to the camera, it broke the material. By simply removing this, everything was fixed.
To get the outline, you need to find the depth of the scene to portray the outline and the vectors from the silhouette of the objects in the scene.
If you want more information about this, you can find it here:
UE4 Tutorial – Toon Shader, June 4 – 2018 How to Outline Objects – UE4’, October 22 – 2017
Without outline
Notice the painted feel.
With outline
Notice that it brings everything increasingly together.
Eventually, I was able to achieve a visually strong and supporting shader that lifted and translated the scene. The images above show the purpose of the shader and how important it is for this project.
Notice that without the shader, the image looks painted (which is good for the base) and all of the objects in the scene blend together. This makes it difficult to read the scene. The shader brings out more of the details in the scene and makes it more readable.
Outline shader for interactable objects
Since I made this project into a small game, I had to implement many details in order to immerse the player into the story, which include interactable environment pieces, audio, VFX, animations, story elements, etc.The player is able to click on certain objects in the scene to interact with, some of these will slowly move closer to the camera, and you will be able to rotate it around its axis to look at it more closely. These objects get an outline. When clicked, it moves closer to the camera and you are able to rotate it until you press the same button again.
I will not go into detail as to how this works since it is mainly based on existing tutorials that you can check out here:
How to create an outline effect through Post Process:
http://www.michalorzelek.com/blog/tutorial-creating-outline-effect-around-objects/
Unreal Engine 4 – Zoom and Rotate Object
https://answers.unrealengine.com/questions/543371/rotating-an-object-relative-to-the-first-person-ca.html
Custom Shadows
The custom shadow shader is subtle and really powerful at the same time. Without it, the scenes would not feel as special. It supports the scenes in a balanced way. It basically changes the shadows of the objects with a custom mask where light hits and adds a raw-edged visually appealing shadow. It also makes the AO in the scene a bit more dramatic/extreme, increases the range of it, and builds around the objects. Finally, the custom shadow adds more life to the scene and the movable lights/objects by increasing and decreasing the size depending on the light, character location, etc.A surprisingly easy post-process material:
Notice the cracked/uneven shadows where the light hits (in picture 1 after the GIF below) and the AO around the objects in the next picture. When you walk around in the scene, you can also see that the shadows kind of liquify together as if they were water. Funnily enough, I didn’t intend to produce this effect; it just happened as a happy accident and added a uniqueness to the scenes.
Image overlays
To get the painted effect in the scenes, a combination of subtle image overlays had to be used to get a more grungy effect. This combined with the materials, post-processing, shadows, and lighting delivered the desired result.
Here, simple grungy images were literally placed over the screen. However, you can’t just place one image and expect to be done. It needs to be built up subtly, otherwise it would be really obvious that you simply placed an image on top of the screen. This works well in combination with a dirt mask.
The image overlays make the whole scene look a bit rougher. If you look at the walls or specific objects in the next images, you may notice the effect.
As said before, all of these are small, subtle changes that work together perfectly and create a strong end result (see the demonstration video).
Materials
The materials are done fairly simple. Each object has an instance from a master material. These instances are just plain simple colors that you can adjust with a tileable texture if necessary. This technique was used instead of making hand-painted textures for each object because it would take an eternity to finish (due to so many objects, individual texturing would be impractical). Of course, you would get a more specific result by hand-painting, but using the colors and post-processing gives you faster results in a more efficient way.Main Material
Difference with and without materials (colors)
Lighting and post-processing
Getting the lighting right is a real hassle since it requires a lot of effort to set everything from the basic directional light to smaller detail lighting (like lamps, candles, etc.).If you look back to the references, you can see that the main tone of the environments is always mysterious, dark/moody, washed out by smoke, and features strong contrasting lights. The first environment needed to be more mysterious and somewhat inviting so it has been given a warmer but washed-out tone.
This also needs to be built up slowly.
To make the scenes more interesting, a GodRay material has been made with a strong orange color that accents the scenes. This combined with sub-lights (like lamps, candles, etc.) that also receive a stronger orange color make the scenes a lot more visually appealing,.
The GodRay itself is built up with a mesh (cone) with a panning (an image that moves from one side to another) material on top that blends with the scene. It also reacts towards the custom real-time shadows to make them move.
Each room also receives a specific post-process volume to change the mood in the scene. This means that when you enter a room, the lighting and color may change since they have a certain range. Once you enter this range, the environment will be changed.
The main lighting is movable together with the sub-lights since everything is real-time.
In the images below, you can see that with just some lighting and post-processing, you can take your project to the next level.
The immersion
Sound
Most of the audio has been downloaded from FreeSounds.org. A lot of sounds were needed to build up the ambiance for each scene because they all have their own “personality,” which include:Environment 1 (Office): Moody, dark, mysterious.
Environment 2 (Blacksad’s Appartment murder scene): Bright, distressed, ominous.
Environment 3 (Ivoc Statoc’s Office): Misleading, worried, dark.
I chose the audio based off my own interpretation of the comics.
To achieve the mood, you need to work with the audio cues. Whenever you enter a room and walk over a collision box, it will trigger a specific sound. Some sounds are only played from a certain distance. For example, the closer you move towards windows, the louder the street sounds get.
Some yellow boxes trigger specific sounds when the player walks over them, others allow the user to interact with them (such as turning on the lamps).
To experience this yourself, watch the full demonstration video or play the game (which can be found on my portfolio)!
Portfolio: https://www.artstation.com/artwork/Bm1yQ8
Video: https://www.youtube.com/watch?v=_VZfcK9Rp1A
The story
The story was an extra component of the project that complimented the environments. I implemented text portions of the graphic novel into the scenes to immerse the viewer in the environment and to make them curious about the story. The dialogue boxes (similar to what you see in a comic book) provide extra information about the specific room. They encourage the player to continue looking for clues, to further explore the environment, and leave them wanting to know more about what happened in each room.The implementation was similar to the audio cues. Trigger boxes placed in the scenes would display the text blocks on the screen like a comic strip (see the images below).
The VFX
VFX inside the scene itself is not that extreme. The main ones are smoke (cigarette, mist, steam etc.), fire, and some detailing.The stylized smoke was made out of a cloud-looking mesh that uses the GodRay Material to make it transparent and misty; then it was placed in a particle system.
The smoke used for the mist is also a particle system using a flipbook image (an animation portrayed on a single image) to portray mist that you can view from afar (for example in the hallways of the scenes).
The fire in the scene is just an image that gets transformed in a material by panning noise textures (distortion) over the specific image to make it seem like it moves. See Blueprints below:
The conclusion
By working with only post-processing shaders, image overlays, and lighting you can create desired stylistic environments much faster. Of course, it may not be 100 percent representative of your reference images because you can only do so much with shaders. To get near perfect results, you would need to hand-paint scenes just like a graphic novel artist, but this would take much longer.If your goal is to get the general effect with shaders, you can work roughly three times faster. As proven by my graduation work, you can make a small game that resembles a graphic novel in a rather short period of time (it took me just six weeks). This effect gives a look that is easily adjustable for each environment. With this, you can create a lot of variations by simply changing the images in the masked materials.
Another piece of advice is that you need to take your time and analyze the project thoroughly. Build up subtle layers like a painting; create smaller pieces that compliment each other (shaders, lighting, colors, VFX, grunge, gameplay, interaction, etc) instead of focusing on one big element. Piece by piece, you will create what has been portrayed in your reference; which, in my case, was to reach the goal of building a 3D graphic novel in UE4.
This was such a fun project and I sincerely hope this blog was a helpful read.
If you are interested to see more of my work or just want to message me, you can reach me at:
ArtStation
Cheers!