Setting Up A Chroma Key Material in UE4

A bit over two years ago, someone on the UE4 AnswerHub asked about chroma keying (a.k.a. green screen) in a material. That post led to me mocking up a very basic chroma key function called Chroma Key Alpha that was added to UE4 a while ago. While it was in the release notes, our notes can be pretty long and it is easy to forget about features that are buried deep within them, especially if you had no reason to use them at the time.

AR is on the rise with many new tools and devices being supported all the time; UE4's recent support of ARKit and ARCore are a testament to this. You can read about that, including some amazing projects, in Tim Sweeney's post surrounding WWDC here.

With all this going on, it seems like a good time for some examples on how you can set up a chroma key material in UE4. While most AR demos are about compositing digital objects onto live video, some projects require mixed reality, involving projecting live subjects into digital scenes. This usually involves a green screen setup of some kind which can be a bit more challenging since you won't have a built-in alpha.

The process of extracting an alpha from a green screen is referred to as chroma keying. We recently added a UE4 plugin called Composure which makes it easier to mix and match post-processing and rendering elements, and that could be a great place to try out using a live chroma key material.

I will preface this by saying that getting high-quality chroma keying results is difficult and often requires different techniques to be mixed. Numerous software packages offer pretty advanced methods for this, such as Nuke. Many of the things Nuke will do to get a very high-quality chroma key alpha can be computationally intensive. Certain projects, such as live broadcasts, will tend to use expensive dedicated hardware solutions.

That means we need to assume that our real-time version will be fairly basic in comparison, and will probably require lots of content-dependent tweaks to look acceptable. But still, the value of being able to previs and test our chroma key right away in the editor can be very useful.

The Basic Methodology

The idea behind chroma keying is to create a color comparison mask from which to generate an alpha mask. Then a secondary mask is used to 'despill' or remove the green cast from objects. The despill mask is usually just a softer, inverted version of the alpha mask result. Finally, a version of the despill mask can be used to add back in some fake ambient lighting, replacing the green cast with a color matching the environment to be composited into.

There are almost infinite ways to go about the relatively simple steps above. The first version I have made is pretty basic and has been in UE4 for a while. As mentioned above, this is Chroma Key Alpha, and a basic example of its usage appears like this:

It takes an input from an image color, a chroma color, and a few inputs for alpha and despill masking. Rather than sticking to just examples using this node, I am going to show how each step of the function is performed as well as some better versions of the original setup.

Color Extraction

The first step in generating the color comparison is to remove luminance from the image so that subtle shadows and creases or lighting gradients on the green screen will not interfere. In the first version of this function, I ended up simply normalizing the colors. This works fairly well, but results in a little bit of edge artifacts.

Another way to remove the brightness it to just divide by the component sum. This gives a nice even tone, but it tends to create floating edge halo artifacts. I realized that using a color layer in Photoshop did not give such edge artifacts so I decided to think about how to remove them.

The answer was to use luminance-based saturation, which prevents very dark saturated pixels from becoming saturated in the color map. As it turns out, the dark saturated edge pixels are from bicubic texture resizing.

This test image is from the TV show Lost in Time, which uses UE4 for rendering scenes. Note that this test image is not high-quality raw source; it is a low-quality passed down jpeg.

To generate the luminance-based saturation map, the image is first desaturated, and a luminance curve is generated using a simple exponential function of e ^ -x. The luminance value x is scaled by a parameter for defining the strength of the luminance mask. It is normalized so that 1 is a good default and 0 will give a result exactly like the divide by sum version. Here is the code for that:

float3 ExtractColor(float3 Color, float LumaMask)

{
	float Luma = dot(Color, 1);
	float ColorMask = exp(-Luma * 2 * PI / LumaMask);
	Color = lerp( Color, Luma, ColorMask);
	return Color / (dot(Color, 2));
}

Once the color map is extracted, the next step is to generate a mask by doing a comparison. This step is pretty simple. First, the ChromaColor should be run through the same ExtractColor function (or normalize etc. if that is used). Then, the difference between the color map and ChromaColor is taken. Then, the length of that difference is calculated and a sharp mask is extracted from the broad gradient to isolate the specific desired range of error.

Here is an example of that in nodes, with the ExtractColor function above:

Note that in the above example I am using 'Chroma Alpha Strength' as a multiplier. I think that is a more intuitive way to specify the edge sharpness. In the UE4 packaged material function, I specified this using a Min and Max. The result is the same, but it means Max always needs to be set higher than Min, so Min needs to be constantly adjusted.

Despill

After extracting an alpha mask using the chroma key, the next important step is to despill the remaining pixels. That means getting rid of all the green cast on the silhouette of the subjects. This is necessary due to a variety of effects from both lighting and cameras. For one, the larger the green screen, the more green bounce light it will cast onto the subjects. Also, cameras tend to pick up a variety of lens artifacts which can cause a bit of bloom from bright pixels onto subjects, even if the bounce light is minimal on set.

A good start to handle the despill is to use the same setup as the alpha mask, but using a wider range of values to give a softer mask. In the built-in function, this is done by exposing a separate Despill Max. The Min used for despill will be the same as the one used for the alpha.

Once you have a despill alpha, it can be used to remove the cast of the chroma color from the image. Originally, I was just desaturating using the despill alpha, but better results can be had by subtracting the portion of the color that matches the chroma color from the source image.

Here is the basic logic for how the Despill Alpha can be used to remove chroma spill from the image:

Note that for this image, a much higher quality result can be achieved by using two Chroma Colors and two Chroma Key Alpha nodes. You can tell because the color map has a pretty different color on the floor versus the wall, so when using only a single chroma comparison, a much wider cutoff must be used. This means you lose flexibility in the edge softness, which makes things like the motion-blurred lever tricky to fix. To use two, the result of each alpha would be combined using Min. The Despill Alphas would be combined using Max.

Adding fake bounce is pretty simple. It is a good idea to desaturate the base image colors before using them with a manually specified bounce or background color. That is for two reasons. First, you need to get rid of any existing cast in the source, and you really just want the luminance. Second, most of the cast will be from specular surfaces due to glancing angles, and for all non-metals, specular is completely desaturated and only colored by lighting.

Somebody who does compositing professionally may have to get pretty detailed about how they mask and add fake lighting, including using separate rendered elements to help pick up the right lighting response. That goes a bit beyond the scope of this article but hopefully this at least gives an idea for how you can approach handling live compositing.

Here is another example, this time using a higher quality image that was given to me by Joe Wilson from Epic's video production team. This is a shot of Tools Programmer Lauren Ridge that was taken during the making of the Star Wars VR Experience demo that was shown at WWDC (link at top of page). With a higher quality source, you can get much better results. This example uses the existing version of Chroma Key Alpha.

The added bounce in the last frame may be difficult to perceive except for around the edges and especially on the darker reflective bits such as the Vive headset and controllers. Notice the added bounce brings back reflections of the sky on those elements which helps them fit into the comp.

Here is the whole material for the above setup with despill and bounce/edgebleed color added:

Composure

At the start I mentioned Composure briefly. I have not yet tried adding a chroma key mode to Composure, but I have dug around in the example and it looks like it would be pretty simple to set up. There is a Final Compositing Material that handles the various compositing setups. For the example project, all of the color masks were imported from a program like Nuke, but the usage in the material could just as easily be replaced with the chroma key node setup described here. 

That is everything for now, and we look forward to seeing what the community does with mixed reality and Unreal Engine.

--

EDITOR’S NOTE: This blog post first appeared on Ryan’s personal blog, ShaderBits.com, and has been repurposed here for broader Unreal Engine community consumption. You can follow Ryan on Twitter, @ShaderBits.