September 12, 2017

How to Add Global Shaders to UE4

By Rolando Caloca Olivares

In Unreal Engine 4, global shaders are shaders that can be used from the C++ side to render post-processing effects, dispatch compute shaders, clear the screen, etc. (i.e., shaders that don’t operate on a material or a mesh). On occasion, more advanced functionality is necessary to achieve a desired look, and a custom shader pass is required. Doing this is relatively simple, as I will explain here.

Unreal Shader Files and How To Use Them

UE4 reads .usf files (Unreal Shader Files) from the Engine/Shaders folder. Any new shaders need their source files placed here. As of 4.17, shaders can also be read from a plugin (Plugin/Shaders). I’d recommend enabling r.ShaderDevelopmentMode=1 in your ConsoleVariables.ini file for ease of development. Check out the documentation for more information.

Let’s start by adding a new .usf file in your Engine/Shaders folder. We’ll call it MyTest.usf. Then add a simple pass-through Vertex Shader and a Pixel Shader that returns a custom color:

// MyTest.usf 

// Simple pass-through vertex shader

void MainVS(
	in float4 InPosition : ATTRIBUTE0,
	out float4 Output : SV_POSITION
)
{
	Output = InPosition;
}

// Simple solid color pixel shader
float4 MyColor;
float4 MainPS() : SV_Target0
{
	return MyColor;
}

Now, in order to get UE4 to pick up the shader and start compiling it, we need to declare a C++ class. Let’s start with the Vertex Shader:

// This can go on a header or cpp file
class FMyTestVS : public FGlobalShader
{
	DECLARE_EXPORTED_SHADER_TYPE(FMyTestVS, Global, /*MYMODULE_API*/);
 
	FMyTestVS() { }
	FMyTestVS(const ShaderMetaType::CompiledShaderInitializerType& Initializer)
		: FGlobalShader(Initializer)
	{
	}
 
	static bool ShouldCache(EShaderPlatform Platform)
	{
		return true;
	}
};

There are a few requirements here:

  1. This is a subclass of FGlobalShader. As such, it will end up in the Global Shader Map  (which means we don’t need a material to find it).
  2. Usage of the DECLARE_EXPORTED_SHADER_TYPE() macro will generate exports required for serialization of the shader type, etc. The third parameter is a type for external linkage for the code module where the shader module will live, if required (e.g. the C++ code that doesn’t live in the Renderer module).
  3. Two constructors, both the default and the serialization.
  4. The ShouldCache() function, needed to decide if this shader should be compiled under certain circumstances (e.g. we might not want to compile a compute shader on a non-compute shader capable RHI).

With the class declared, we can now register the Shader Type to UE4’s list:

// This needs to go on a cpp file
IMPLEMENT_SHADER_TYPE(, FMyTestVS, TEXT("MyTest"), TEXT("MainVS"), SF_Vertex);

This macro maps the type (FMyTestVS) to the .usf file (MyTest.usf), the shader entry point (MainVS), and the frequency/shader stage (SF_Vertex). It also causes the shader to be added to the compilation list as long as its ShouldCache() method returns true.

Note: Whichever module you add your FGlobalShader to has to be loaded before the actual engine starts, or you will get an assert like, “Shader type was loaded after engine init, use ELoadingPhase::PostConfigInit on your module to cause it to load earlier.” We currently do not allow a dynamic module that is loaded after a game or editor has launched to add its own shader type.

Let’s now declare the more interesting Pixel Shader:

class FMyTestPS : public FGlobalShader
{
	DECLARE_EXPORTED_SHADER_TYPE(FMyTestPS, Global, /*MYMODULE_API*/);
 
	FShaderParameter MyColorParameter;
 
	FMyTestPS() { }
	FMyTestPS(const ShaderMetaType::CompiledShaderInitializerType& Initializer)
		: FGlobalShader(Initializer)
	{
		MyColorParameter.Bind(Initializer.ParameterMap, TEXT("MyColor"), SPF_Mandatory);
	}
 
	static void ModifyCompilationEnvironment(EShaderPlatform Platform, FShaderCompilerEnvironment& OutEnvironment)
	{
		FGlobalShader::ModifyCompilationEnvironment(Platform, OutEnvironment);
		// Add your own defines for the shader code
		OutEnvironment.SetDefine(TEXT("MY_DEFINE"), 1);
	}
 
	static bool ShouldCache(EShaderPlatform Platform)
	{
		// Could skip compiling for Platform == SP_METAL for example
		return true;
	}
 
	// FShader interface.
	virtual bool Serialize(FArchive& Ar) override
	{
		bool bShaderHasOutdatedParameters = FGlobalShader::Serialize(Ar);
		Ar << MyColorParameter;
		return bShaderHasOutdatedParameters;
	}
 
	void SetColor(FRHICommandList& RHICmdList, const FLinearColor& Color)
	{
		SetShaderValue(RHICmdList, GetPixelShader(), MyColorParameter, Color);
	}
};

// Same source file as before, different entry point
IMPLEMENT_SHADER_TYPE(, FMyTestPS, TEXT("MyTest"), TEXT("MainPS"), SF_Pixel);

In this class we are now exposing the shader parameter MyColor from the .usf file:

  • The FShaderParameter MyColorParameter member is added to the class, which will hold information for the runtime to be able to find the bindings, allowing the value of the parameter to be set at runtime.
  • In the serialization constructor we Bind() the parameter to the ParameterMap by name, this has to match the .usf file’s name.
  • The new ModifyCompilationEnvironment() function is used when the same C++ class define different behaviors and be able to set up #define values in the shader.
  • The Serialize() method is required. This is where the compile/cook time information from the shader’s binding (matched during the serialization constructor) gets loaded and stored at runtime.
  • Finally, we have a custom SetColor() method which shows how to set the MyColor parameter at runtime with a specified value.

Now, let’s write a simple function to draw a fullscreen quad using these shader types:

void RenderMyTest(FRHICommandList& RHICmdList, ERHIFeatureLevel::Type FeatureLevel, const FLinearColor& Color)
{
	// Get the collection of Global Shaders
	auto ShaderMap = GetGlobalShaderMap(FeatureLevel);

	// Get the actual shader instances off the ShaderMap
	TShaderMapRef MyVS(ShaderMap);
	TShaderMapRef MyPS(ShaderMap);

	// Declare a bound shader state using those shaders and apply it to the command list
	static FGlobalBoundShaderState MyTestBoundShaderState;
	SetGlobalBoundShaderState(RHICmdList, FeatureLevel, MyTestBoundShaderState, GetVertexDeclarationFVector4(), *MyVS, *MyPS);

	// Call our function to set up parameters
	MyPS->SetColor(RHICmdList, Color);
 
	// Setup the GPU in prep for drawing a solid quad
	RHICmdList.SetRasterizerState(TStaticRasterizerState::GetRHI());
	RHICmdList.SetBlendState(TStaticBlendState<>::GetRHI());
	RHICmdList.SetDepthStencilState(TStaticDepthStencilState::GetRHI(), 0);
 
	// Setup the vertices
	FVector4 Vertices[4];
	Vertices[0].Set(-1.0f, 1.0f, 0, 1.0f);
	Vertices[1].Set(1.0f, 1.0f, 0, 1.0f);
	Vertices[2].Set(-1.0f, -1.0f, 0, 1.0f);
	Vertices[3].Set(1.0f, -1.0f, 0, 1.0f);

	// Draw the quad
	DrawPrimitiveUP(RHICmdList, PT_TriangleStrip, 2, Vertices, sizeof(Vertices[0]));
}

If you want to test this in your codebase, you can try declaring a console variable so it can be toggled at runtime like so:

static TAutoConsoleVariable CVarMyTest(
	TEXT("r.MyTest"),
	0,
	TEXT("Test My Global Shader, set it to 0 to disable, or to 1, 2 or 3 for fun!"),
	ECVF_RenderThreadSafe
);
 
void FDeferredShadingSceneRenderer::RenderFinish(FRHICommandListImmediate& RHICmdList)
{
	[...]
	// ***
	// Inserted code, just before finishing rendering, so we can overwrite the screen’s contents!
	int32 MyTestValue = CVarMyTest.GetValueOnAnyThread();
	if (MyTestValue != 0)
	{
		FLinearColor Color(MyTestValue == 1, MyTestValue == 2, MyTestValue == 3, 1);
		RenderMyTest(RHICmdList, FeatureLevel, Color);
	}
	// End Inserted code
	// ***
	FSceneRenderer::RenderFinish(RHICmdList);
	[...]
}

At this point you should be able to test our your new global shader! Run your project, then pull up a console using tilde (~) and type r.MyTest 1. Then type r.MyTest 2 and/or r.MyTest 3 to change colors. Use r.MyTest 0 to disable the pass.

Debugging the Generated Source

Take a look at the blog post Debugging the Shader Compiling Process if you want to be able to debug the compilation of your .usf file and/or see the processed file.

Bonus!

You can modify the .usf file while an uncooked game/editor is running, and then press Ctrl+Shift+. (period) or type recompileshaders changed in the console to pick up and rebuild your shaders for quick iteration!

Enjoy!
 

Notes on Terminology

Global Shaders - Shaders that are not created using the material editor, typically compute shaders, post-processing shaders, etc. This is one of the meta types (i.e., Global, Material, Mesh); the remaining meta types will be covered in the future.
FShaderType - A ‘template’ or ‘class’ specified by shader code, which maps to a physical C++ class specified in the code.
FShaderResource - The compiled shader microcode and its runtime RHI resource.
FShader - A compiled shader instance of an FShaderType, built from the information of an FShaderResource.
TShaderMap - A collection of shaders of different FShaderTypes, but with the same meta type (Global).