November 27, 2019

Exploring SculptrVR’s efficient lighting solution for the Oculus Quest

Art

Blueprints

Community

Design

Learning

Mobile

Product Design

Visualization

By Creator of SculptrVR Nathan Rowe

Hi, I’m Nathan Rowe and I’m the creator of SculptrVR, which is a multiplayer voxel sculpting application for virtual reality. The app has been released on SteamVR, Oculus Rift, PlayStation VR, and recently, mobile standalone VR headsets like the Oculus Quest and Oculus Go. Comparatively speaking, the Quest has a limited graphical computing budget and, as a result, a new, efficient lighting solution had to be made to achieve optimal performance.

Voxel sculpting is notorious for high-poly-count models. SculptrVR’s octree system produces reduced triangles, but it’s still a large number. To have a good experience, SculptrVR needs to be able to push a lot of triangles on screen. On Quest, I get up to 350,000 triangles and maintain a rock steady 72 fps stereoscopic. Unfortunately, this makes the default UE4 lighting model a bit too expensive.

A model created in SculptrVR and rendered using the efficient mobile lighting described in this article.

The simple default lit shader that just takes a color and drags it into Base Color is hundreds of operations. Even with “fully rough” checked, a default lit Material can only afford to render about 50,000 triangles on screen on the Quest. This was not enough to get a good experience sculpting.

So, instead of using the default UE4 lighting, I built my own phong lighting by using unlit Emissive Materials. This Material looks expensive, but is only 17 operations.

Want to see how this works and perhaps use it yourself? Here's a pastebin link: https://blueprintue.com/blueprint/1gp1z-xh/

Let’s go through what’s happening here:

First, notice that three different Material types are supported: clay, metal, and glowing. Clay is emulating a fully rough Material, metal is shiny and reflective, and glowing is clay plus a constant glow amount. The Material type is encoded in the 8-bit ambient channel of each vertex color.

The world is lit by a single directional light and an ambient skybox (the skybox in SculptrVR is actually using the same ambient section code, but with slightly different constants). I am using Blinn-Phong lighting to get diffuse lighting with a specular highlight, and to that I’m adding ambient light from the analytical skybox.

The result of this gets multiplied by a vertex color and a per-vertex ambient occlusion (AO) term. The AO term is computed on the CPU for each vertex using a custom variant of voxel cone tracing.

Let’s go through each of the lighting components:

Diffuse:

Diffuse lighting is independent of camera position, and is completely determined by the sun position. So all we do here is dot product the sun direction with the surface normal. The final result here ranges from 0.2 to 1.0 for clay, and .05 to 0.25 for metal. Theoretically, purely metallic surfaces shouldn’t have any diffuse lighting, but adding a bit of diffuse light helps the eye to better understand the shape, which is important in a sculpting application.

Specular:

Specular light just shows the reflection of the sun on the surface of objects. The sharpness of the highlight is usually based on roughness, and with Blinn-Phong lighting, you simply raise the dot product to different powers to adjust sharpness. Here the power is 6 for clay and 64 for metal. We also adjust the max specular brightness up and down for metal/clay.

Ambient:

The ambient lighting is my attempt at adding an efficient “skylight” equivalent to the shader. The top three comment boxes are interpolating between the sky color and the ground color based on the Z coordinate of either the surface normal (for clay) or the reflection vector (for metal). The interpolation is sharpened by multiplying that Z coordinate by a constant. The fourth comment box is adding in a horizon line when that Z coordinate is near zero. The horizon line is absolutely crucial for giving a sense of shape to metallic shiny objects.

Emissive:

Emissive light is completely view independent and simply adds a base amount to every glowing pixel. In SculptrVR, the “GlowMultiple” parameter is oscillating to provide a blinking look. That oscillation is controlled by the CPU in a Blueprint. Doing it on the CPU once saves a few operations that would have been redone on millions of pixels.

Total Lighting:

Tech-Blog_ScupltVR_LightingFunctionMaterial_Combining.png

Diffuse, Specular, and Ambient light get summed together and multiplied by the sun color to control brightness for day/night, and then Emissive light gets added in after that.

Adjusting the Parameters for a Dynamic Sky

The Sun Position, Sun Color, Zenith Color, Horizon Color, and Ground Color are all dynamically changed as you move around the sun as seen below

The colors are all stored in curve tables based purely on the z-component of the sun’s direction. Those tables were hand tuned to provide good brightness and contrast with SculptrVR’s color palette.

Seen above: White metal, green glowing, and cyan clay Materials. Note how much shape information you can perceive despite the objects being solid colors.

In SculptrVR, every single Material is emissive and uses this Material function to multiply in lighting. Some Materials/models use vertex colors, some are textured, but they all get consistent lighting.

I hope you found this useful and can either make use of it directly or make your own variant with this as a starting point. Thanks for reading.