My name is Aljosha Demeulemeester. I’m the CEO and co-founder of Graphine, a Belgian middleware company based in the city of Ghent. Graphine creates texture streaming and texture compression technologies for the video game and 3D visualization industries.
In the last 7 years, we’ve been dedicated to this one question: how can we remove the technical boundaries to using massive amounts of texture data in video games? The aim is, of course, to have extremely sharp graphics with virtual worlds full of unique detail. Not every game will need this, and adding more textures is not always enough to raise the bar in terms of visuals, but we’ve wanted to solve one piece of the puzzle and do it right.
Video memory is one of the limiting factors if you’d like to bump up the texture resolution or use more textures in a game. Graphics card vendors keep adding more memory but developers keep running out of it. Instead of loading their textures at the start of a game level and waiting for the loading to end, developers have already been using some form of texture streaming for years. However, even advanced systems like the mipmap streaming in Unreal Engine 4 has its limits.
With Granite SDK, our flagship middleware product, we set out to solve the issues that existing systems have and add capabilities like using very large textures (>16K). At the heart of Granite is a technique called virtual texturing, our approach to mega-texturing pioneered by id Software. You may have also heard of Tiled Resources (DX11.2) or Partially Resident Textures (OpenGL). These are technologies that accelerate virtual texturing in the graphics hardware, though they are not essential to running Granite.
We’ve had a sample integration of our SDK for UE4 for a while now but we recently launched Granite for Unreal, our first polished and user-friendly extension for UE4. Unreal is the perfect choice if you’re aiming for high-end graphics, and Granite is a great match. We also launched our new seat-based license for Unreal because we believe that every team can benefit from this kind of technology, small or large. And, we’re working on an indie license for which we'll have more info on later.
Why would you use Granite?
In short, Graphine for Unreal is a solution to use higher resolution texture maps or more textures in general without hitting limits like popping artefacts, memory shortage or long loading times. It allows you to use textures up to 256Kx256K, UDIM textures, or hundreds of individual 4K or 8K textures.
Granite has been used for a range of applications. Companies like Oculus, Allegorithmic and The Mill are using the technology to push the boundaries of real-time graphics quality. Dragon Commander by Larian Studios uses it for baked 16K landscape textures, and so does Get Even by The Farm 51 for handling unique high-resolution 3D scans for achieving a photorealistic look. For VR, Granite can help produce high-fidelity graphics by prebaking large ultra-detailed textures, handling them well even when rendering at a solid 90FPS.
Environment in Get Even by The Farm 51
In this article I’ll show how you can use larger texture resolutions in Unreal with Granite. You can find some technical information about texture streaming in the next section, so you can skip it if you just want to learn how to use Granite.
Texture streaming is the process of loading texture data from disk in the background while playing a game. Only a small subset of the textures in the world is actually kept in memory. The streaming system decides automatically when to load specific textures from disk, or the programmer explicitly requests textures to load, or a combination of both.
Unreal has its own texture streaming system that automatically loads textures based on a number of parameters, with the most important factor being the distance to the camera. Unreal will load a new mipmap of a texture once the camera gets within a certain distance of that texture. Such a system enables large open world games. However, having dense scenes is challenging. By dense, I mean many objects closely placed together or many textures per object. If you use too many textures in one specific area of your world, all of those textures get close to the camera at a specific time and need to be loaded by the streaming system. This can result in visual artefacts like popping and blurring. While you can set the size of the memory cache higher, this solution obviously has its limits.
With Granite SDK, we set out to create a solution to these challenges. It’s packed with technologies like tile-based texture streaming, virtual texturing shader libraries, a range of texture compression formats, texture transcoders, prediction systems, cache managements systems and more. Without going too deep into the technical details, Granite will divide all of the mipmaps of textures into small texture tiles, usually 128x128 pixels. While running a game, Granite will automatically figure out which tiles are actually visible by the camera and load only those tiles, as only parts of a texture mipmap are visible in most cases. For example, only the top is visible from the airplane wing in the image below. While in your typical streaming engine all of the textures of the plane would be loaded (8K diffuse, 8K normal and 8K specular), Granite will only load the tiles that are actually needed. This results in much less data to load, allowing you to use less texture memory which achieves a higher effective texture resolution.
One major benefit from the use of virtual texturing is that you can use much larger textures, even up to 256Kx256K. A texture of that size compressed with DXT1 would occupy 32GB of memory, and loading that texture would take minutes. With Granite, you need only 32MB of VRAM for that texture, and the loading time is unnoticeable.
One thing to take into consideration is that Virtual Texture sampling adds a small performance cost compared to regular texture sampling. We therefore limit the amount of VT textures to 16 per material and a maximum of 4 different UV coordinates to sample those textures. This way, we can make sure that the performance impact of the VT samples is almost unmeasurable in most practical scenarios (e.g., games or VR experiences).
Granite for Unreal Workflow
The workflow to import textures is a little different when using Granite. Here are the different steps:
- Import your textures into a Tile Set using one of our tools (see below). Tile Sets are our containers to store textures in a tiled format for efficient streaming from disk. A Tile Set can be thought of as a texture database or one huge texture atlas up to 256K.
- Import the .GTS Tile Set file into Unreal just as you would do with any asset. This will automatically create ‘Granite Texture’ objects in Unreal for all the textures that you imported in your Tile Set. These objects are similar to standard Unreal textures.
- Use your Granite Textures in Granite Material Nodes. These material nodes allow you to sample Granite Textures in your material.
That’s it! It’s really straightforward. The Granite Texture node can be used like a conventional texture node in your material graph, and all the Granite textures will automatically be streamed by Granite in the background.
Granite texture nodes can be combined with standard Unreal textures. So, which of your textures do you stream with Granite? Generally speaking, using Granite makes most sense for large textures (>1K) that are unique for a specific object or area. Smaller textures, textures that are sampled multiple times with different UVs, or textures that are visible most of the time are better handled by the Unreal streaming system or not streamed at all.
To create Tile Sets (.GTS files) and import your texture image files into this format, we have a handy tool called Tile Set Studio. It supports most common image formats like .jpg, .png, .exr, .tga, etc.
Importing Large Textures
Tile Set Studio can import individual image files up to 32Kx32K. One of its great features support for textures that are stored on disk as a grid of image files. Most tools that handle large image files allow you to export them as a set of images. To import such a grid of files, the files need to be named using letters or numbers to indicate the position in the grid. Examples are ‘landscape_02_06.jpg’ or “satellite-A-3.png”. Just select one of the images, and Tile Set studio will automatically detect that the image is part of a set. The combined resolution of these tiled images can be up to 256Kx256K.
Importing VFX-Quality UDIM textures
Tile Set Studio can easily import UDIM textures as well. Just select one of the UDIM patches, for example ‘robot_1016.exr’. Or, you can use our exporter directly from MARI. All the UDIM patches end up as one Granite Texture in Unreal after importing the Tile Set. Granite can handle UDIM UV coordinates so you don’t need to modify your meshes, and you can use these directly in Unreal.
Another essential element is that Granite can actually handle the large amount of textures that are typical for VFX-quality UDIM textures. For example, the Robot in the Rise demo by Nurulize (see image) has 700 4K maps. Granite will only load the subparts of the UDIM patches that are actually visible at a given time. This way, you can see the textures at the highest quality without any issues.
Importing Hundreds of Individual Textures
Finally, to avoid any tedious tasks, Tile Set Studio supports wildcards and batch importing. This allows you to import entire folders or a set of consistently named files like ‘plane_diffuse.jpg’, ‘robot_diffuse.jpg’ with one action. In this example, you would use ‘*_diffuse.jpg’ to import all the diffuse textures in that folder. The Granite textures in Unreal (after importing) will be named according to the unique part in the filename (‘plane’, ‘robot’, and so forth).
Other Notable Features
If you’re using Unreal's Lightmass system, you can easily stream your lightmaps using Granite. Just tick the selection box that you see below, and you’re set. Feel free to bump your lightmap resolution, as Granite can handle hundreds of gigabytes of lightmaps using only 64MB of video memory.
Granite can handle cubemaps just like any other texture. Just select your .DDS file and make sure to tick the cubemap box in Tile Set Studio. Granite will then automatically create a GraniteTextureCube object once you import your Tile Set in Unreal. You can use this streamed cubemap similar to a regular cubemap.
Granite contains a proprietary compression codec that is able to compress textures by 60% or more compared to a block-based format like DXT1, BC5 or BC7. The compression scheme is also scalable, allowing you to set a quality level per texture map (low, medium, high and lossless). The ‘Low’ and ‘Medium’ quality will result in a very small size on disk and better streaming performance. The ‘High’ quality setting will give you good quality while still compressing the texture on average by 60%. This results in efficient streaming and a reduced disk footprint.
Fine-Grained Performance Controls
As it should for any component in a 3D engine, Granite is designed to scale well from low-end to high-end machines. In a typical scenario, Granite needs around 512MB of video memory when rendering at 1080p. Setting the cache size higher will help Granite to cache more non-visible data. More caching can reduce the workload of the system at the cost of more memory. It’s not necessary, however, even if you’ve added tons of textures to your level. If you have only 256MB to spend on low-end machines for example, Granite will automatically scale back the texture resolution to ensure stable performance. You don’t need to manually tweak your texture settings.
The same scaling system can also be controlled to temporarily stream less data. For example, if you’re adding motion blur when the camera is moving fast, you don’t need to stream tiles from the most detailed mipmaps as the detail will be blurred again. The setting enables you to counter the effect of a fast moving camera and stream 4 or 16 times less without an impact on render quality in the example above.
Other settings control how much Granite can stream per frame. This is especially useful if you are targeting high framerates, or if you want to leave more room for other subsystems of your game. Lowering these settings ensures that Granite will use fewer resources per frame, essentially spreading out the work over more frames. The downside is that this can result in longer delays until all needed texture tiles are loaded at peak moments.
When mentioning texture streaming, people instinctively think of large open worlds. Granite is of course a good solution for that, but Unreal does a pretty good job itself. Where Granite really shines is when you have very dense scenes with lots of unique textures. In most cases, only parts of every object are visible, and Granite takes advantage of that by loading only those texture parts. Because of that approach, Granite is a great solution if you’d like to use large pre-baked landscape textures or satellite images, UDIM textures with many patches, large material masks, high-resolution lightmaps or photogrammetry scans. It can also help increase the resolution of object and character textures to 4K or 8K without popping issues or needing gigabytes of VRAM.
A big part of the work on Granite has been spent making sure that it uses as little resources as possible. Additionally, we’ve exposed a number of parameters so you can tweak the trade-off between performance, memory usage, disk usage and render quality. This way, you can get the best possible quality while constraining the resource budget of your application. This is especially important for virtual reality where hitting a solid 90FPS is crucial for a good experience. It’s exciting to see how people are using Granite for VR. Bumping up the texture resolution is a great way to add visual detail without impacting the framerate if you’re not bandwidth-limited on the graphics card.
With every Granite for Unreal release, we’re exposing more functionality of Granite SDK in Unreal. For example, future releases will have even more control over performance. We have already hooked up Granite to Unreal’s lightmap system, and we’re looking into Unreal’s terrain system. Granite for Unreal already supports heightmaps and displacement maps, so we might add automatic terrain support soon.
We’re constantly adding new features based on customer demand, so let us know what you think. We love getting feedback from the community! You can try it here if you’re not using Granite yet.
The first preview for Unreal Engine 4.19 is now available for download o...