Courtesy of Inflexion Games

Taking the leap on Nightingale: How Inflexion Games upgraded from UE4 to UE5 mid-development

Inflexion Games Technical Director of Nightingale Jacques Lebrun and Inflexion Games Environment Director Noel Lukasewich |
25 de outubro de 2023
Inflexion Games is an Edmonton, Canada-based studio formed in 2018 by a group of industry veterans. It has since grown to over 100 passionate developers with a range of backgrounds including AAA, as well as those new to the industry—all united in the pursuit of enriching games with depth, history, and opportunity. Previous credits include venerated franchises Mass Effect, Dragon Age, Wipeout, and Far Cry. Supported by Tencent, the studio’s first self-published game will be a shared world survival crafting title, Nightingale.
We are Jacques Lebrun and Noel Lukasewich, two of several developers at Inflexion who worked on upgrading our upcoming title Nightingale to Unreal Engine 5.
 
With last year’s initial release of UE5, our team got very excited about taking advantage of the new features and opportunities, but the idea of migrating a project from Unreal Engine 4 initially appeared to be a daunting task. In this article, we’ll explore what it takes to upgrade a project in production and offer some suggestions that might help you take the leap yourself. 

A note about source control tools: In our case we obtain engine updates through Github, and then maintain our own source control through Perforce, but you may still benefit from this article if you’re using a different setup. 

Planning and preparation

Before we talk about the upgrade process, it’s important to understand our motivations and why we took on the added effort of being an early adopter of UE5. 

The most compelling motivators for us centered around the visual enhancements that are made possible with the combination of Lumen and Nanite. We initially didn’t think moving to Nanite would be possible for our project, given all of our assets were authored without Nanite in mind (spoiler: we ended up moving to Nanite, more on this later), but the improvements we saw from Lumen alone were compelling enough for us to consider. Additionally we saw a lot of workflow improvements on the horizon when it comes to open world systems, asset management, and overall developer productivity.
Courtesy of Inflexion Games
Finally, with our project moving towards an Early Access and games-as-a-service lifecycle, we knew it would only be a matter of time before our player community would push us to move our technology and visuals forward.

Once we determined our motivation, we considered the overall cost to upgrade. Epic had estimated the cost of upgrading from UE4 to UE5 to be about four times the cost of working through a minor release upgrade of UE4. We had been spending about one to two calendar months on a minor upgrade, so we estimated about six calendar months. In the end, this estimate was spot on. 

Beyond this, the simplest way we found to identify the hurdles was to simply start the work and review the issues that arose. In our case, the major issues came from migrating the physics engine to Chaos physics, but given that we didn’t have too many custom physics features in our game, we found this risk to be manageable.
Courtesy of Inflexion Games

Processes to manage an engine upgrade

If you have a Blueprint project, upgrading the engine should be very straightforward: there is no code to update, you simply load your project in the new engine version, see what asset errors come up, and fix what the engine is complaining about.

Upgrading a code project is more involved, and is what we will cover here. The key is how heavily you’ve customized the engine. If you’ve left the engine unmodified, this should again be very straightforward to upgrade; however the effort increases quickly if you’ve made significant modifications to the engine. We fell somewhere in the middle, and the following practices helped us manage the upgrade process.
Managing engine divergences
It helps to mark up and track your changes to the engine, as this will simplify the process of merging code conflicts. For Nightingale, we follow this process:
  • All changes to the engine are reviewed by a group of approvers, for us these are the same programmers who handle the engine upgrades so they can call out changes that would be difficult to maintain.
  • We encourage as many changes as possible to be pushed back to Epic, see Contributing to the Unreal Engine. This allows us to remove code divergences when taking a future release.
  • All changes are tagged in code, and tracked with a Jira ticket. For example:

// NTGL_BEGIN - NTGL-1234 - [IMPROVEMENT] Fixing an uninitialized variable
// int IntVar;

int IntVar = 0;
// NTGL_END

 
  • Divergences are categorized as follows:
    • [IMPROVEMENT] - change is suitable to be sent to Epic
    • [TEMP] - the change is needed only temporarily, usually until the next major engine update
    • [PREPORT] - the change was pulled from a future update
    • [DIVERGENCE] - we need to make this change to ship our game, but it’s not a suitable change to be pushed back to Epic (we generally want to avoid making these changes)
  • The Jira ticket allows us to track additional context around the change, such as links to UDN, links to other tasks that required the change, and tasks required to remove the change.
  • When a divergence can’t be easily merged with an engine update, we typically drop our changes and re-implement them around the new code, or abandon our changes entirely.  
Integration process
Updating the engine can initially be as simple as obtaining the update from Epic’s Github, replacing everything under the Engine folder, and reconciling offline work through P4V; however, this will quickly become unmanageable as you make changes to engine code.

Leveraging Perforce’s stream depots, we established the following branches:
 
 
   Depot / stream    Description

//ue4/release-4.27

//ue5/release-5.0

Unmodified mirrors of the UE4 / UE5 releases
//ntgl/dev
Main development stream, where most of your team is working
//ntgl/integrate
Child stream of dev, where you will be merging engine updates
  • This stream is expected to get into a broken state until all issues are resolved

Before beginning the upgrade to UE5, you will want to establish revision history by merging from the final version of UE4 (//ue4/release-4.27) to your integrated stream. If you’re not already on 4.27, you might be thinking about jumping through multiple versions at once. In our experience it’s always easier to upgrade incrementally one version at a time.

You would then establish your release-5.0 branch as follows:
  • p4 copy //ue4/release-4.27 -> //ue5/release-5.0 (initially created as a copy of 4.27)
  • Update the contents of //ue5/release-5.0 from the official release

The update and merge steps now become:
  1. Merge all changes from your development stream to an integration stream
    • If there are ever any conflicts with content changes, always accept what comes from dev so that you’re not losing any work from your content creators
  2. Mirror the desired Unreal release into a separate Perforce depot (for example //ue5/release-5.x)
    • This mirror should remain unmodified and serves to track the engine changes in Perforce’s revision history to help with the merge step
  3. Merge/integrate from //ue5/release-5.x -> //ntgl/integrate
  4. Resolve merge conflicts and submit

These steps can all be done manually, but greatly benefit from a bit of effort to automate through scripting.

After merging the update you can break down the task of an engine upgrade into these discrete steps:
  1. Fix warnings and errors related to code project generation
  2. Fix compile and link errors
  3. Fix editor and cook errors
  4. Fix autotesting failures
  5. Fix game bugs
Automation
Investing in automation can really help to streamline the upgrade process. Ideally, you would have a repeatable build process that can be run against your integration stream, and validate that everything is working before your merge to your dev stream. Some additional areas that helped us:
  • Investing in and maintaining a comprehensive suite of automated tests can really help to catch a lot of issues before you need to even do any manual testing. In our case with our initial move to UE5, our automated tests were able to flag a lot of inconsistencies between the physics engine from UE4, and UE5’s Chaos physics.
  • Writing scripts can greatly speed up the process of mirroring releases, and the various steps for merging.
  • Adding additional automation checks to run asset validators and compile all Blueprints (-run=CompileAllBlueprints) before cooking your game will help to catch additional issues in your assets.
  • Frequently running your integration stream through the exact build process your dev stream undergoes helps make sure that any hidden edge cases are caught before your final merge.
  • Investing in systems to distribute test builds of your integration stream to allow for larger-scale playtests can help expose game issues early.
Tips and tricks
These are an assortment of tips we follow to streamline the process of going through the upgrade.
  • When taking a major engine upgrade, most files will automatically resolve but you will be left with a changelist containing tens of thousands of changes. The following p4 command will let you separate out the files that need to be manually resolved into a separate changelist:

p4 -F %localPath% resolve -n | p4 -x - reopen -c [new changelist #] 
 
  • When fixing compile and link errors, compile with -DisableUnity. This will turn off the compile optimization in UnrealBuildTool which combines many .cpp files into one. Compiling will take longer, but tracking down compile issues will be much easier, and this will also let you catch issues relating to missing include statements.
  • If you get Unreal Engine through github, you will need to decide how to populate the files obtained from GitDependencies.exe. You can either have developers update these files individually, or in our case we incorporated these files into the mirror automation scripts, submitting these files into our own Perforce mirror.
  • Pay close attention to Unreal Engine’s deprecation warnings as these often point you in the right direction of upgrading your project. Maintaining a 0 warning compile makes it much easier to keep on top of these deprecations.
  • Including all your assets in your integration stream means you have the ability to upgrade individual assets if needed. Although we didn’t need to do a lot of this, it can be cumbersome to manage if those assets are getting modified regularly in your main development stream. A few suggestions to help with this:
    • If you can cherry-pick some minimal engine changes into your dev stream, and update the assets over in dev, this will eliminate the possibility of running into conflicts.
    • If the assets can be upgraded using a commandlet, you can build a commandlet that runs after each merge to ensure all relevant assets are kept up to date. In many cases the asset simply needs to be resaved in the latest version of the editor.
  • Epic allows you to get access to the release branches even before the release enters an early preview state. This enables you to get a head start on merging an upcoming release, and to narrow the scope of how much code you merge at one time. Similarly, merging daily from your dev line enables you to intercept difficult merges right as they land so you can work with the relevant developers to address the issue.
  • If you rely on third-party plugins (from Marketplace or from other vendors), you’re likely better off to wait a bit until those plugins get updated. We often want to upgrade ahead of these updates, which means spending additional effort to update the plugins that you would otherwise get for “free” by waiting a bit longer.
  • With every new engine upgrade comes some really useful features. As tempting as it is to begin taking advantage of new features immediately, our goal is always to initially push a new engine update with as few meaningful changes to the game or developer workflows as possible. Only after we determine the upgrade to be stable in our dev stream do we start trying out new features.
 

Benefits of Unreal Engine 5

Upgrading to Lumen and Nanite was an exciting prospect. Who wouldn’t want more accurate bounced light and the ability to significantly increase the poly budget for your game?
Courtesy of Inflexion Games
As easy as both Lumen and Nanite are to turn on, it took some time to prepare Nightingale, which had initially been optimized to launch on UE4. Nightingale was already strictly adhering to PBR rules across all components of our pipeline, so Lumen made our game shine even more quickly after we had integrated it, while additional optimizations could be realized after converting our meshes over to Nanite.
Courtesy of Inflexion Games
Converting our project to Nanite took longer as we also wanted to take advantage of Virtual Shadow Maps. With Unreal Engine 5 supporting interoperability of Nanite and non-Nanite assets we were able to handle the conversion gradually, which enabled us to have a fully playable experience as we transitioned. That being said, you can expect to see improved performance with Virtual Shadow Maps as more assets get converted to use Nanite, so there’s some incentive to work through this conversion to understand how your game will perform.

The conversion process for an individual asset tended to fall in one of these categories: 
  • Some assets were already high enough fidelity and we simply enabled Nanite on the mesh. 
  • Some assets would benefit from being reauthored at higher fidelity, so we rebuilt these with Nanite in mind. 
  • Our player-built structures had a custom shader to render with transparency as the player is placing a structure. Since Nanite is only supported with opaque objects, we implemented a system where we would fall back to a non-Nanite mesh with transparency when the mesh was in this special placement mode.
Courtesy of Inflexion Games
Another area that really helped our workflows was with the introduction of Level Instances and Packed Level Actors. On UE4, we struggled with various implementations of prefabs but the introduction of Packed Level Actors unlocked new possibilities with how we procedurally assemble environments in Nightingale.
Courtesy of Inflexion Games
As the versions progressed from 5.0 through 5.2, we saw improvements across many of the features of the engine: Volumetric Cloud rendering was improved, Foliage rendering with Lumen was improved. In the end, the combination of Nanite, Lumen, and Virtual Shadow Maps make for some fairly impressive visuals.
Courtesy of Inflexion Games
We’re very pleased with how Nightingale looks after the evolutionary process it has gone through with UE5. The fidelity we have been able to achieve wouldn’t have been possible without the Lumen and Nanite, and we’re looking forward to pushing this even further with future updates to Nightingale and to Unreal Engine 5.

    Download Unreal Engine 5 today!

    If you’re an existing Unreal Engine user, you can download Unreal Engine 5 from the Epic Games launcher. If you're looking to dive in for the first time, click the link below to get started. Either way, we hope you enjoy all the new features and upgrades, and as always, we encourage your feedback!
    Conteúdo gratuito em destaque da Loja da Unreal Engine - Fevereiro de 2024
    Notícias
    6 de fevereiro

    Conteúdo gratuito em destaque da Loja da Unreal Engine - Fevereiro de 2024

    Interiores de fantasia, minas de cristal estilizadas, uma viagem de volta aos anos 90 e muito mais. Chegaram os ativos 3D gratuitos de fevereiro da Loja da Unreal Engine!
    Conteúdo gratuito em destaque da Loja da Unreal Engine - Fevereiro de 2024
    Notícias

    Conteúdo gratuito em destaque da Loja da Unreal Engine - Fevereiro de 2024

    Interiores de fantasia, minas de cristal estilizadas, uma viagem de volta aos anos 90 e muito mais. Chegaram os ativos 3D gratuitos de fevereiro da Loja da Unreal Engine!
    Promoção de fevereiro na Loja: mais de 2.000 produtos com 50% de desconto
    Notícias
    5 de fevereiro

    Promoção de fevereiro na Loja: mais de 2.000 produtos com 50% de desconto

    Cafeterias retrô, interiores rústicos, cidades com baixa contagem de polígonos, PNJs personalizáveis: a promoção de fevereiro na Loja tem algo para todos. Compre agora para obter os itens selecionados do mês pela metade do preço.
    Promoção de fevereiro na Loja: mais de 2.000 produtos com 50% de desconto
    Notícias

    Promoção de fevereiro na Loja: mais de 2.000 produtos com 50% de desconto

    Cafeterias retrô, interiores rústicos, cidades com baixa contagem de polígonos, PNJs personalizáveis: a promoção de fevereiro na Loja tem algo para todos. Compre agora para obter os itens selecionados do mês pela metade do preço.