March 3, 2020
How Epic Games is handling auto exposure in 4.25
While we do provide an upgrade path that should maintain visual consistency in the look of your projects, it does break backward compatibility with the existing auto exposure system in previous versions of Unreal Engine.
Throughout this blog, we’ll explore the pain points of the current system and why we made these changes. We’ll also explain how the changes affect auto exposure (for the better) going forward.
Histogram and Basic Algorithm
The Histogram and Basic exposure algorithms have not been as intuitive for our users as we would have liked, nor have they always been the best at behaving in expected ways. To understand these changes and why we made them in Unreal Engine 4.25, let’s look at how they have worked previously:- The Basic algorithm uses the average of the log luminance of the scene to determine the metered exposure.
- The Histogram algorithm—instead of using the entire scene—discards the pixels above and below the histogram target range. This method takes the average of the log luminance of the remaining pixels to determine the target exposure.
These algorithms produced different results based on the exposure settings of the Post Process Volume. In order to determine how these two algorithms should best behave in an expected and intuitive way, we came up with two rules:
- Grey should be grey. This means that if we have an 18% grey card lit with even lighting (or an emissive surface at 0.18 cd/m^2), the target color on the screen should be 0.18.
- Full Histogram should match Basic. This means that if we set the histogram’s low and high range to 0–100%, it should look the same as the Basic exposure algorithm.
With these rules as guiding principles, let’s look at an example of why these algorithms and their properties caused much confusion and how we’re improving it in Unreal Engine 4.25.
In 4.24, if we start with a scene that has a single surface that can be measured at 0.18 cd/m^2, and we set the histogram range to be 1–99%, we get the following results in the Level viewport:
(Left) Histogram Mode (RGB 254), UE 4.24; (Right) Basic Mode (RGB 111), UE 4.24
The Histogram exposure algorithm converges to nearly white (measured RGB 254), while the Basic exposure algorithm converges to grey (measured RGB 111), which is 16% grey in gamma 2.2 space.
So, why does this happen and how are we improving it for 4.25?
For the Histogram mode, the original intention was that you would not want to adjust for the middle of the histogram. Rather, the default values are between 80–98%, making the average of your histogram the White Point.
For the Basic mode, the calculation of the average scene luminance is used for the Grey Point, and that is defined by the Calibration Constant, which has a default of 16% grey.
In isolation, both interpretations are reasonable. It is sensible to have a histogram exposure adjusted for the white point and a full scene mode adjusted for the grey point, but it is confusing to have the similar modes share parameters that produce such different behavior to one another.
Through our investigations, our options were:
- Do nothing and let the Auto and Histogram modes behave differently.
- Make them the same, but add flags and parameters for backwards compatibility.
- Make them unified, but break backwards compatibility.
We elected for option number three.
Option one is problematic because artists would still be faced with confusing, unintuitive parameters. Option two would create even more unintuitive parameters, compounding the problems we already have. Unifying these approaches with option three enables us to reduce complexity and confusion, while making the tools more intuitive for artists to use.
In Unreal Engine 4.25, both the Histogram and Basic exposure modes converge to the grey point. This means that for the Histogram mode, we are now interpreting the average of the Histogram as the grey point. For the Basic exposure mode, we have removed the calibration constant.
These are the results of these changes to auto exposure in 4.25:
(Left) Histogram Mode (RGB 116); (Right) Basic Mode (RGB 117)
Both the Histogram mode and Basic mode converge to the same middle grey when measuring their RGB values (ignoring slight differences due to roundoff error). Both are within a single value of 18% grey when using the following formula:
(0.18^(1.0/2.2))*255=116.95
Note: If you do want the Histogram to go towards the white point, you still can do this by changing the Exposure Compensation parameter.
Changes in scale from 1.2 to 1.0
The exposure values are interpreted as EV100 in Unreal Engine. So, what is EV100? It’s complicated, but you can read about it in more detail here.Given a luminance value in cd/m^2, the EV100 value of that light value is usually calculated by the formula:
EV100 = log2(luminance/1.2)
For many users, this only added to confusion because many prefer to work in “unitless” environments, where an emissive value of 0.18 should appear on screen as 0.18 (accounting for gamma correction). And to do this in previous versions, you had to use the unintuitive Exposure Compensation value of log2(1/1.2) = -0.263.
This extra bias caused much more confusion than it solved, and has been removed by default going forward. However, you have the option to add it back in, if needed.
The Physical Basis of 1.2
Why did we have the value of 1.2 in the EV100 formula used with auto exposure in previous Unreal versions?
The formula is defined by the ISO standard: 12232:2019, which we use as the physical basis of our cameras. In the real world, lenses are not perfect, and not all the light that goes into the lens actually hits the sensor. Most applications assume that the lens has a transmittance of 0.65, meaning that 65% of the light that enters the lens hits the sensor with the remaining 35% lost to absorption, inter-reflections, and vignetting.
Under this ISO standard, at a saturation based speed of ISO 100, a lens would saturate at the value of 0.78/q (where q is the lens transmittance). Since most applications use q=0.65. This gives us the following result:
0.78/0.65 = 1.2.
However, Unreal Engine’s virtual lens does not actually behave that way, because it does not lose light to interreflections or absorption. So, we have decided to make q a user-defined variable that can be controlled with r.EyeAdaptation.LensAttenutation.
Note: By default, Unreal Engine’s lens attenuation is set to 0.78. At a value of 0.78, the math cancels out and you can interpret the lighting as unitless. If you wish to maintain the previous 1.2 scale, you can set the lens attenuation variable to the previous implicit value of 0.65.
Improved Physical Camera Parameters
Another area of confusion has been around the physically based camera parameters. When using the Manual metering mode (available in the Post Process Volume and Camera settings), the Exposure/ISO/Aperture/FrameRate would be taken into consideration so that the manual camera would behave similarly to a real-world physical camera.The downside is that this approach created workflow issues. A common use case would be that a scene would be set up with auto exposure and achieve a reasonable look. The Histogram mode (bottom left) would look correct. When switching to Manual mode (bottom right), the exposure would become overly dark.
(Left) Histogram Exposure Near 0; (Right) Manual Exposure Near 10
Because the auto exposure is accounting for the camera’s parameters when using Manual metering mode, it causes the scene to become dark. The original intention of this mode was to make it easier to adjust real-world lighting with accurate aperture and depth of field values that mimic physical cameras. However, the downside is that it is impractical to use manual cameras in the same scene as automatic ones.
To improve this workflow and to make it more explicit, we have added a new flag to the Post Process Volume: Apply Physical Camera Exposure. It enables you to control whether physical camera parameters affect auto exposure or not. This means that a manual camera with an exposure compensation of 0.0 will exactly match an auto exposure scene that is set to zero when this property is disabled.
New Exposure Metering Mask
To further give artists control over auto exposure in their environments, we’ve added a Material slot for adding your own exposure metering mask. It enables artists to create their own texture that covers the whole screen controlling where the influence of each pixel is weighted in importance by the texture mask.
(Left) Exposure Metering Mask Material slot in the Post Process Settings; (Right) Example Exposure Metering Mask Texture
The example metering mask above enables you to give more influence to pixels towards the center of the screen rather than along the edges. This helps stabilize auto exposure. For example, bright objects appearing at the edge of the screen can potentially cause an abrupt change in the expected auto exposure value. This type of mask reduces the influence of samples towards the edge, allowing for a more stable and less jittery auto exposure result.
(Left) Using Whole Screen Exposure; (Right) Using an Exposure Metering Mask.
Note: We have removed the console command r.EyeAdaptation.Focus, which would either apply weighting to give values in the center more influence, or allow you to apply weighting uniformly across the screen. While this parameter functioned similarly to this new exposure metering mask, it would only affect the Auto Exposure Basic metering mode, and not Histogram mode. The new exposure metering mask offers artists finer control that works consistently across Basic and Histogram modes.
Improved HDR Visualization Mode
Continuing to improve the auto exposure workflow means that we could also improve our debugging and visualization mode.NOTE: You can enable this view from the Level viewport by going to Show > Visualize > HDR (Eye Adaptation), as well as from the console using the command ShowFlag.VisualizeHDR 1.
- The blue line is the target EV100 exposure value for the view.
- This is where the average scene exposure is at in the histogram depending on whether you’re using Histogram or Basic exposure metering mode.
- The purple line is the actual EV100 exposure value in the current scene view.
- This is the current exposure that exists in this view. It will always be moving towards the blue line.
- The white line is the final EV100 exposure value after adjusting the exposure compensation.
- This is the actual exposure value in the display after exposure compensation has been adjusted in the Post Process parameters. The white and purple lines will maintain a difference in spacing based on the value set in the exposure compensation. In this example, the white and purple lines will maintain a 2.0 difference in the histogram.
Note: The HDR (Eye Adaptation) visualization mode must be already enabled. Then, use the console variable r.EyeAdaptation.VisualizeDebugType to switch the debug mode. 0 (default) gives you the scene color after tone mapping, and 1 gives you the histogram debug mode.
The histogram debug visualization in this mode ensures that very bright pixels (like the sun) and very dark pixels (like deep shadows) are not involved in the HDR calculation that is happening. In this scene, anything below the target histogram range will be shown in red, anything above the range will be in blue. It ensures that the high and low percentile ranges that we’ve set are removing the unwanted pixels from being calculated.
Exposure Compensation Curve
We’ve also made updates to the Exposure Compensation Curve this release to be more clear about where the X and Y-axis values are taken from.With that in mind, it is understandable that the artists want a different exposure compensation while in bright areas versus dark areas. By assigning a Curve Asset in the Post Process Volume, they are able to achieve a desired look while also having finer control of which values are affecting exposure compensation in their environment.
(Left) Post Process Curve Asset Assignment; (Right) Curve Asset
Note: Add a Curve Asset to your project using the Content Browser by clicking the Add New button and navigating to the Miscellaneous category to select Curve. Then select CurveFloat.
The HDR Visualisation mode contains all the information found in the curve that represents the X and Y-axis. The Average Scene EV100 value (1) and the blue line (the actual exposure) value in the Histogram (1) represents the X-axis found in the Curve Asset. The Exposure Compensation (Curve) represents the Y-axis of the curve, which is added with the Exposure Compensation (Settings) value to get the sum value of Exposure Compensation (All).
- The Average Scene EV100 is the calculated target EV100 value, and is also represented by the blue line in the Histogram.
- The Exposure Compensation (Settings) is the exposure compensation set in the Post Process Volume.
- The Exposure Compensation (Curve) is the resulting Y value from the Curve asset.
- The Exposure Compensation (All) is the sum of the previous two values, resulting in the final exposure compensation.
By splitting the exposure compensation into a “settings” and “curve” value, you are given more precise control over the settings as well as a clearer understanding of why the exposure compensation is behaving a certain way.
Exposure Change Speed
There have been several overarching changes to how the exposure adjusts from the current value to the target exposure value. You can see this change in the histogram view by following the purple line (the current exposure value) adjusting to the blue line’s (the target exposure value) position over time.
Log-space Adjustment
The first significant change is that all exposure movement now happens in terms of F-stops, as opposed to linear values. It applies exposure adaptation in logarithmic steps so that it perceptually moves up and down at the same speed, which the old settings (demonstrated in the graphs below) did not do.For instance, using the old setting, if we had a current linear average of 1.0 and were moving to a target of 1,000, it would get to the halfway point of 500.5 after two seconds.
Similarly, if going the opposite direction at the same speed, we would start at 1,000 and move down to 1.0, hitting the halfway point in two seconds.
Intuitively, this behavior makes sense, but it becomes problematic when you think about it in terms of perception. Linearly, the same curve is being followed each way, but we perceive the intensity in log-space, such as F-stops. For example, going from 1.0 to 500.5 is about 8 F-stops, but going from 500.5 to 1000.0 is 1 F-stop.
In this case, when moving up, we move about 8 F-stops in the first two seconds and then 1 F-stop for the rest of the curve. However, when moving down, we move only 1 F-stop in the first two seconds, and 8 F-stops over the remaining time. This behavior creates the perception that exposure is shifting very quickly when going up to a target luminance and very slowly when coming down to a target luminance. The change for Unreal Engine 4.25 makes the exposure adjustment perceptually linear over time.
New Linear and Exponential Movement
In Unreal Engine 4.24 and prior versions, all exposure adjustment was exponential. This behavior caused two secondary issues:- It caused jittery behavior because the auto-exposure was quickly adapting to small fluctuations in luminance.
- Large luminance transitions were always fast, without a practical way to slow down the transition. It was not possible to make a slow transition from a dark to a bright area.
To address these issues, the new algorithm starts with a linear curve (2) when far away from the target exposure value but switches to an exponential curve (1) when it is closer to the target value.
Using the previous example (in the Log-space Adjustment section), the initial movement is linear when adapting to a brighter scene. The Speed Up and Speed Down parameters in the Post Process are now measured in units of F-stops per second.
As we approach the target exposure, the movement switches from linear to exponential using an exponential curve that maintains continuity of the first derivative. You can control this value by using the console variable r.EyeAdaptation.ExponentialTransitionDistance. This variable sets the linear to exponential transition at r.EyeAdaptation.ExponentialTransitionDistance F-stops away from the target.
The default exponential transition distance is 1.5 F-stops. The speed up and speed down will adjust the exposure linearly until it is 1.5 F-stops away before switching to exponential movement.
This change in movement provides some key advantages over the previous behavior:
- The controls are less jittery. Since the adaptation is linear when far away from the target exposure, it avoids sudden jumps when bright spots appear on screen.
- These parameters provide more precise control when slower adaptation is desired.
Mobile Parity
Thanks to the efforts of our Epic Games China mobile team, we have been able to add support and optimize auto exposure for all of our supported mobile platforms in Unreal Engine 4.25. This means that mobile is now at feature parity with our desktop and console platforms, supporting all three auto exposure modes: Histogram, Basic, and Manual.Note: Auto Exposure parity on mobile requires support for ES 3.1. With Unreal Engine 4.25, support for ES2 is removed from the engine.
New Auto Exposure Defaults
The new algorithm changes make the default values different for the Histogram exposure mode. Previously, it used 80% low percentile and a 98.3% high percentile range that was designed to ignore most of the Histogram and only focus on the highlights. The new default values range from 10% low percentile and 90% high percentile, applying a result that is similar to the Basic exposure mode for consistency.After making these changes, we found that the default Exposure Compensation value of 0.0 looked too dark for typical scenes. So we changed the default Exposure Compensation value in new Post Process Volumes to 1.0. However, you have the option to change that default with the “Auto Exposure Bias” project setting.
If you decide that you want to mimic the behavior of Unreal Engine 4.24 or a prior version, you can change the Histogram range to be 80% and 98.3%. Then, apply an exposure compensation of log2(1.0/0.18)=2.47.
Both versions are reasonable and one may work better for your project. However, the new exposure defaults provide a wider histogram that provides better consistency between different exposure modes by default.
Auto Updating Existing Projects and Content
Lastly, these changes left us one final, tricky problem to solve. How do we handle existing content when upgrading to Unreal Engine 4.25?Ideally, we want content to look the same from one version of the engine to the next, but in this case, that option was not practical. The approach we have chosen is to apply an automatic update to exposure compensation when you upgrade your project to 4.25.
The logic for the update is stored in CalculateEyeAdaptationExposureVersionUpdate() in Scene.cpp in your Engine\Source\Runtime\Private folder.
The main change to consider here is that if your project has Histogram exposure enabled, and you have extend default luminance range enabled in your project settings, we apply a small exposure compensation to adjust for the changed behavior. For example, if the average of the min and max histogram is 89.15, an exposure compensation of 1.5 is applied. However, if you have a wider Histogram, less exposure compensation is applied. With a min and max average of 50, an exposure compensation of 1.0 will be applied. While we would have liked to guarantee an exact match of previous settings, such a conversion is not possible because it will depend on the content of the scene. We found that these defaults provided reasonable values for existing content.
Note: If you should find that the conversion is problematic for your project, your original exposure compensation value has been stored in the hidden value AutoExposureBiasBackup.
Conclusion
While these changes may break backwards compatibility and change the look of an existing project, our hope and goal is that it makes Auto Exposure much easier to work with for artists and developers going forward. We strive to continually improve and update our tools and sometimes this means making broader changes that have a higher rate of impact that we would normally be comfortable with.Hopefully, the changes here provide you with a more user-friendly interface and intuitiveness than has previously been available. Look for upcoming changes to the documentation for Auto Exposure to cover these topics and changes, as well.