WebGPURenderer: Only use HalfFloatType for the framebuffer target if necessary.

[Mugen87]

Related issue: #30387 (comment)

Description

This was found during investigating performance behavior in WebXR.

Right now, WebGPURenderer always creates a RGBA16F framebuffer for the beauty pass. Using HalfFloatType is not always required though. If we switch to UnsignedByteType whenever possible, we can improve renderer performance especially on mobile GPUs.

The idea is to identify HDR workflows by checking toneMapping and introduce a new method that allows backends to force HalfFloatType if they require it (e.g. when outputType is set to HalfFloatType).

As a reminder: The internal framebuffer target is only relevant when rendering directly to screen since it is the input for the internal tone mapping and output color space conversion pass. It is not used when rendering into a render target e.g. when doing post processing.

[github-actions]

📦 Bundle size

Full ESM build, minified and gzipped.

	Before	After	Diff
WebGL	336.27 78.29	336.27 78.29	+0 B +0 B
WebGPU	512.43 142.36	512.68 142.41	+258 B +55 B
WebGPU Nodes	511.89 142.25	512.15 142.31	+258 B +55 B

🌳 Bundle size after tree-shaking

Minimal build including a renderer, camera, empty scene, and dependencies.

	Before	After	Diff
WebGL	465.25 112.13	465.25 112.13	+0 B +0 B
WebGPU	586.36 159.02	586.61 159.08	+256 B +61 B
WebGPU Nodes	541.74 148.57	542 148.63	+256 B +62 B

[Mugen87]

Side note: When using UnsignedByteType, it is in general recommended to use a sRGB storage format for the framebuffer for better storage precision (see https://stackoverflow.com/a/47098507/5250847). However, these framebuffer formats are buggy in WebGL and do not consistently work across all platforms.

[sunag]

I'm not sure about this, nodes like viewport*Texture() make use of this, which may imply the use of refraction. This may also degrade the quality of color space conversion.

[Mugen87]

I've checked the demos using the viewport textures and did not see any visual regressions so far. The precision of a 32bit color buffer should be okay as long as the colors do not exceed the [0,1] range. If that is the case, the application should configure tone mapping in the first place.

I understand the considerations about quality but the performance implications of always using 64 bit color buffers is something to think about as well. Mobile devices tend to be quite bandwidth limited especially when rendering with full native resolution. If we always render into a 64bit color buffer, we will face performance issues on "weaker" mobile devices and mobile XR since these render in very high resolutions. This is also an issue in context of post processing.

The WebXR integration into WebGPURenderer revealed some severe performance issues on mobile devices. The switch to an intermediate render target for tone mapping and color space conversion produces more overhead than expected. We need a solution that is more bandwidth conservative and the usage of a 32 bit color buffer for non-HDR workflows is a first step in that direction I think.

In any event, I'll move the PR to the r174 milestone so we have more time to think about the change.

This may also degrade the quality of color space conversion.

Maybe it would be good to figure out a live example that demonstrates potential precision issues.

[sunag]

Maybe this is not noticeable because most examples use tone mapping or post-processing or do not have much HDR contrast, but if you do this test on webgpu_backdrop_water disabling post-processing the change is quite noticeable.

Using HalfFloatType

Using UnsignedByteType

The version using UnsignedByteType gets the contrast and colors washed out while the HalfFloatType version preserves the contrast and colors.

[sunag]

The WebXR integration into WebGPURenderer revealed some severe performance issues on mobile devices. The switch to an intermediate render target for tone mapping and color space conversion produces more overhead than expected. We need a solution that is more bandwidth conservative and the usage of a 32 bit color buffer for non-HDR workflows is a first step in that direction I think.

It seems like a very relevant point, I'm ok with the PR if it improves performance. Mainly because we can use it the previous way with a parameter.

[Mugen87]

@donmccurdy Regarding the visual regression in #30392 (comment): What do you think about using a 32 bit color buffer when no tone mapping is configured and outputType is not set to HalfFloatType?

Can you imagine other use cases where this approach produces more severe visual errors? TBH, I did not expect the difference is so visible like in the modified version of webgpu_backdrop_water.

I wonder if we should introduce a separate renderer flag instead of using 32 bit automatically 🤔 . The XR demos could set this flag to true so all other applications are not affected.

[donmccurdy]

Personally I don't feel very comfortable with trying to automatically guess what precision is required in the framebuffer target, based on the presence or absence of tone mapping. A lit rendering workflow should use tone mapping to produce a picture — whether RGB values in the buffer happen to exceed [0,1] does not change this.

I'd be OK with exposing an option to manage the type of that internal buffer, and/or with providing an option to skip the intermediate buffer and render directly to the WebXR render target.

Tone mapping is an essential part of a lit rendering workflow, so I think it's important that tone mapping be possible on mobile and XR devices with WebGPU — even if that requires the same unfortunate limitations we have had in WebGLRenderer where alpha blending is in sRGB space.

[Mugen87]

Closing. A flag that controls the precision of color buffers sounds indeed more appropriate. The default could be 64 bit but it could be set to 32 bit in scenarios where memory and bandwidth are limited.

The flag could then be evaluated in the renderer and PassNode so all components that produce beauty passes honor it. Using a 32 bit buffer might limit the possibilities in FX workflows and degrade quality, but tradeoffs are inevitable.