Introduce postprocessing API

[dmarcos]

This is a proposal for a built-in post-processing API. Setting up post-processing in a comprehensive manner that works well in 2D and VR modes is not trivial to do. It requires tight integration with the render loop. A built-in effects API will help raise the visual quality of the content and will make very easy for everybody to experiment with the different effects while ensuring good performance. At first we would not have a public API for user defined effects since many are not suitable for stereo rendering. By providing the effects we can control how and what order are applied to guarantee consistent results and FPS. Once we settle on an API we can consider open it up.

The API is built by defining effect- prefixed scene level components for each effect. The order they are applied is fixed and independent on how and when they are defined. e.g:

  <a-scene effect-bloom="strength: 1.6" effect-sepia background="color: black">
      <a-box color="white" position="0 1.6 -2"></a-box>
  </a-scene>

This PR just contains a bloom and sepia effects but I will expand before merge. I would love to hear what effects people want to see first.

[ngokevin]

This PR just contains a bloom and sepia effects but I will expand before merge. I would love to hear what effects people want to see first.

SSAO please!

[ngokevin]

Having functionality based on a prefix and HTML attribute ordering is a bit magic. I would define the order effects: effects-bloom, effects-sepia.

[dmarcos]

The order does NOT depend on attribute position. The order is fixed and will be documented. e.g: 1. tint, 2. vignette, 3. sepia, 4. bloom... it won’t be user configurable

[dmarcos]

Sketchfab also has a fixed effects order: https://help.sketchfab.com/hc/en-us/articles/115000728943-Post-Processing-Filters?mobile_site=true

[feiss]

I would love to hear what effects people want to see first.

Santa is coming! :D My wish list, in order: SSAO, Screen space reflections, Chromatic Aberration, Noise/grain.

[zomboh]

This is awesome and will give webVR visuals a much needed push! feiss's list is pretty much it, I think those are the most needed (SSAO above all). As a super low-priority one I'd suggest a glitch/digital distortion effect.

[TechThomas]

I would love to see effects related to simulations of how people would see aframe experiences with various vision problems. Filters for viewing red/green color blindness would be the first one I would appreciate! Reference: Be Kind to the Color Blind

[vylevylevyle]

THANK YOU!!!!! SSAO, Screen Space Reflections, Chromatic Abberation, Motion Blur would definitely be wanted. PLEASE PLEASE ADD DoF for 2D!!!!!

[alfa256]

Looks great, color grading/tonemapping/color curves or however you like to call it would be high on the list for me.

[musicscene]

Based on some basic paint.net effects I like, my favorites are:

Artistic - Ink, Oil, and Pencil
Blurs - Gaussian, Motion, Radial
Distortion - Crystalize, Tile Reflection, Dents
Style - Edge Detect, Emboss

All should be animatable! :)

[arpu]

looks very nice but why not use the soucecode from Threejs examples as import ?
like the https://github.com/mrdoob/three.js/blob/dev/examples/js/postprocessing/UnrealBloomPass.js

[arpu]

it would be possible with the webpack three plugin
mrdoob/three.js#9562 (comment)

[arpu]

or maybe using this threejs https://github.com/vanruesc/postprocessing
i only add some findings to this PR

some more shaders
https://github.com/libretro/glsl-shaders

[arpu]

@dmarcos mrdoob/three.js#15840

[Francois-Esquire]

Please let me know if I can help with this!

[Brian-DP]

Hello, is this project still going on or did you drop it? We'd like to integrate post processing into our application (mainly for ssao and aa) and we'd like to know if it's possible or you stopped working on it because it's not feasible.

[samyH]

Any chance you could release the API just with bloom & sepia working? Been trying to get this working using aframe-effects.js to no avail :(

Thanks for all your hard work on making A-frame as great as it is!

[Brian-DP]

I'm trying to implement/adapt the SSAO pass from the three.js example, but I cant seem to be able to get the depth texture. This is the snippet of my code in the render function:
this.quad.material = this.basic; this.basic.map = readBuffer.texture; renderer.render(this.scene, this.camera, this.beautyRenderTarget, true); this.depthRenderMaterial.uniforms[ 'tDepth' ].value = this.beautyRenderTarget.depthTexture;

The render target is initialized as following:
this.beautyRenderTarget = new THREE.WebGLRenderTarget( this.width, this.height ); this.beautyRenderTarget.texture.format = THREE.RGBFormat; this.beautyRenderTarget.texture.minFilter = THREE.NearestFilter; this.beautyRenderTarget.texture.magFilter = THREE.NearestFilter; this.beautyRenderTarget.texture.generateMipmaps = false; this.beautyRenderTarget.stencilBuffer = false; this.beautyRenderTarget.depthBuffer = true; this.beautyRenderTarget.depthTexture = new THREE.DepthTexture(); this.beautyRenderTarget.depthTexture.type = THREE.UnsignedShortType;

I tried to render the depth texture but it's all black. What am I doing wrong?

[Dante83]

This is definitely something that's important and I wanted to poke this again. It's critical for something like water, where you want to have subsurface light scattering based on depth buffer as well as reflection from the surface. Three.js, however, requires a modification to the core rendering loop to access the post-processing code in this. Ideally, it would allow the render passes to be done in a user specified order. It would be a perfect place to have a kind of,

<!-The order of the elements could then tell you the order of applying the effects, top to bottom.->
<a-effects>
     <a-effect effect-ssao></a-affect>
     <a-effect effect-custom></a-effect>
     <a-effect effect-bloom:"strength: 1.6"></a-affect>
    <a-effect effect-sepia></a-affect>
</a-affects>

I suppose this isn't super critical however, as long as creators can get access to the effect composer, we can always include a parameter that allows the user to specify the order of post-processing effects. I know there's a bit of a problem waiting for the whole DOM to load with this, as I've had similar issues with the experimental version of a-sky-forge which is going to a more custom XML format to avoid giant inline commenting issues, but it's much easier to read as the number of applied post processing stages can become non-trivial and being able to comment out or swap around effects like the above could be really intuitive to how the render passes are applied.

[dmarcos]

This work depends on mrdoob/three.js#15840

[Dante83]

Well at least that seems due in r112, that will be a nice Christmas present :D.

[b00lean]

This work depends on mrdoob/three.js#15840

@dmarcos it seems that mrdoob/three.js#15840 got closed.

[ryanaltair]

hi, is this pr still active?

[dmarcos]

hi, is this pr still active?

Still depends on THREE work as mentioned above

[arpu]

https://github.com/vanruesc/postprocessing works fine with threejs and aframe in webxr

wups this was not right it does not work in webxr for now

[marcusx2]

Would love to see an aframe component for this working in VR mode!

[dmarcos]

Would love to see an aframe component for this working in VR mode!

Still depends on THREE work mrdoob/three.js#18846

[jywarren]

This is happening!!! At least the postprocessing plugin has been added and is live with the "bloom" effect: Hubs-Foundation/hubs#5742

Some video about it here: https://www.youtube.com/watch?v=1-ca5qKivpY

[Dante83]

@jywarren, oh, that's helpful! I had a bloom pass for my sky, but it would be great if I could add this with specular highlights on my water in the future. Custom shader support would be particularly valuable for me as skies are meant to be post processing effects and water, too.

[vincentfretin]

I started a postprocessing page on the wiki https://aframe.wiki/en/#!pages/postprocessing.md

[Mrbside]

I think some useful would be bloom, outline, depth of field, blur, god rays as seen in postprocessing demo

[vincentfretin]

FYI model-viewer introduced a postprocessing api with tags that may be of some inspiration:
https://twitter.com/modelviewer/status/1651007361862225921
https://modelviewer.dev/examples/postprocessing/

[dmarcos]

thanks. now I would probably do:

<a-scene post-processing=“effects: bloom, sepia”>

[vincentfretin]

thanks. now I would probably do:

<a-scene post-processing=“effects: bloom, sepia”>

How would you set the parameters for an effect in this case?

For example with the new SSAO effect https://twitter.com/N8Programs/status/1658820364313452550
you can tune some parameters.

[Mrbside]

thanks. now I would probably do:

<a-scene post-processing=“effects: bloom, sepia”>

How would you set the parameters for an effect in this case?

For example with the new SSAO effect https://twitter.com/N8Programs/status/1658820364313452550 you can tune some parameters.

This would set the post effect available in the scene? but how would it be applied to selected entities? as an atribute on each entity ? and the parameters you mention for the effect would then be the same for all entities?

[vincentfretin]

r3f postprocessing integration has a similar API as model-viewer, having an effect-composer element that includes a child for each effect.
The effect-composer tag can have some parameters as well, see the demo https://codesandbox.io/s/react-postprocessing-dof-blob-pqrpl?file=/src/App.js
So in aframe this could be:

<a-scene>
  <a-effect-composer multisampling="0" disable-normal-pass="true">
    <a-depth-of-field-effect focus-distance="0" focal-length="0.02" bokeh-scale="2" height="480"></a-depth-of-field-effect>
    <a-bloom-effect luminance-threshold="0" luminance-smoothing="0.9" height="300" opacity="3"></a-bloom-effect>
  </a-effect-composer>
</a-scene>

[dmarcos]

<a-scene post-processing=“effects: bloom, sepia”>

How would you set the parameters for an effect in this case?

can be attributes as well

<a-scene post-processing=“effects: bloom, sepia” bloom-effect=“…”>

Postprocessing as html elements doesn’t feel right. Every element in A-Frame outside a-assets is an entity.

[vincentfretin]

aframe-effects was doing something similar to what you suggest @dmarcos with only attributes on a-scene.

As an alternative to the a-effect-composer tag I wrote in #3645 (comment)
it could be:

<a-scene
  post-processing="effects:depth-of-field,bloom;multisampling:0;disableNormalPass:true"
  depth-of-field-effect="focus-distance:0;focalLength:0.02;bokehScale:2;height:480"
  bloom-effect="luminanceThreshold:0;luminanceSmoothing:0.9;height:300;opacity:3">
</a-scene>

The idea I think is to write an aframe post-processing system with minimal amount of code and use directly the postprocessing library. So creating EffectComposer and all the EffectPass with the corresponding effect for each listed effects and getting the params from ${effectName}-effect attribute.

[joao-pratas-darabase]

aframe-effects was doing something similar to what you suggest @dmarcos with only attributes on a-scene.

As an alternative to the a-effect-composer tag I wrote in #3645 (comment) it could be:

<a-scene
  post-processing="effects:depth-of-field,bloom;multisampling:0;disableNormalPass:true"
  depth-of-field-effect="focus-distance:0;focalLength:0.02;bokehScale:2;height:480"
  bloom-effect="luminanceThreshold:0;luminanceSmoothing:0.9;height:300;opacity:3">
</a-scene>

The idea I think is to write an aframe post-processing system with minimal amount of code and use directly the postprocessing library. So creating EffectComposer and all the EffectPass with the corresponding effect for each listed effects and getting the params from ${effectName}-effect attribute.

would it also be possible to have a way of adding those effects only to certain entities like in the ThreeJS example ?

[marcusx2]

This PR just contains a bloom and sepia effects but I will expand before merge. I would love to hear what effects people want to see first.

Bloom per object and not per scene would be nice. More often than not I want to make only a particular object glow, not the entire scene.

[koktavy]

I agree with @vincentfretin that A-Frame should wrap the pmndrs Post Processing library, which Don McCurdy also points to here.

What steps are necessary to make that happen? Having an outline would help to make contributions.

[dmarcos]

Not sure about taking pmndrs as a dependency on core but would be cool to see if some has tried to integrate as a 3rd party component. Is there any previous work or anyone wants to tackle it?

[dmarcos]

Anyone that commented here would like to take a stab at a 3rd party post-processing component? Thanks

[dmarcos]

Closing this and can reevaluate in the future.

[enzofrancescaHM]

Hi, sorry to re-open this, but I think post-processing is a must have in 2025. I've done some experiments and tested pmndrs's library but unfortunately is working only in desktop mode, it fails in VR mode. So i've implemented the PR that Cody Bennet suggested in three (mrdoob/three.js#26160) into supermedium three and after that I've recompiled a-frame 1.6, then modifiyng akbartus component (https://github.com/akbartus/A-Frame-Component-Postprocessing) I've managed to obtain a working post-processing chain in a-frame. Is it possibile to implement the necessary changes into supermedium three and a-frame in order to allow post-processing to work both in desktop mode and vr mode?

[dmarcos]

@enzofrancescaHM concerns are performance, maintainability of the three fork and clarity of the API (many post effects don’t work in stereo / vr)

I’m happy to provide an avenue for people to experiment if doesn’t compromise maintainability.

Can you open a PR with your three changes?

https://github.com/supermedium/three.js/pulls

Thanks

[enzofrancescaHM]

@dmarcos Thanks for your kind answer, yes, I totally understand your concerns. Performance are very good for some effects like bloom. I talk about bloom because it is the effect that give most realism in change of a very small performance cost and for many applications is almost mandatory (i.e. product configurators and customizers). Again, I reached my custom solution modifying supermedium three and recompiling A-Frame, so I'm ok and my apps are working good, but I would like to share the same possibility with others implementing the feature for everyone or at least write a wiki with the necessary steps to reach the goal. I'am going to open a PR is supermedium three as you suggested, even if the right person to do so should be Cody Jason Bennet that made the same mods to the original threejs.

[dmarcos]

Main perf issue from postprocessing is due to memory copy on mobile hardware that standalone headsets are built on. independent of particular effect.

This is personal opinion but I think bloom is overused as a “one click” way to make your scene look more “sophisticated”. Performance and interaction model have way more impact on user experience and engagement hence my hesitation about post processing.

Thanks for the PR. Will be useful for discussion / reference

[vincentfretin]

From all the comments above, there was two choices of api, the one based on tags #3645 (comment) where the order of effects is clear and is used by other frameworks like r3f and model-viewer.

and the attributes syntax in this comment #3645 (comment) It's a similar api as the previous aframe-effects and also the aframe-extras's movement-controls component with controls property.
But to me that syntax is not so clear about the order and enabled effects.

For example what does it do?

<a-scene
  post-processing="effects:depth-of-field;multisampling:0;disableNormalPass:true"
  bloom-effect="luminanceThreshold:0;luminanceSmoothing:0.9;height:300;opacity:3"
  depth-of-field-effect="focusDistance:0;focalLength:0.02;bokehScale:2;height:480"
>
</a-scene>

• Enable depth-of-field and then bloom effects?
• Enable bloom and then depth-of-field effects?
• Enable depth-of-field effect only?

For me it should only do depth-of-field.

What about this one:

<a-scene
  post-processing="effects:depth-of-field,bloom;multisampling:0;disableNormalPass:true"
  bloom-effect="luminanceThreshold:0;luminanceSmoothing:0.9;height:300;opacity:3"
  depth-of-field-effect="focusDistance:0;focalLength:0.02;bokehScale:2;height:480"
>
</a-scene>

For me it should do depth-of-field then bloom.

It you choose that implementation, you need to add proper warning saying the bloom effect is not enabled because it's not in the list of effects.

attributes based:

• register post-processing system
• register a component for each effect
• enabled effects are specified with effects property on post-processing component
• add proper warning for effect components that are not currently enabled in effects property

tags based:

• register an a-effect-composer primitive mapping to properties of post-processing system
• register a component and a primitive for each effect
• enabled effects are specified as children

For inspiration on how to integrate it, see

• https://github.com/wizgrav/aframe-effects (attributes based)
• https://github.com/pmndrs/react-postprocessing/tree/master/src (tags based)
• https://github.com/google/model-viewer/tree/master/packages/model-viewer-effects/src/effects (tags based)

@dmarcos are you still against tags based approach?