Three.js AAA lighting pipeline: Ben Houston's PR roadmap

Ben Houston, who writes in the first person as the author of the pull requests he describes, has published a 21-minute article in a series on bringing an AAA-style lighting pipeline to Three.js. The previous part picked a pipeline for indoor scenes: lightmaps for static meshes, a light probe grid for moving objects and reflection probes for each room, with screen-space reflections (SSR), screen-space global illumination (SSGI) and ambient occlusion on top. This article measures Three.js against it one stage at a time: what games require, what Three.js has today, and what is missing. He credits the Three.js maintainers for the pace. Michael Herzog (@Mugen87) has reviewed most of the PRs and more than once reworked one into a better design that fits the whole library, and mrdoob has expressed strong support for many of them. The PRs have so far been proposed and reviewed one at a time, and the series is Houston's attempt to show how they fit together. He says the biggest remaining gaps are in the SSR, SSGI and BRDF rows of the pipeline: Hi-Z tracing and coverage for SSR, the same coverage for SSGI, and above all pluggable indirect lighting that ties both cascades into the BRDF.
G-buffer. Screen-space effects need per-pixel depth, normals, roughness, metalness and motion vectors. Because Three.js renders forward, its G-buffer fits in 16 bytes per pixel. WebGPURenderer already renders a G-buffer prepass with multiple render targets, and the SSR examples use it. What is missing is normal precision: SSR is unforgiving of normal error, and RGB8 normals make smooth curved surfaces snap to a limited set of directions, so reflections show banding and clumping. AAA engines use octahedral encoding, which Houston calls the most accurate of the encodings surveyed by Cigolle et al. for its size. PR #35480 adds packNormalToOctahedral() and unpackOctahedralToNormal(). RG16 snorm is the target format, with RG16F as the fallback.
Global illumination. Games bake bounce light into lightmaps for static geometry and use a probe volume for dynamic objects. Every standard material in Three.js takes a lightMap, and a complete in-browser baking pipeline now exists (covered in a later part). For dynamic objects, LightProbeGrid places probes through a scene, each storing incoming light as L2 spherical harmonics. Its limit was capture time. Profiling showed the bake was CPU-bound on submitting render passes, so batched capture now renders eight probes per pass with one array camera, and a resolution-12 grid dropped from 5.6 s to 1.2 s. Culling each draw against its own sub-camera doubled that again. The grid now bakes about eight times faster than at the start of September, is fast enough to re-capture a few probes per frame, and supports multi-bounce light. Integrating the spherical harmonics exactly cut their error from 2.4% to 0.08%. What is missing is leak control: a probe on one side of a wall can light an object on the other. DDGI stores the distance to the nearest surface per direction with each probe and ignores probes that cannot see the point being lit. Houston has DDGI-style visibility weighting working as a draft PR. Testing all eight surrounding probes was costly, so the draft can use tetrahedral interpolation, which blends four.
Ambient and specular occlusion. Lightmaps are too coarse for contact shadows and know nothing about dynamic objects, so games add screen-space occlusion, with Activision's GTAO as the standard. GTAO also yields a bent normal (the average open direction), which gives specular occlusion (GTSO). Three.js's horizon-based SSGI runs the same depth-buffer search and gathers light along the way; with SSGI on, its pass supplies occlusion and the bent normal, and GTAO runs only when SSGI is off. Occlusion is a shared buffer set with builtinAOContext, and each source decides whether to read it. The lightmap, probe grid and environment apply AO to indirect diffuse, combined with the material AO map as min( materialAO, gtao ). Reflection probes and the environment apply GTSO to indirect specular. SSR and SSGI apply neither, since a screen-space hit is already correctly occluded. Behind SSGI the diffuse sources skip AO because SSGI's coverage is the occlusion itself. Occlusion darkens a source's color but leaves its coverage alone, otherwise the next source's unoccluded light would leak into corners. GTAO already feeds lighting through a context in Three.js. Missing: specular occlusion, because GTAO outputs one visibility value and the lighting model guesses with a heuristic, so metal keeps reflecting the sky inside folds. Bent normals and GTSO are up as a draft PR, at the cost of an AO target that grows from one byte per pixel to four. Also missing is edge-aware upsampling: half-resolution GTAO upsampled bilinearly leaves dark outlines, and jointBilateralUpsample() (after Kopf et al. 2007) uses full-resolution depth and normals as guides. It is drafted and not yet submitted, and half-resolution SSR and SSGI need the same filter. A third gap is bent normals from SSGINode, so specular occlusion needs no second pass when SSGI is on.
Screen-space reflections. This section summarises Houston's separate SSR series. Shipped engines agree on a recipe: trace the previous frame's lit image, trace exactly over any distance with a Hi-Z depth pyramid, trace at half resolution and reuse rays between pixels (which brings the trace under a millisecond), output radiance plus coverage so probes can fill misses, and denoise that radiance and coverage while leaving the BRDF to the material. SSRNode in Three.js importance-samples the full GGX lobe with correct Fresnel, can validate and refine hits, and its result is reprojected and accumulated over time. But the trace is a fixed-step march with short rays. The three SSR examples find 98%, 82% and 46% of hits within their own ray length, in 2 to 3 ms, and the denoiser fills gaps from neighbouring pixels. All five requirements are missing, each prototyped or in review except ray reuse.
Exact tracing: a Hi-Z pyramid is a mip chain where each texel stores the nearest depth of the four below it, so the ray takes large steps through empty space and single-pixel steps near surfaces, finds every hit the depth buffer holds and has no maximum distance to tune. Houston's HiZNode builds the pyramid in a compute pass. Half-resolution tracing with a joint bilateral upsample is written; ray reuse, as Frostbite does it, is still to build.
Coverage: today a ray that misses samples an equirectangular environment texture passed as environmentNode, which cannot be the PMREM-filtered scene.environment or a local probe, and adding SSR over the image counts the environment reflection twice. In a cascade, SSR would output only radiance premultiplied by coverage (1 for a hit, 0 for a miss, fading toward the screen border), with the environment input and BRDF estimate removed. This is prototyped as part of a Pluggable Indirect Lighting proposal.
Previous-frame tracing: SSR cannot be part of lighting while reading lighting's output. As in DOOM, the trace uses the current depth buffer but reads colors from the previous frame by reprojecting each hit into last frame's TRAA output; a hit that fails the check against last frame's depth gets zero coverage. A mirror facing a mirror gains a bounce each frame. This is in review. Two rules apply: the image must be linear HDR radiance taken after TRAA but before tone mapping and bloom, and the fed-back radiance must be clamped so one firefly does not compound frame after frame.
Denoising: the pair used by the SSR example, TemporalReprojectNode and RecurrentDenoiseNode, treats every black pixel as missing data and fills it from neighbours, which hides tracer misses and leaves nothing for probes to answer. zalo's SVGF-based DenoiseNode replaced the pair in the SSR example and was taken back out five days later. Houston traced the flicker to missing coverage: at a reflection edge a hit and a miss were averaged as two samples of one signal, and the history kept resetting. His ssr-coverage branch carries coverage through every stage of DenoiseNode and takes statistics from covered samples only. Still to do are a two-lookup history, a history cap that follows roughness, and skipping mirrors and matte pixels. Because the denoiser filters incoming light and the material applies its BRDF afterwards, it needs no albedo in the G-buffer and no demodulation step. The text available ends partway through this denoising section, before the later stages.
Key facts
- Ben Houston maps a lightmap, light probe and reflection probe pipeline, plus SSR, SSGI and ambient occlusion, onto Three.js, listing for each stage what games need, what exists and what is missing.
- LightProbeGrid now bakes about eight times faster than at the start of September; a resolution-12 grid dropped from 5.6 s to 1.2 s with batched capture of eight probes per pass, and spherical harmonics error fell from 2.4% to 0.08%.
- Open work includes PR #35480 for octahedral normal encoding, a draft DDGI-style visibility weighting to stop probe light leaking through walls, a draft for bent normals and GTSO, and a drafted but unsubmitted jointBilateralUpsample().
- Three.js SSR still uses a fixed-step march; its three examples find 98%, 82% and 46% of hits within their own ray length in 2 to 3 ms. Hi-Z tracing, coverage, previous-frame tracing and denoising are prototyped or in review, and ray reuse is still to build.
- Michael Herzog (@Mugen87) has reviewed most of these PRs and mrdoob has expressed strong support for many of them.
Why it matters
Three.js is a widely used web 3D library, and this article lays out a concrete route to the lighting techniques AAA engines rely on: baked lightmaps, probe volumes, reflection probes and screen-space effects. Until now the pieces have been proposed and reviewed one at a time. Houston says nobody has been asked to sign up to the whole, and the series puts them in one picture so each PR can be judged for where it fits as well as what it does. He names SSR, SSGI and the BRDF integration as the biggest remaining gaps.
Who it affects
Developers building indoor or architectural scenes in Three.js with WebGPURenderer, who would get lightmaps, probe grids and better reflections. Also the Three.js maintainers and reviewers, notably Michael Herzog (@Mugen87) and mrdoob, who are weighing these pull requests, and contributors such as zalo whose denoiser work is discussed.
How to use it
Some of this exists now. Every standard material takes a lightMap, LightProbeGrid has an example, and the SSR examples already use the WebGPURenderer G-buffer prepass with multiple render targets, with SSR composited over the image. Houston shows code for the MRT setup, for feeding GTAO through builtinAOContext and for the SSR composite. Octahedral normal packing is in PR #35480. Draft PRs for DDGI-style visibility and bent normals, and the ssr-coverage branch, are not merged. No release timeline is given for any of them.
How solid is it
The author writes about his own PRs and drafts, and he gives concrete numbers: the 5.6 s to 1.2 s bake, the 2.4% to 0.08% spherical harmonics error, and the 98%, 82% and 46% SSR hit rates. The Hi-Z benchmark table and the pipeline overview table are not in the text available here, so Hi-Z timings cannot be checked. The text available is truncated in the SSR denoising section, so later stages such as temporal upscaling are not covered.
Risks and caveats
Much of the work is unfinished: previous-frame SSR tracing is in review, DDGI-style visibility and bent normals are drafts, jointBilateralUpsample() is not yet submitted, ray reuse is still to build, and the denoiser still needs a two-lookup history, a roughness-following history cap and skipping of mirrors and matte pixels. Bent normals and GTSO cost an AO target growing from one byte per pixel to four. zalo's SVGF denoiser was taken out of the SSR example five days after it replaced the older pair, which shows how fragile the denoising stage has been. No timeline is given for the pending PRs.
“Nobody has been asked to sign up to the whole.”
— Ben Houston, on how the PRs have been proposed and reviewed so far