At Three.js Paris I opened with a render that took 5 seconds on an NVIDIA RTX PRO 6000, then asked the room how long a single frame of the same scene takes on an iPhone. A day? An hour? A minute?
Eight milliseconds. 120 frames per second, on a phone, from a payload containing no geometry, no materials, and no lights. A short while earlier, that asset had been a 40 GB OpenUSD file.
This is the long version of that talk, for Three.js developers who want to know what's happening on the GPU, on the wire, and in the scene graph.
Three claims:
- Radiance fields can be rendered in real time on commodity hardware.
- They reach a fidelity a rasterized PBR pipeline structurally can't, because a field records what light did instead of simulating what light would do.
- A recording of appearance streams the way video streams, so time-to-interactive stops depending on how big your asset is.
The dioramas throughout show the same cherry tree, the same virtual camera, and the same 16×16 sensor hanging between them; each animates how one family of renderers computes the sensor's pixels. They're live Three.js scenes. Drag any of them to orbit.
The renderer's job
The primary job of a renderer is to turn a 3D scene into something useful from a particular viewpoint. For a final image, that means figuring out what is visible at every pixel - and, ultimately, what each pixel should contain. Everything else is scaffolding around that job, performed roughly two million times a frame at 1080p. There are two classical ways to do it. Rasterization starts with a shape and finds which pixels it lands on. Path tracing starts with a pixel and finds what it's looking at. Both consume triangles; they ask opposite questions of them. Radiance fields add a third option: don't compute the pixel's color at all. Instead, look it up.
The terms of the comparison, so nobody cheats: a static scene (only the camera moves), a photoreal target (the path-traced reference, not "looks fine"), and every bake trick legal for PBR (frozen shadow maps, lightmaps, prefiltered environments). Fields need the static assumption; handing it to PBR as well gives PBR its best case, which is what makes the result mean something.
Path tracing
The precise way to color a pixel is to follow the light. Cast a ray from the camera through the pixel. When the ray hits a surface, the material decides where it goes next. It decides probabilistically: roughness isn't an angle but a distribution over microfacet orientations, so the renderer samples a direction from the BRDF, the material's description of how much light it scatters from one direction into another. The path continues along that direction, and each surface it touches multiplies its contribution into a running product. At every bounce, the renderer also checks whether the light is directly visible from that point. The path ends when it escapes, hits an emitter, or gets cut off. Do it again through the same pixel with different random choices, thousands of times, and average: the estimate converges on the rendering equation.
One ray, one pixel. Every bounce is a real BVH raycast against 2.26 million triangles. It exits parallel to the sun, because a directional light has no position.
What's remarkable is what you don't have to add. Follow the light and here's you get as a consequence:
- shadow sides that pick up color from the objects around them (global illumination)
- shadow edges sharp at the contact point and soft as they fall away (a true penumbra)
- crevices that darken exactly where light can't reach (occlusion, with no halo)
- glossy surfaces reflecting parts of themselves that aren't on screen (reflections)
- light entering a translucent material and leaving somewhere else (subsurface scattering)
- volumetric scattering in participating media (smoke, steam)
- reflective and refractive caustics (focusing of lights off curved metals or through transparent materials)
Same pixel, several rays, different fates. Rays that never find the sun fade out; that's the noise a path tracer averages away with thousands of samples.
The whole sensor as a wavefront: the sample population launches together and advances in per-bounce waves, thinning as paths terminate. This is often how GPU path tracers often actually schedule.
Two million pixels, times on the order of a thousand samples, times up to ten bounces, is tens of billions of intersection tests per frame. GPUs handle this badly: paths have different lengths, so lanes diverge and idle, and rays scatter through memory, so caches stop helping. The native answer is silicon: RT cores for traversal, ML denoisers to clean up one noisy sample per pixel.
Native real-time graphics increasingly makes photorealistic path tracing practical by combining dedicated ray-tracing hardware with neural reconstruction. The web doesn't expose that same hardware through its portable graphics APIs. It has exactly one other route to the same place, and we'll get to it.
Rasterized PBR
Rasterization traces nothing. That's the trick, but it has consequences.
Fragments shade in warps dispatched in submission order (draw order, then triangle index), so the sensor fills following the geometry buffers rather than scanlines.
That's the mechanism. Now the compromises, each one a small departure from the reference image.
The shading pass is efficient, and enormous
The base pass is exactly the kind of workload a GPU is good at: millions of highly parallel programs with coherent memory access. It can still be the most expensive stage in a frame with frozen shadows, because of what it has to read and compute . Per pixel: four to six filtered texture fetches (base color, normal, ORM, an AO or lightmap); a normal-map transform; nine to sixteen PCF (percentage-closer filtering) taps into the shadow map; ambient from a prefiltered environment. Call it 300 flops and twelve to fifteen fetches per pixel, times two million pixels, times overdraw.
The flops are negligible. The fetches are billions per second, and that's where the frame's time actually goes. Saying the shading pass saturates VRAM bandwidth doesn't mean the shader is wasteful; it means the memory bus is pinned at 100% moving bytes the model can't do without. Every fetch is an input, so there's nothing to optimize away. It's simply a very large amount of data, and a devicePixelRatio of 2 quadruples it.
Direct light only, and everything else faked
Direct illumination is evaluated from the light's direction alone. No rays are cast. A fragment knows its position, normal, material parameters and the direction to each light, and nothing else. Anything indirect, light bouncing off one thing onto another, has to come from precomputation (lightmaps, probes, environment maps) or from screen space.
Screen-space ambient occlusion (SSAO) and screen-space reflections (SSR) work from the depth and normal buffers: a 2D picture of a 3D world, not the geometry. Per pixel, each runs a loop of sixteen to sixty-four dependent depth fetches, every one waiting on the last and landing unpredictably in memory, recomputed every frame because it depends on the camera. And the output can't be better than the input. SSAO is meant to darken corners and crevices where light struggles to reach, but working only from a depth image, it instead draws halos where a depth discontinuity gets mistaken for a crevice, and misses any occluder that isn't in the picture. SSR is meant to fake reflections without ray tracing, but it can't reflect anything that's off-screen or facing away from the camera.
Shadows re-render the scene
Shadow maps re-render the scene's depth from each shadow-casting light: a full vertex pass per light, four for a cascaded sun, plus a pass of percentage-closer filtering (PCF) per pixel — several extra samples around each shadow-map lookup, averaged together to soften what would otherwise be a single hard-edged pixel boundary. The result is resolution-bound, prone to the visible artifacts every three.js developer has fought at some point, and its blur radius stands in for the sharp-to-soft gradient a real penumbra has. In the static arena you can freeze them, but three.js re-renders every shadow map every frame by default; setrenderer.shadowMap.autoUpdate = false and flip needsUpdate when something moves.
Per light, per bounce, per frame
With the shading pass saturated and shadows multiplying vertex work per light, the remaining lever is feeding the pipeline less: the diorama's own tree drops from 2.26 million triangles in path mode to 203 thousand in raster mode, and simplification costs silhouettes that normal maps can only partly fake.
Everything above is per pixel, most of it per light and per effect, all of it per frame even when nothing changed. PBR's formula inventory grows with the realism you chase: another light is another GGX-Smith-Fresnel-PCF stack per pixel, reflections are a sixty-four-step march per pixel, global illumination is a separate machine. Frame cost tracks the scene's optical complexity. And after all of it, the image still isn't the reference. It isn't close.
The ceiling is the format
More hardware doesn't close the gap. Practically, your users won't buy a new machine to view your scene; the phone in their pocket is the render target. Structurally, more compute converges on your model of reality, not on reality. Every piece of PBR is a model, and the model is simpler than the thing: a surface isn't a roughness value, it's microgeometry; a light isn't a point, it has size; skin isn't a surface, light wanders around inside it. As John Salvatier put it, reality has a surprising amount of detail. Give the pipeline infinite compute and it renders the description perfectly. The description was never reality.
A mesh-and-materials asset, glTF or USD, is a description of a rendering simulation: here's a surface, here are its parameters, here are some lights, compute what this should look like. glTF is an excellent asset format, but when what's inside is geometry, materials and lights, the best possible render is bounded by the simulation. Not by the artist. Not by the GPU. By being a simulation at all.
So what if we didn't simulate?
Radiance fields: recording light instead of simulating it
Instead of computing what color a pixel should be, record what color it is: the light arriving at the camera from every position, in every direction. Let the light transport happen once, offline, with no frame budget (by reality itself when the subject is real, by a reference renderer with hours to spend when it isn't), and look up the answer at runtime.
That function, from a position and a direction to a color, is a radiance field: a photograph you can walk around in. Every bounce, every penumbra, the color bleed and the caustic are in the recording because they were in the light. There's nothing to trace because there's nothing to trace against; geometry, materials and lights are all inside the field, dissolved into appearance. There's no scene left to describe, only the light it gave off.
The function sounds impossibly large, and for decades it was. But real scenes are mostly empty space and smooth surfaces, so it compresses; neural fields showed in 2020 that it could be learned from images, and Gaussian splatting in 2023 gave us an encoding that rasterizes in real time with the only primitives the web has: instanced quads, alpha blending, and a sort. The web skipped the ray-tracing era. Splatting gave it a route to photorealism through data instead.
The splat
A Gaussian splat is one encoding of a field, today's and not the last. Picture it as a small, soft, stretched blob of light: it has a position, a shape (which direction it's stretched in, and by how much), a transparency, and a color. Instead of one fixed color, it's stored as a small set of numbers that let the renderer work out a different color depending on which direction you're looking at the splat from. That's what gloss and translucency are in a field. A texture gives a surface one fixed answer everywhere; a splat's answer changes with your viewpoint. When a highlight slides across the tree as you orbit it, that isn't a shader effect. It's baked into the data.
One globally sorted draw. The compositing front is a flat plane sweeping toward the camera, because the sort is by view-space depth rather than distance, and every pixel converges at once.
Why this is the most efficient thing a GPU can do
Add it up and the splat frame can burn more raw arithmetic than the PBR frame; both are trivial for a multi-teraflop GPU and neither is ALU-bound. What differs is which math, and where it lives.
GPUs are fast for one specific reason: they're built to run the exact same simple operation on millions of independent pieces of data at once, reading memory in predictable, sequential chunks. When work looks like that, the GPU can hide the time it takes to fetch anything, because there's always some other piece of work ready to run while one is waiting on memory.
Photoreal PBR keeps breaking that pattern. A screen-space march has to look up one pixel's result before it can decide where to look next, so it's not independent, it's a dependency chain. Different pixels take different lighting paths depending on how many lights hit them and what material they're on, so the work diverges instead of staying identical. Reflections and shadows read memory at addresses that depend on the scene rather than a predictable pattern, so the reads scatter instead of streaming. And PBR runs several separate passes, shadows, base color, screen-space effects, post-processing, each of which stops and restarts the whole pipeline.
The splat pipeline breaks none of this. Every fragment runs the same tiny calculation. Fragments don't depend on each other. Each one reads from one predictable spot in memory. It's close to the ideal shape of work for a GPU, top to bottom.
PBR can't stop breaking that pattern, because photorealism is made of global questions (what blocks this light, what reflects into this pixel, where did this photon come from) and a rasterizer is a local machine: millions of workers, each holding one pixel's facts. Global questions forced through a local machine are inherently dependent, scattered and divergent. The field's entire move is that its global questions were answered offline and compressed into per-splat records. What's left at runtime is local: where am I relative to the camera, what color am I from here, who's in front of me. Local questions are what the local machine was born to answer.
Two things follow from that. First, the splat's list of steps is short, and it doesn't get longer as the scene gets more complicated: work out a color, project a shape, fade it out, blend it in. Add a caustic, a mirror, glowing skin, whatever — those same few steps still apply, so a complicated-looking scene doesn't cost any more to draw than a simple one. Cost tracks how many splats are on screen, not how optically complicated the scene looks.
Second, it's a cache. Ordinary real-time rendering recomputes the lighting from scratch, every single frame. A field skips that: the lighting was already worked out once, offline, and baked into the splats. Your GPU isn't computing light when it draws a frame — it's replaying a recording of it. That's the real secret behind eight milliseconds.
Fidelity, and what we're holding constant
Performance alone would make this "faster but worse," and it isn't. Every effect on that list, the color bleed, the penumbra, the occlusion, the reflections, the subsurface scattering, is in the field because it was in the light when the field was trained; nobody implemented any of it. Fidelity is bounded by the training views and the encoding's capacity, not by a BRDF. No UVs to unwrap, no materials to author, no light rig. The light already happened.
In the workflow below, the field is trained from the same source file you'd otherwise render live, from reference-quality renders of it. Same file, two ceilings. Rendered live, it's bounded by direct-light shading and screen-space guesses. Recorded, it's bounded by the reference. The gap is a property of when the light transport happens, not of the asset.
To keep the comparison honest: a field, as shipped today, is a static scene under fixed lighting. Only the camera moves; it won't take your scene's lights or cast shadows onto your meshes. That's the current contract, not a law of physics, and it's the same gift PBR got. PBR keeps one real advantage: memory. A mesh's footprint scales with the description; a field's scales with appearance, millions of splats at 48 coefficients each. So the scoreboard reads: milliseconds, the field. Photons, the field. Megabytes, the mesh.
The problem: it doesn't fit

A full-fidelity field of a single asset, in raw form.
A full-fidelity field of a single asset runs on the order of a gigabyte of raw data. That's why 3D photorealism has never fit on the web, and why product pages are static JPEGs or video frames painted to a canvas wired to your scroll position. The mesh formats we have don't help: a glTF draws nothing until its last byte lands, and every progressive variant you build (LOD swaps, placeholders, srcset-style candidates) is a manual asset that multiplies your build matrix. This was never just a rendering problem. It's a payload problem.
Spatial streaming: we don't ship the scene, we ship what it looks like

Refinement happens in place: no loading bar, no pop-in. One asset, every device.
The web's answer to an impossible payload is to stream it. Spatial streaming works like video streaming (progressive JPEGs, YouTube switching resolutions on the fly), except the thing coming into focus is something you can orbit. The first usable frame lands after about 150 KB and is already interactive. Everything after is refinement, in place: no loading bar, no pop-in.
The field is hierarchical: a coarse level describes the whole object with a handful of large, soft shapes, and deeper levels add detail region by region. Because a field is built from many small overlapping pieces rather than one connected surface, finer levels sharpen what's already drawn instead of replacing it. You can't draw half a mesh. You can draw a coarse field.
Refinement is driven by the screen: each node's error projects through your camera into pixels, and nodes whose error would be visible get their children first. That one rule is foveated (detail in front of the camera first), scene-aware (the stream is an Object3D, so screen size and visibility fall out of your own camera and transforms) and culled (what you can't see isn't fetched). And it adapts: a controller watches actual frame time and throughput and hands the solver a budget, which it spends on detail by value per byte. A workstation on fiber pulls the top tier; a mid-range phone on 4G settles where it's stable. One asset, every device, no tiers to author.
By the numbers
Same asset, same device, same connection: slow 4G at 1.4 Mbps. Only the format changes. Three milestones: interactive (grab it and turn it), sharp, and photoreal(indistinguishable from the reference at viewing distance).
| Format | On the Wire | Interactive | Sharp | Photoreal |
|---|---|---|---|---|
| OpenUSD (source) | 40 GB | ~2.5 days | ~2.5 days | ~2.5 days |
| glTF, optimized (Draco + KTX2 + Meshoptimizer) | ~4 GB | ~6 hours | ~6 hours | never |
| Radiance field stream | streamed | ~10 s | ~30 s | ~2 min |
USD is all-or-nothing: 40 GB at 1.4 Mbps is about 63 hours, and every milestone arrives together. Optimized glTF lands in roughly a tenth of the time and tells the same story, one download and then a frame, except that it never reaches photoreal, because real-time shading approximates what the reference computed exactly. The stream is interactive in about ten seconds, sharp at thirty, photoreal in two minutes, from a 40 GB source on slow 4G.
| Speedup vs. USD | Interactive | Sharp | Photoreal |
|---|---|---|---|
| glTF, optimized | ~10× | ~10× | — |
| Radiance field stream | >20,000× | ~7,600× | ~1,900× |
And the source size doesn't matter. The first level of detail is about 150 KB: a megabyte or a terabyte, the coarsest perception of anything is small. Time-to-interactive for a photoreal 3D asset is the time it takes to download a JPEG.

Source size across four orders of magnitude; a flat ~150 KB to first interaction.
Every number here comes out of a pinned harness: device, network profile, asset and glTF pipeline held constant. Run it against your own asset.
Two lines of code
None of that speed asks much of you. Here's the whole workflow, end to end. You hand us a glTF or USD asset. A GPU spends a couple of hours learning what it looks like from every angle. You get it back as a stream. Then:
import { MirisStream } from "@miris-inc/three";
scene.add(new MirisStream({ uuid, viewerKey }));
That's the integration. MirisStream extends Object3D: position it, scale it, parent it to a Group with your meshes, and your camera, controls and render loop just work. Because it lives in your scene graph, the streaming section comes for free: it refines where you're looking, sizes detail to its footprint on screen, and culls what you can't see. The field dioramas on this page are exactly this code, streaming the tree from our CDN.
Looking forward
Three.js r186 ships a native GaussianSplatMesh for the WebGPU renderer. The primitive is going mainstream; rendering a splat is about to be a weekend project. But splats are one encoding among several, compression and streaming are moving fast, and the first-party renderer deliberately leaves out streaming and level of detail, the part that decides ten seconds versus six hours. What doesn't change is how a stream plugs into your scene. The encoding will change. Those two lines won't.
