How Ray Tracing Works: From Appel’s 1968 Plotter Shadows to Real-Time RTX

Every time a reflection in a modern game looks like a mirror instead of a blurry guess, or a movie frame lets light bounce softly from a red wall onto a white shirt, you are looking at a trick that started as a research stunt on 1960s plotters. Ray tracing asks a simple question for every pixel on the screen: if I shoot a straight line of sight into this imaginary world, what do I hit, and what color should that hit be?

The answer sounds obvious — follow the light — until you try to do it on a computer that has milliseconds to spare. Real photons leave lamps and bounce forever. Ray tracers cheat by casting rays backward, from the camera through each pixel into the scene, because almost every photon that leaves a light never reaches your eye anyway. That backward walk, refined over half a century, is now the backbone of film rendering and, since 2018, a feature on consumer graphics cards.

Appel, 1968: a ray for every spot on the page

In 1968, Arthur Appel at IBM Research published “Some techniques for shading machine renderings of solids.” His goal was not a Hollywood still. He wanted shaded pictures of machine parts that looked solid — with tone and cast shadows — so that engineers and architects could read depth from a plotter drawing.

Appel’s method, which today would be called ray casting, worked like this. For a roster of spots on the picture plane, reproject each spot back to the observer and form a line of sight. Find the first surface that line pierces. Then ask a second question: is that piercing point hidden from the light by some other surface? If the light cannot see the point, draw it dark. If it can, size the mark by how brightly the surface is lit. Shadows fell out of geometry instead of being painted by hand.

That was already a conceptual leap. Older line drawings could not tell you whether a panel sat above or below a console. Once shadows appeared, the spatial story snapped into place. Appel’s pictures were crude by modern standards — staggered plus signs on a Calcomp plotter — but the ray-from-the-eye idea stuck. MAGI and other labs were exploring related techniques around the same time; Appel’s paper remains the classic early citation for shading and shadow testing with rays.

Think of Appel’s roster as a very low-resolution screen. Each plotted mark stood in for a pixel. The expensive step was not drawing the plus sign; it was deciding which surface owned that line of sight and whether a shadow ray to the lamp was clear. Those two tests — closest hit, then light visibility — are still the first two verbs of every ray tracer written today.

Whitted, 1980: mirrors, glass, and a tree of rays

A decade later, Turner Whitted at Bell Labs turned ray casting into recursive ray tracing. His SIGGRAPH paper, “An improved illumination model for shaded display” (presented in 1979, published in 1980), kept the camera-to-scene primary ray. When that ray hit a surface, it did not stop. It spawned more rays: a reflection ray in the mirror direction, a refraction ray through transparent materials, and shadow rays toward each light.

The result is a tree of rays per pixel. A chrome ball no longer needs a fake environment map glued on by an artist; you see whatever the reflection ray hits. A glass sphere bends the view according to an index of refraction. Shadows remain honest: if a shadow ray meets an opaque object before the lamp, that lamp contributes nothing at that point.

Whitted also showed why tracing from the lights toward the eye is wasteful. Most of those rays miss the camera. Tracing from the viewer presupposes that a ray matters for a pixel, which is the only place the image cares about. His short film The Compleat Angler (1979) made the point visually: recursive reflections and refractions that rasterizers of the era could not fake cleanly.

There is a practical limit. Reflection trees can grow without bound if mirrors face each other, so renderers cap recursion depth or stop when a ray’s contribution falls below a threshold. Whitted’s insight was not infinite bounce counts; it was that global effects could be expressed as more rays instead of more special-case shaders.

Kajiya, 1986: one equation to rule the lights

Recursive Whitted trees handle sharp mirrors and clear glass beautifully. They struggle with the soft, everyday light that fills a room — color bleeding from a green wall onto a gray floor, soft shadows from an area lamp, the glow under a table. Those effects need many samples of incoming light from many directions, not a single perfect reflection ray.

At SIGGRAPH 1986, James T. Kajiya published “The rendering equation,” an integral equation that says, in essence: the light leaving a point in a direction equals the light the surface emits there, plus every bit of light that arrives from the hemisphere and is reflected according to the material. Almost every serious lighting algorithm is a way of approximating that integral.

Kajiya’s practical answer was path tracing. Instead of branching a full tree at every bounce, follow one random path: bounce, bounce, bounce, until you hit a light or give up. Average many such paths per pixel. The image is noisy at first — a snow of speckles — but unbiased: with enough samples it converges to the physically correct answer, including caustics and color bleeding that classic Whitted trees miss.

Distributed ray tracing, introduced by Cook, Porter, and Carpenter in 1984, had already shown that spreading many rays could buy soft shadows, blurry reflections, depth of field, and motion blur. Kajiya’s equation explained why those tricks worked and unified them under one integral. Path tracing was the starkest algorithm that fell out: one random continuation at a time, averaged until the noise gave up.

Path tracing is why modern “physically based” renderers talk about samples per pixel, variance, and denoising. The math is elegant; the bill is statistical. Film frames may use hundreds or thousands of samples. Games often afford one, then lean on clever reconstruction.

Hollywood learns to trust the path

For years, feature animation approximated global illumination with armies of fill lights. Ray tracing changed the economics: let the computer carry light from surface to surface, and let artists place fewer, more natural lamps.

Studios did not all jump at once. Sony Pictures Imageworks helped prove production path tracing on films such as Monster House (2006) and Cloudy with a Chance of Meatballs (2009). At Pixar, Monsters University (2013) pushed a physically based, ray-traced lighting pipeline that still mixed distribution ray tracing with radiosity caching rather than a pure path tracer. The studio’s modern RenderMan path-tracing architecture followed in subsequent features — including Finding Dory, Cars 3, and Coco — and became a reference design for production path tracers elsewhere.

Offline, the cost is measured in core-hours, not milliseconds. That patience buys soft shadows, accurate reflections, and the quiet realism of light that has already bounced twice before it reaches the camera. Textbooks such as Physically Based Rendering turned those production lessons into open, teachable code: a full renderer that implements the rendering equation rather than a pile of special cases.

The speed problem: BVHs, noise, and AI cleanup

A naive ray tracer tests every triangle in the scene for every ray. A film set with tens of millions of triangles makes that impossible. The standard fix is a bounding volume hierarchy (BVH): nest objects in boxes, nest those boxes in bigger boxes, and test the big boxes first. If a ray misses a box, skip every triangle inside it. Intersection cost drops from “linear in scene size” toward something much closer to logarithmic for well-built trees.

Even with a BVH, path tracing is noisy when samples are few. Offline renderers long used filters and, more recently, machine-learning denoisers (NVIDIA’s OptiX denoiser is a widely used example) to turn a speckled preview into a clean frame. In games, related ideas show up as temporal accumulation and neural upscaling. DLSS (Deep Learning Super Sampling) and its ray-reconstruction modes let a title render fewer pixels — and fewer ray samples — then reconstruct a sharper, stabler image with a trained network that looks at motion vectors and history. Denoising does not replace correct light transport; it buys back frame rate when the Monte Carlo estimate is still half-cooked.

RTX, 2018: ray tracing at sixty frames

At Gamescom on August 20, 2018, NVIDIA unveiled GeForce RTX 20-series GPUs based on the Turing architecture. Alongside traditional shaders sat new RT cores for traversing BVHs and testing ray–triangle hits, and Tensor cores for AI features. The company pitched a hybrid model: rasterize most of the frame the old way, then fire selective rays for reflections, shadows, ambient occlusion, or global illumination where they matter most.

Microsoft’s DirectX Raytracing (DXR) API gave engines a common way to submit those rays. Early titles such as Battlefield V shipped with ray-traced reflections; others followed with shadows, ambient occlusion, and full lighting experiments. AMD later added hardware ray-acceleration in its RDNA 2 GPUs (Radeon RX 6000 series, announced in 2020), and consoles of that generation — PlayStation 5 and Xbox Series X/S — included dedicated ray-tracing support as well. What had been a film-farm technique became a checkbox in graphics settings.

Hardware matters because BVH walks and ray–triangle tests are irregular. Ordinary shader cores hate that divergence. RT cores package the traversal so a game can ask for thousands of rays per frame without rewriting the whole pipeline as a software ray tracer. The hybrid bet — rasterize the cheap parts, ray-trace the hard ones — is why “RTX on” usually means selected effects, not a full cinematic path trace.

Real-time ray tracing is still a compromise. Games rarely path-trace an entire frame to film quality. They shoot a limited budget of rays, denoise aggressively, and blend with rasterized lighting. The visual tell is still there when it works: reflections that match the room, shadows that soften with distance, glossy surfaces that stop looking like flat plastic.

What the half-century arc actually bought us

Start with Appel’s plotter shadows. Add Whitted’s recursive mirrors and glass. Replace ad-hoc shading with Kajiya’s rendering equation and path tracing. Let film studios spend the samples; invent BVHs so the rays can find needles in haystacks; invent denoisers and DLSS so games can fake a higher sample count. Put BVH traversal in silicon in 2018 so a living-room GPU can do in a frame what once took a night.

Ray tracing did not kill rasterization, and it did not make lighting “solved.” It changed the default question from “How do I fake this reflection?” to “How many honest rays can I afford, and how do I clean up the rest?” That is a quieter revolution than a blue flash in a Los Alamos lab — but it is the reason the virtual world keeps looking a little more like the one your eyes already know how to parse.

Further reading