A few weeks ago I posted a photo of a scene on the canal where I was doing basic endmember mapping — I think I had 15 or 20 endmembers at the time. That pipeline has since been deprecated.
Also, the pipeline for another post has been deprecated.
🔄 Revisiting both scenes with modern workflows — updates to come.
The Shape of the Problem
A hyperspectral image is not like the photographs we pin to the wall. It is a quiet stack of narrowband veils, a volume of measured light where each small square holds not just what it shows, but what it keeps inside. The question the machine asks is so simple, so almost childlike: for every tiny square, what truth lives here, and how sure can we be? To answer it, pixel by pixel, across a single frame, the machine must make thirty trillion floating-point instructions. They do not rush. They move through the machine in several gentle rounds.
The numbers are large, yet the design is clear. A deep sequence model drinks in the data, turns each pixel's story through a hidden room it has learned, and leaves behind a map of answers, and of doubt. The whole quiet business finishes before a single human thought can fully unfold.
The Cores
Thousands of tensor cores join the work. Each one is like a small, well-ordered room, built for a single purpose. They excel at fused multiply-add operations, a steady, unbothered act: to multiply two numbers and add a third in a single breath. This is the smallest meaningful step of deep learning. Thirty trillion of these little steps unfold together, like hands working in unison.
The cores do not wait. The work is arranged so that one core's result slips directly into the next layer's waiting hands, without pausing to rest in the wider memory. The mathematics stays close to home. Registers, shared memory, and cache hold the entire working set of each round inside the silicon, and only the finished answer steps outward.
The Passes
Inference is not a single moment. It is a gentle sequence of steps, each one turning the pixel picture through a different window of understanding.
Pass one brings a sense of place. The raw vectors enter the model. A scanning eye treats the two dimensions of the picture as a single line, applying a learned weight that remembers what comes before. Each pixel moves through the first hidden layer, carrying both its own voice and the quiet whisper of its neighbors.
Pass two through N refine the thought. Later rounds give new measure to what has been learned. A handful of attention dimensions speak to every pixel at once. The work builds steadily upon the step before it. There is no clearing of the slate, only a continuous deepening. Early stages catch the broad outlines; later stages untangle the subtlest threads. Each turning is a quiet matrix multiplication: pixel vectors guided into the hidden space, shaped by the learned shift, and guided back.
Pass N plus one offers reconstruction and doubt. The refined vectors are read aloud. A mirror image of the original appears, and the difference between the mirror and the true image is measured. Squares the model cannot perfectly recreate, because their quiet signature does not match any pattern it has learned, are marked for closer looking.
The final pass brings classification and certainty. The gathered understanding is compared to a well-thumbed book of known things. Each square receives a name and a measure of trust. The map of doubt is formed by crossing several independent threads. When they agree, certainty is strong. When they part ways, the place asks for human eyes.
The Nanosecond
A single multiply-add on a modern core finishes in less than a breath. At that pace, thirty trillion instructions, spread across thousands of working minds, shrink into tens of milliseconds per round. The whole multi-round process, from first sight to final name, takes less time than a thought settles in a sleeping mind.
We rarely wait for the work itself. The true measure lies in the movement of data. Keeping the working letters close to the silicon, avoiding long journeys to distant memory between rounds, is what saves our quiet budget. Each round inherits what came before, passed through the cache like a familiar note.
The Transposition
"The pixel vector turns," is the quiet step that makes all this possible. A hyperspectral pixel is a single line of numbers, a record of light at different places along the spectrum. The model speaks in sequences, so the two dimensions of the picture must be folded into a line. But the order must be true. A simple scan would leave quiet breaks at the edges. The model's learned attention mends this by letting the picture guide the line. Each square decides how much to listen to what stands before it.
When a pixel vector turns through the hidden space, it changes its bearings. The original lines, the measurements at different points along the spectrum, give way to learned lines that hold what matters. A pixel that began as a list of numbers becomes a point in a small, clear world where distance means kinship. The turning is complete. Every original line speaks to every new line, which means the matrix multiplication at this stage alone holds a large share of the thirty trillion.
Why Thirty Trillion
The count grows naturally from the chain of matrix multiplications for each square in each round: the first guiding, the inner shift, the return to the known space, the comparison, the recreation, the measuring of the gap. For a grid of millions of squares, with spectral and hidden dimensions in the tens, and a book of references wide and deep, the little counts grow quickly. Across several rounds, they reach thirty trillion.
This is not a heavy number for a hyperspectral process. It is, if anything, light. The design achieves it with grace because the sequence form avoids the steep hill of quadratic cost that waits for other paths. The scan is the quiet key. It moves in a straight line through the sequence, walks easily across the features, and divides the work so each core knows its task.
The Silence at the End
When the last round completes, the answer map is written down. A coloring is placed upon it. A faint layer settles over the scene. The observer sees, for the first time, a square by square breaking of the world into its parts. One substance lifts from the ground, pigment from canvas, living trace from stone, all held within doubt markers that whisper where the model knows its place and where it pauses.
Thirty trillion instructions have passed. Not one of them calls out to the one watching. The only proof that anything occurred is a correctly named square in the corner of the map, where a soft signature met its match in the book, and the model's trust was deep enough to point it out for human hands. The pipeline leaves a quiet room. A map, a score, a gentle suggestion. The trillion-instruction heart behind it works unseen, by design.
sha512sum -c SHA512SUMS.CA42 47E8 9A5E FEAB 36DC 6A42 C547 9171 B69A 3CFB 887D B92C 3FB1 480A 2993 57A3.ots file proves that SHA512SUMS existed at or before the Bitcoin block timestamp below.