An interactive course · Dolphin emulator

How the GameCube made sound.

We start from the very beginning — what a digital sound signal even is — assuming you know nothing about audio engineering. Once the vocabulary is solid, we follow the signal through Nintendo's tiny audio chip and out through Dolphin's emulator. Sixteen modules, each with a live synthesiser so you can hear every idea, not just read it. No game audio ships with this page; every sound is generated in your browser.

Begin Part I
Part I

Digital signal processing

This first part is pure fundamentals — nothing about any specific console or chip yet. It's the toolkit that every piece of digital audio gear is built from, whether that's a phone, a synth, a CD player, or the chip we'll meet in Part II. We build the vocabulary one word at a time — sample, rate, bit depth, compression, mixing, filtering — and you can play and see each idea with a simple synthesised tone. If a term is ever used before it's explained, that's a bug; tell us. Everything in Parts II and III is just these five ideas wired together.

Modules 01–08Difficulty Absolute beginnerLabs 7 interactive
Module 01 · Part I

Sound as numbers

GoalUnderstand, from scratch, how a real sound in the air becomes a list of numbers a computer can store and process.

Let's start with what sound physically is. When something vibrates — a guitar string, your vocal cords, a speaker cone — it pushes and pulls on the surrounding air, creating tiny, fast changes in air pressure. Those pressure changes ripple outward as a wave and reach your eardrum, which vibrates in step. That's all sound is: a wobble in air pressure over time.

To put that wobble inside a computer, we do something wonderfully simple. Many thousands of times per second, we measure the pressure and write down the result as a number. Each individual measurement is called a sample. The size of the number — how far the wave has pushed away from its resting point at that instant — is the amplitude. String all the samples together in order and you have the whole sound as a plain list of numbers. Feed that same list back to a speaker at the same speed, and the wave — and the sound — reappears.

Analogy · A flipbook

A movie isn't really moving — it's a stack of still photos shown fast enough that your eye fills in the motion. Digital sound works exactly the same way. Each sample is a still “photo” of the wave's height at one instant, and playing the photos back quickly recreates smooth sound. Take too few photos per second and, just like a choppy movie, the result looks — or here, sounds — wrong. That “how many per second” question is Module 03.

Two numbers describe this whole scheme, and they're the backbone of the next two modules. The first is how often we take a sample — the sample rate (Module 03). The second is how precisely we record each measurement — the bit depth (Module 04). Get comfortable with those two words; you'll see them constantly.

0 50 87 100 87 50 0 -50 -87 -100 -87 -50 0 Continuous pressure wave measured at each tick ↓ → a list of numbers
Sound as numbers. The smooth wave is measured at regular ticks; each measurement — a sample — becomes one number in a list. Replay the list at the same speed and the wave (and the sound) returns.

What makes one sound different from another

If you play a flute and a trumpet at the same pitch, they sound different. Why? It comes down to the shape of the pressure wave. That shape is called the waveform, and the character it gives a sound — reedy, bright, hollow, harsh — is the timbre (pronounced “TAM-ber”). A few shapes are so basic they have names, and a synthesiser like the one on this page builds everything from them:

  • Sine — the purest possible tone, a single smooth frequency with no extra colour. Think of a tuning fork or a gentle whistle.
  • Square and saw(tooth) — jagged shapes that are secretly a pure sine plus a stack of higher tones piled on top. Those extra tones are called harmonics, and they make the sound bright and buzzy — classic 8-bit game “bleeps.”
  • Triangle — in between: a little richer than a sine, but still mellow.
  • Noise — every frequency at once, no pitch at all. This is your “shh” sound: wind, static, a snare drum, a waterfall.

The frequency of a wave is how many times it repeats per second, measured in hertz (Hz) — so 440 Hz means the wave completes 440 full cycles every second. Higher frequency = higher pitch. Every sound a chip produces is ultimately one of these shapes (or a recorded snippet) at some frequency, scaled in volume and combined with others. Play with the scope below: switch the shape, sweep the frequency slider, and watch the trace change while you listen.

Signal lab — waveforms on a scopeWeb Audio · live

The green trace is an oscilloscope — it draws amplitude (up/down) against time (left/right), so you're literally seeing the stream of sample values as a line.

Key takeaways
  • Sound is a pressure wave; a digital signal is that wave measured as regularly-spaced numbers (samples).
  • A sample's value is its amplitude — the wave's height at that instant.
  • Waveform shape = harmonic content = timbre (why a flute ≠ a trumpet).
  • Two numbers define the format: sample rate (how often) and bit depth (how precise).
Module 02 · Part I

The frequency domain

GoalSee that every sound is really a stack of pure sine waves, and learn to read a spectrum — the picture used by filters and aliasing later.

Back in Module 01 we said a square or saw wave is “secretly a sine plus a stack of higher tones.” That wasn’t a metaphor — it’s one of the most important facts in all of audio, and this module makes it concrete. The insight (we owe it to the mathematician Fourier) is this: any sound, however complicated, is just a sum of simple sine waves at different frequencies and volumes. Add the right sines together and you can build any waveform; pull a sound apart and you always find sines underneath.

That gives you two different ways to look at the very same sound. The first is the one you already know: the time domain — amplitude wobbling up and down over time, the wiggly line on an oscilloscope. The second is the frequency domain — a picture of which pitches are present and how strong each one is. That picture is called the spectrum, and it’s drawn as a bar chart: frequency runs left (low) to right (high), and each bar’s height is how much of that frequency the sound contains.

Analogy · A recipe

The time domain is like tasting a finished cake — the whole thing at once, moment by moment. The frequency domain is the recipe: how much flour, sugar, and egg went in. Both describe the same cake; the recipe just lists the ingredients — here, the pure sine tones.

Reading a spectrum quickly becomes second nature:

  • A pure sine is a single bar — one frequency, nothing else.
  • A square or saw is a tall bar at its base pitch — the fundamental, the note you actually hear — plus a row of shorter bars at whole-number multiples of it. Those multiples are the harmonics, and they’re exactly what makes those shapes sound bright.
  • Noise fills the whole chart — every frequency at once, which is why it has no pitch.
Time view amplitude over time (the scope) Frequency view (spectrum) how much of each pitch is present fundamental
Two views of one sound. The same sawtooth wave shown as amplitude-over-time (left) and as its spectrum (right) — a tall bar at the base pitch (the fundamental) plus shorter bars at its harmonics. A pure sine would be a single bar; noise would fill the whole chart.

This is the mental model the rest of the course leans on. When a filter “removes the highs” (Module 08), it’s erasing the right-hand bars of the spectrum. When a tone “folds back” as aliasing (next module), it’s a bar sliding to the wrong place. Keep both views in mind and everything downstream gets easier.

The lab plays a tone and shows it both ways at once: the scope on the left (time), the live spectrum on the right (frequency). Switch the waveform and watch the bars change — a sine collapses to one bar; a saw grows a whole ladder of harmonics; noise becomes a wall.

Spectrum lab — the same sound, two viewsWeb Audio · live
Left = time · right = frequency. Watch harmonics appear as you switch shape.
Key takeaways
  • Any sound is a sum of pure sine waves — that’s the Fourier idea.
  • The same sound has two views: time domain (wave over time) and frequency domain (its spectrum).
  • A spectrum is a bar chart: the fundamental is the pitch you hear; harmonics are the brightness above it.
Module 03 · Part I

Sampling & the Nyquist limit

GoalSee why the sample rate sets a hard ceiling on the highest sound you can capture — and hear what happens when you break that rule.

In Module 01 we took a measurement — a sample — many times per second. The exact number of samples per second is the sample rate, measured in kilohertz (kHz), where 1 kHz = 1,000 samples per second. A music CD, for example, uses 44.1 kHz — 44,100 measurements every second, per channel. That sounds like overkill, but there's a precise reason it needs to be that high.

Analogy · Spokes on a wheel

In old films, fast-spinning wagon wheels sometimes appear to slow down, stop, or even turn backwards. The camera only takes 24 photos per second, and the wheel spins faster than that — so between photos it has turned almost a full spoke, and your eye is fooled into seeing slow backwards motion. Sound has the identical trap: if a tone wiggles faster than you're sampling it, your measurements can't keep up, and the tone gets recorded as a completely different, lower one.

This is captured by the Nyquist theorem, and it's the single most important rule in sampling: you can only faithfully capture frequencies up to half the sample rate. That halfway point is the Nyquist limit. Sample at 32 kHz and everything up to 16 kHz records perfectly — and since a healthy young ear tops out around 20 kHz, that's plenty for game sound. But push a tone above the limit and it doesn't just vanish. It folds back down into the audible range and shows up as a false, lower-pitched tone that was never in the original.

That false tone is called aliasing (the real frequency wearing a fake “alias”), and it sounds like harsh, metallic gravel or whistling. Avoiding it is a huge part of an audio system's job — you'll see the specific tool that fixes it in Module 08.

Worked example

Suppose you sample at 8 kHz. Your Nyquist limit is 4 kHz. Now try to record a bright 5 kHz whistle — 1 kHz over the line. It doesn't record as 5 kHz; it folds back the same 1 kHz it went over, landing at 3 kHz. A rising sweep, then, seems to rise, hit the ceiling, and come back down — the tell-tale sound of aliasing.

Real (fast) tone False “alias” tone the samples trace violet = sample points (too few)
Aliasing. The cyan wave is too fast for the sample points (violet dots). Join those samples and you get the magenta wave — a slower tone that was never played. That is aliasing, and it’s why the sample rate caps the highest capturable frequency.

The lab plays the exact same rising sweep at three different sample rates. At 32 kHz it glides up cleanly. Crush it to 8 kHz or 4 kHz with no protection and you'll hear the top of the sweep dissolve into that metallic gravel — aliasing, made audible. Try them in order.

Sample-rate & aliasingWeb Audio · live
Key takeaways
  • Sample rate = samples per second (kHz). CD is 44.1 kHz.
  • Nyquist: the highest capturable frequency is exactly sample rate ÷ 2.
  • Go above that limit and the tone folds back as a false one — aliasing.
  • A dedicated filter (Module 08) is what keeps aliasing out.
Module 04 · Part I

Quantisation, PCM & ADPCM

GoalLearn how bit depth trades file size for fidelity, and how a clever compression trick (ADPCM) squeezes audio into just 4 bits per sample.

The sample rate answers “how often do we measure?” This module answers the other question: “how precisely do we write each measurement down?” That precision is the bit depth.

A computer stores numbers in bits (binary digits), and the number of bits decides how many distinct values you can record. With n bits you get 2n possible levels: 8 bits gives 256 levels, 16 bits gives 65,536. Recording a smooth wave with only a handful of levels is like drawing a curve on graph paper but only being allowed to put dots on the grid lines — you have to round every point to the nearest line. That forced rounding is called quantisation, and the error it introduces — a gritty, fuzzy crackle riding on top of the sound — is quantisation noise. More bits = finer grid = less noise, at the cost of bigger files.

Analogy · Rounding money

Imagine you could only record prices to the nearest dollar. $4.99 becomes $5, $0.25 becomes $0 — small amounts get mangled. Allow cents (more “bits”) and everything is captured faithfully. Bit depth is exactly this: how fine the smallest step you're allowed to record is.

Original smooth wave Rounded to 8 levels level 0 level 7
Quantisation. A smooth wave (cyan) forced onto a coarse grid of allowed levels (magenta staircase). The gap between them is quantisation noise — the crunch you hear. More bits = more grid lines = a smaller gap.

PCM: storing the raw numbers

The most straightforward format just stores each rounded amplitude value as-is. That's called PCM — Pulse-Code Modulation, a fancy name for “the plain list of sample numbers.” 16-bit PCM sounds clean and is what CDs use, but it's big: every second of mono audio at 32 kHz is 32,000 × 2 bytes = 64 KB. For a console juggling dozens of sounds from a small memory, that adds up fast — which motivates compression.

ADPCM: storing the surprises, not the values

Here's the key insight: audio doesn't jump around randomly. From one sample to the next it usually changes only a little, and smoothly. So instead of storing each full value, you can predict where the wave is probably going and store only the small correction — the difference between your guess and reality. Because the guess is usually close, the correction is small, so it needs far fewer bits.

That's the idea behind ADPCM (Adaptive Differential PCM), and it's remarkably efficient: it fits each sample into a nibble (4 bits — a quarter the size of 16-bit PCM). Each nibble scales a step size and adds a prediction computed from the two previous samples and a small pair of per-block coefficients. Because audio is locally predictable, four bits of correction go a surprisingly long way. This is why nearly all of a console's short sound effects are stored as 4-bit ADPCM — you'll meet the hardware that decodes it in Part II.

Drag the bit-depth slider below from high to low and listen to the crunch appear as the grid coarsens; then hit the reference button to hear clean 16-bit for comparison. Watch the scope trace turn blocky — that visible staircase is quantisation.

Quantiser & ADPCMWeb Audio · live

Peeking ahead at the hardware. A dedicated decoder later reconstructs each sample with the formula val = scale·nibble + ((0x400 + c1·yn1 + c2·yn2) >> 11), clamped to 16 bits, refreshing the block's step scale every 16 bytes. Don't worry about the exact math — the point is just: scaled correction + prediction from the last two samples (yn1, yn2). You'll see the chip that runs it in Module 09.

Key takeaways
  • Bit depth sets how many amplitude levels exist; too few = audible quantisation noise.
  • PCM stores raw amplitudes (clean but large); ADPCM stores small predicted corrections in 4 bits.
  • ADPCM works only because audio is locally predictable from recent samples.
Module 05 · Part I

Mixing, gain & panning

GoalUnderstand how many separate sounds combine into one output, why volume changes must be gradual, and how a sound gets placed left or right.

A game is never playing just one sound. Footsteps, music, a distant explosion, menu blips — dozens at once. Combining them into the single two-channel stream your speakers receive is called mixing, and at its heart it is astonishingly simple: you add the sample values together. If footstep sample #500 is 1,200 and music sample #500 is −4,000, the mixed sample #500 is −2,800. Do that for every sample of every sound, and they're combined.

Gain: the volume of each sound

Before adding, each sound gets its own volume control, called its gain — simply a number you multiply every sample by. Gain 1.0 leaves a sound untouched; 0.5 halves its loudness; 0 silences it. A mixer is really just “multiply each source by its gain, then add them up.”

But there's a catch that trips up every beginner. If you change a gain instantly — say a sound is at full volume and you snap it to zero — the waveform makes a sudden vertical jump, and your speaker reproduces that jump as an audible click or pop. The fix is to ramp the gain: slide it from the old value to the new one smoothly over a few milliseconds so there's no jump. Audio hardware ramps every volume change for exactly this reason.

Analogy · A dimmer switch

Flick a light switch and the room snaps from dark to bright — jarring. A dimmer eases it up over a moment — smooth. A gain ramp is a dimmer for volume: the ear hates sudden jumps, so we always fade.

Panning: placing a sound left or right

A single recorded sound is usually mono — one channel, no sense of direction. To make it seem to come from your left, you send more of it to the left speaker and less to the right; the balance between the two channels is panning, and the left–right space it creates is the stereo field. Pan a bird call hard left and a river hard right and suddenly the flat wall of sound has a layout.

The mini-mixer below is four synth voices. Give each a volume (the tall slider) and a pan position (the wide slider), mute a couple, and the mix updates as you move the controls — the gains glide over 250 ms exactly the way real hardware ramps them, so you never hear a click. This little loop — decode, set gain, pan, sum — is precisely what a console's audio mixer does, just with 4 voices instead of 64. You'll meet that hardware version in Module 11.

Voice 1 gain pan Voice 2 gain pan Voice 3 gain pan Voice 4 gain pan Σ Mix sum voices L R volume · ramped place L↔R
Inside the mixer. Each voice is scaled by its own gain (ramped so it never clicks), placed left↔right by its pan, and then all voices are summed into the L/R output. This is the exact loop a hardware voice mixer runs — just with more voices.
Mini-mixer — four voices, ramped & pannedWeb Audio · live
Voice 1
Voice 2
Voice 3
Voice 4
Tall slider = volume · wide slider = L↔R pan · changes apply live, gains ramp over 250 ms.
Key takeaways
  • Mixing = multiply each source by its gain, then sum the samples.
  • Gains must ramp, never jump, or you hear a click (this is “de-popping”).
  • Panning splits one mono source across the two channels to place it in the stereo field.
Module 06 · Part I

Envelopes — shaping a sound over time

GoalUnderstand how a note’s loudness rises, holds and fades (the ADSR envelope), and why that shape defines an instrument’s character.

In Module 05 a voice’s volume was a single number — its gain. But real notes aren’t one steady volume. Pluck a string and it snaps loud then rings away; bow it slowly and it swells in and holds. That shape of loudness over time is called the note’s envelope, and it’s a huge part of why two instruments playing the same pitch sound completely different.

The classic way to describe an envelope has four stages, abbreviated ADSR:

Attack
How long the sound takes to rise from silence up to full volume when the note starts. Short = a sharp hit; long = a gentle fade-in.
Decay
After the peak, how long it takes to fall to the steady level below.
Sustain
The steady level held while the note is still down. Note this is a level (a volume), not a length of time.
Release
After the note is let go, how long it takes to fade from the sustain level back to silence.
A attack D decay S sustain R release loud sustain level
The ADSR envelope. A note’s loudness over time: Attack rises from silence to the peak, Decay drops to a held Sustain level, and Release fades back to silence when the note ends. Same waveform, different envelope = a pluck vs. a slow pad.
Two instruments, one waveform

A drum or plucked string: near-instant attack, no real sustain, a quick release — a sharp spike that immediately rings away.

An organ or synth pad: a slow attack, a high sustain held as long as the key is down, and a slow release — a smooth swell. Feed both envelopes the exact same sawtooth and they sound like different instruments. That’s the power of the envelope.

Under the hood, an envelope is nothing exotic: it’s just the gain ramp from Module 05 applied in a scheduled sequence — ramp up (attack), ramp down (decay), hold (sustain), ramp to zero (release) — all smooth, so no clicks. When you reach Part II you’ll find an “envelope” step sitting right in the console’s per-voice pipeline; this is that step.

Move the four sliders below and watch the envelope shape redraw live, then press play to hear a voice shaped by it. Try to dial in a percussive pluck (fast attack, low sustain, short release), then a slow pad (long attack, high sustain, long release).

Envelope lab — shape a note with ADSRWeb Audio · live
Key takeaways
  • An envelope is a note’s loudness shaped over time; ADSR = Attack, Decay, Sustain, Release.
  • Sustain is a level (how loud while held), not a duration.
  • The same waveform with different envelopes becomes different-sounding instruments.
Module 07 · Part I

Loudness, headroom & clipping

GoalLearn what happens when the mix gets too loud — clipping — and how headroom, a master gain and decibels keep sound clean.

Module 05 said mixing is just adding sample values together. That raises a question we dodged: what if the sum gets too big? Every digital audio format has a hard ceiling — the largest number it can store. For 16-bit audio that’s about ±32,767. Add several loud voices and their combined value can shoot past that ceiling… but the format simply cannot represent a bigger number.

So the hardware does the only thing it can: it clamps any over-the-ceiling value back down to the maximum. On the waveform, the peaks that tried to go higher get sliced off flat. That flat-topping is called clipping, and it sounds harsh and buzzy — a gritty distortion — because those sudden flat edges inject a spray of new high-frequency harmonics that weren’t in the original.

+max −max Clean signal (fits) Over-driven → flat-topped = clipping
Clipping. The dashed lines are the format’s ceiling (its maximum value). The clean wave (cyan) fits inside. Turn the level up too far (magenta) and the peaks slam into the ceiling and get sliced flat — that flat-topping is clipping, heard as harsh distortion. Headroom is the space you leave below the ceiling to avoid it.

The cure is headroom: deliberately leave some empty space between your normal levels and the ceiling, so that even a sudden loud moment has room to grow without hitting the top. In practice a mixer keeps a master gain comfortably below maximum, and often adds a limiter as a safety net to catch stray peaks before they clip.

Analogy · Pouring a glass

Fill a glass to the very brim and any extra just spills and makes a mess — that’s clipping. Leave a finger’s width at the top and there’s room for a splash without overflowing — that’s headroom.

One more piece of vocabulary, because it ties a neat bow on Module 04. We measure loudness in decibels (dB), a scale built to match how ears actually judge volume. The distance between the quietest and loudest a format can hold is its dynamic range. Here’s the satisfying connection: each extra bit of depth buys about 6 dB of range, so 16-bit audio spans roughly 96 dB — an enormous span from a whisper to a roar, which is exactly why 16-bit sounds so clean.

In the lab, the drive slider sets how hard a voice is pushed toward the ceiling. At ×1 it’s clean; turn it up and — live — watch the scope’s peaks flatten while the tone turns to buzz. That’s the exact moment headroom runs out.

Headroom lab — drive it into clippingWeb Audio · live
×1 fits under the ceiling; push higher and the peaks clip flat.
Key takeaways
  • Summed voices can exceed the format’s ceiling; the hardware clamps them, slicing peaks flat — that’s clipping (harsh distortion).
  • Headroom — space left below the ceiling — plus a master gain and limiter prevent it.
  • Loudness is measured in dB; each bit ≈ 6 dB, so 16-bit ≈ 96 dB of dynamic range.
Module 08 · Part I

Filters & resampling

GoalMeet the last two fundamental tools: filters, which reshape a sound's tone, and resamplers, which change its pitch — cleanly.

Two tools round out the toolkit. The first shapes tone; the second changes pitch. Both show up everywhere in Part II.

Filters: turning tone up or down

A filter lets some frequencies through while reducing others. The most common is the low-pass filter: it lets low frequencies pass and quietens the highs — the audio version of a “treble down” knob. It's how you make a sound muffled, or tame a harsh edge, or simulate something heard through a wall. Its mirror-image, a high-pass, keeps the highs and removes the lows (think a thin, tinny telephone voice); a band-pass keeps only a middle slice.

A biquad is a small, versatile filter recipe that can be configured as any of these — low-pass, high-pass, band-pass, or a “peak” that boosts one narrow band — just by changing a few numbers. It's the workhorse building block of digital tone control, and you'll see consoles hand each voice one.

Analogy · Frosted glass

Frosted glass lets soft light through but blurs sharp detail. A low-pass filter does that to sound: the body of the tone passes, the crisp high detail is smoothed away, and the result sounds muffled or distant.

gain frequency → passband lows pass through roll-off highs quietened cutoff
A low-pass filter. Low frequencies pass untouched (flat, left); past the cutoff the response rolls off, quietening higher frequencies more and more. Configure a biquad differently and this curve becomes a high-pass or band-pass instead.

Resampling: one recording, many pitches

Say you recorded a single piano note but the game needs it at ten different pitches. Re-recording all ten would waste memory. Instead you resample: to play the note higher, you step through the stored samples faster (reaching the end sooner, raising the pitch); to play lower, you step through them slower. The read position advances by a fractional step each time.

The snag: that read position almost never lands exactly on a stored sample — it lands between two of them. So you have to guess the in-between value, a process called interpolation. How well you guess is a quality ladder:

  • Nearest — just grab the closest stored sample. Cheapest, but rough and noisy.
  • Linear — draw a straight line between the two neighbours and read off the point. Noticeably smoother.
  • Polyphase FIR — the high-quality option: a small filter that blends several nearby samples using carefully precomputed weights, so it interpolates and suppresses the aliasing from Module 03 at the same time. Each weight is a tap; the weights come from a stored table.
FIR filter
“Finite Impulse Response” — a filter that outputs a weighted sum of a handful of input samples. A 4-tap FIR blends 4 samples.
Coefficient
One of the pre-calculated weight numbers a filter multiplies its samples by.
Fixed-point
A way to store fractional numbers using plain integers (e.g. a 16.16 value: 16 bits of whole part, 16 bits of fraction). Fast for cheap hardware that lacks a floating-point unit.

The high-quality resampler is what keeps re-pitched audio from turning to gravel — remember, you already heard what a bad resampler does back in Module 03. Here's the flow a good one runs for every output sample:

Read position
16.16 fixed-point pointer steps through the sample
Fractional part
Picks a set of coefficients from a table
4-tap FIR
Weighted sum of nearby samples = clean output

And that's the whole toolkit: samples, rate, bit depth, compression, mixing, filtering, resampling. Everything in Part II is these seven ideas cast into silicon. Time to meet the chip.

Key takeaways
  • Filters reshape tone: low-pass muffles, high-pass thins, a biquad can be configured as any of them.
  • Resampling changes pitch by stepping through stored samples faster or slower.
  • Since the read position lands between samples, you interpolate; better interpolation (polyphase FIR) means less aliasing.
Part II

The GameCube DSP

Now we put the Part I toolkit into real silicon. Nintendo's console routes all of its audio through one small, programmable chip. These four modules cover that chip itself, the interchangeable little programs (ucodes) that give it a job, the “AX” mixer most games rely on, and the clever trick that squeezes surround sound out of just two channels — Dolby Pro Logic II. From here on we'll name real hardware, but every piece maps back to a fundamental you already know.

Modules 09–12Difficulty HardwareLabs 3 interactive
Module 09 · Part II

The audio DSP

GoalMeet the actual chip — what a DSP is, the memories it uses, and the special hardware that decodes samples for it.

First, the term itself. DSP stands for Digital Signal Processor: a small processor built not for general computing but for doing the same simple math — multiply, add, repeat — on long streams of numbers, very fast. Since Part I showed that audio is a long stream of numbers, and that mixing and filtering are just multiply-and-add, a DSP is the perfect tool for sound.

GameCube (and Wii) audio all flows through one custom 16-bit fixed-point DSP made by Macronix. The main CPU never mixes sound itself — that would waste the processor the game needs for physics and graphics. Instead it hands the audio work to this dedicated chip and gets on with the game. The DSP uses a Harvard architecture — meaning it keeps program instructions and data in separate memories so it can read both simultaneously — and it has 40-bit accumulators sized to sum up lots of multiply results without rounding errors piling up. Nintendo never documented this chip publicly; almost everything known comes from the reverse-engineering work of Duddie & Tratax, still credited in Dolphin's source today.

GameCube audio DSP architecture The main CPU and main RAM on the left exchange messages with the DSP through two mailboxes and DMA a ucode program into the DSP's instruction RAM. Inside the DSP, four small memories (IROM, IRAM, DRAM, coefficient ROM) feed an ALU with 40-bit accumulators and a sample accelerator. The accelerator streams ADPCM or PCM samples from ARAM on the right and the DSP writes a finished stereo mix out to the audio interface and DAC. Main CPU Gekko Main RAM MRAM · 24 MB Audio DSP 16-bit · Harvard · Macronix IROM boot code IRAM ucode here DRAM work data COEF filter taps ALU 40-bit accumulators multiply · add · shift Accelerator ADPCM / PCM decode + loop 2 × Mailbox 32-bit · CPU ⇄ DSP Mix out 32 kHz stereo + S ARAM audio RAM · 16 MB AI → DAC to speakers mailbox messages DMA ucode → IRAM sample stream
Press ▶ to watch the boot handshake, then one 5 ms audio frame after another — slowed ~900×.
The whole chip at a glance. The CPU talks to the DSP only through two mailboxes, and loads a program by copying it into IRAM. The accelerator streams samples from ARAM, the ALU does the Part I mixing/filtering math, and a finished 32 kHz stereo mix (plus a surround channel, Module 12) heads out to the DAC.

The chip's memories are tiny by modern standards — each of the main ones is only a few thousand 16-bit words:

Word size
16-bit
Fixed-point, 40-bit accumulators
IRAM
0x1000 words
Instruction RAM — ucode copied here
IROM
0x1000 words
Boot ROM
DRAM
0x1000 words
Data RAM
Coefficient ROM
0x800 words
Resampling filter taps
Opcodes
230
Many carry a parallel load/store
IRAM8 KB
IROM8 KB
DRAM8 KB
COEF4 KB

What its code actually looks like

Those 230 opcodes aren’t general-purpose — they’re shaped around one job: multiply, accumulate, and keep the data moving, every single cycle. Here’s an illustrative fragment in the style of Dolphin’s disassembler — one tap of the resampling filter from Module 08:

; one tap of the 4-tap resampler FIR — the Module 08 math, in silicon
BLOOP   $AX1.H, tap_done   ; hardware loop — repeats with ZERO branch cost
  LRRI  $AX0.H, @$AR0     ; load filter coefficient from COEF ROM, pointer++
  LRRI  $AX0.L, @$AR1     ; load audio sample from DRAM,         pointer++
  MULAC $AX0.H, $AX0.L, $ACC0  ; ACC0 += coef × sample — into the 40-bit accumulator
tap_done:
SRRI    @$AR2, $ACC0.M    ; store the mixed sample, pointer++ — done, next voice

Notice what’s free: the loop has no branch overhead, and most arithmetic ops carry a parallel load or store in the same instruction. That’s why a 16-bit chip from 2001 could mix 64 voices in real time.

The star of the chip is the block labelled accelerator. The DSP's own memory is far too small to hold a game's sounds, so those live in a separate 16 MB pool called ARAM (Audio RAM). The accelerator is hardware that streams compressed samples straight out of ARAM and decodes them on the fly — this is where the 4-bit ADPCM math from Module 04 physically happens. It understands ADPCM plus 8- and 16-bit PCM, tracks where a looping sound starts and ends, and on each loop restores the predictor's state (pred_scale, yn1, yn2) so the waveform continues seamlessly without a click.

Two boot ROMs. Actually running the DSP for real (Part III's LLE) needs its built-in IROM and coefficient ROM. Dolphin verifies them by hash — the genuine Nintendo dump is irom 0x66f334fe / drom 0xf3b93527 — and ships a clean-room “free” replacement for people who can't legally dump their own console's ROMs.

Key takeaways
  • A DSP is a small chip specialised for fast multiply-add on number streams — ideal for audio.
  • One 16-bit Macronix DSP handles all console audio; the CPU just feeds it work.
  • It has four tiny memories (IRAM, IROM, DRAM, coefficient ROM) and 40-bit accumulators.
  • The accelerator is dedicated hardware that streams and decodes ADPCM/PCM from ARAM.
Module 10 · Part II

One chip, many ucodes

GoalSee how the very same DSP is re-tasked by small uploadable programs — for audio, memory cards, even link-cable crypto.

Here's the surprising part: the DSP ships blank. There is no single, fixed “GameCube sound engine” baked into it. Instead, each game uploads its own little program that tells the chip what to do. That program is called a ucode — short for “microcode,” a tiny program written in the DSP's own instruction set.

At power-on the DSP runs a minimal built-in ROM ucode that just waits for the CPU to hand it a real program. The CPU DMAs (bulk-copies) the ucode into the DSP's instruction RAM and tells it to jump in and start running. Because the program is swappable, the same physical chip can be re-tasked mid-game: one moment it's mixing audio, the next it's driving the memory card, the next it's handling the GameCube-to- Game Boy Advance link cable. All of this coordination rides on two 32-bit mailboxes using a small message protocol (you'll watch the full handshake in Module 11).

Analogy · A pocket calculator that swaps its brain

Imagine a calculator where you could pop out its logic and drop in a different one — now it's a stopwatch, now a thermometer, now a music player — all using the same buttons and screen. The DSP is that calculator: identical hardware, completely different behaviour depending on which ucode you load.

How does Dolphin's high-level emulator (Module 13) know which ucode a game just uploaded? It computes a CRC — a short fingerprint number — of the ucode's code and looks it up in a table. Here are the main families, with real fingerprints and the games behind them, straight from Dolphin's dispatch table.

AX audio mixer Zelda synth ucode CARD memory card GBA link crypto DSP same blank chip upload one → IRAM mixes game audio drives the memory card runs GBA link crypto
One chip, many jobs. The DSP itself is blank. Whichever ucode the CPU uploads into its instruction RAM decides what it becomes — an audio mixer, a memory-card driver, or Game Boy Advance link-cable crypto — all on the identical hardware.
UcodeExample CRCWhat it doesSeen in
AX 0x4e8a8b21
0x3ad3b7ac
The workhorse GameCube audio mixer — linked-list voices, ADPCM, resampling, aux buses, compressor. Melee, Monkey Ball, Paper Mario TTYD, F-Zero GX, Zelda: OoT (GC)
AXWii 0x347112ba
0xfa450138
AX extended for Wii: an extra aux bus, Wiimote-speaker channels, a biquad, explicit DPL2 output. Wii Sports, Raving Rabbids, Elebits, Wii System Menu
Zelda 0x86840740
0xd643001f
The "synth"/IPL ucode with real positional Dolby matrix mixing driven by a quarter-wave sine table. Wind Waker, Twilight Princess, Mario Sunshine & Galaxy, Pikmin, GC BIOS
CARD 0x65d6cc6f Not audio — drives memory-card access, then returns the DSP to the ROM ucode. Any save/load
GBA 0xdd7e72d5 GBA JoyBoot cipher for the link cable — pure crypto, no sound. Four Swords, Crystal Chronicles
ROM / INIT 0x00000000
0x00000001
The boot-ROM stub and audio-system initialiser — internal handshakes. Every boot
ASnd / AESnd HASH_2008… Homebrew sound libraries (libogc / libaesnd), matched by named hash constants. Homebrew & demos

Dolphin's table lists dozens more CRCs — one per known regional build. An unlisted fingerprint triggers a “try LLE” warning and a best-effort fallback.

The stress test: Factor 5. The Rogue Squadron games (Rogue Leader, Rebel Strike) shipped their own radically customised ucode, grown out of the MusyX middleware — streaming-heavy, voice-hungry, and nothing like AX inside. It was long the textbook case for Module 13’s advice: HLE’s dispatch table had no implementation that behaved right, so to get correct audio you flipped to LLE and simply ran Factor 5’s real code on the emulated chip.

Key takeaways
  • The DSP is a blank worker; an uploaded ucode gives it a job.
  • Audio ucodes come in families: AX (GameCube), AXWii (Wii), and Zelda/IPL synth.
  • Non-audio ucodes (CARD, GBA) share the very same chip.
  • Dolphin identifies each ucode by a CRC fingerprint of its code.
Module 11 · Part II

AX: the voice pipeline

GoalTrace a single sound through the AX ucode and watch it use, in order, every fundamental from Part I.

AX is the ucode most GameCube games use for sound, so it's worth knowing well. In audio, one playing sound — one note, one footstep — is called a voice. AX doesn't have a fixed number of voices baked into hardware; instead it walks a linked list of Parameter Blocks, where each Parameter Block (PB) describes one voice — its sample, pitch, volume, pan, and so on. It follows the chain until it ends, so the practical ceiling of about 64 voices is a matter of how much CPU time you want to spend, not a hard silicon limit. AX always does its math at 32 kHz, in small batches called frames of 5 ms × 32 samples on GameCube (3 ms × 96 on Wii).

Here's the satisfying part. Every voice runs the exact same five-step pipeline — and you already learned every step in Part I:

(Mod 09) ARAM samples (Mod 04) Decode ADPCM/PCM (Mod 08) Resample pitch (Mod 06) Envelope ramp (Mod 08) Filter low-pass (Mod 05) Mix → L/R/S one voice · repeated for ~64 voices every 32 kHz frame
The AX voice pipeline. Every voice flows through the same chain — and each stage is a Part I fundamental (shown in brackets). AX walks a linked list of these voices every frame, then writes stereo plus a matrix-encoded surround channel.
  1. Fetch & decode — the accelerator streams ADPCM/PCM from ARAM (Modules 04, 09).
  2. Resample — change the pitch with a 16.16 read pointer; nearest, linear, or a 4-tap polyphase FIR (Module 08).
  3. Envelope — a per-sample volume ramp so gains glide, never click (Module 05).
  4. Filter — an optional low-pass (plus a biquad on Wii) (Module 08).
  5. Mix — summed into the MAIN L/R/S output and the extra AUX buses, each switched on or off by a mix-control bitfield (Module 05).

The finished result is clamped to 16 bits and written out as interleaved big-endian R, L sample pairs, plus a separate surround buffer carrying a special encoded channel — which is exactly what the next module explains. If you want to feel how this works, re-open the Module 05 mixer with this five-step list in mind: that little lab is AX in miniature, one step per control.

One more Part I echo before we move on: as its very last step, AX runs the finished mix through a compressor stage — a table-driven limiter, configured by the game. Dozens of summed voices can easily overshoot the 16-bit ceiling, and Module 07 taught you what that sounds like — so the ucode tames the peaks instead of letting them clip. The headroom lesson, shipped in microcode.

The AUX detour — where reverb lives

The pipeline above actually ends in more than one place. Besides the MAIN output, every voice can send a copy of itself into the AUX buses (AUXA/AUXB, plus AUXC on Wii) — and here's the surprise: AX doesn't process them. Each frame it DMAs the AUX mixes back to main RAM, where the CPU applies the effect — reverb, echo, chorus — and returns the processed buffers to be folded into the next frame's mix. Every echoing cave and cathedral on the console is this round-trip: the DSP does the per-voice grind, the CPU does the fancy effects. Try it — the send knob below is exactly a voice's AUX send level, feeding a synthetic cave (the reverb is a generated impulse response; nothing is recorded).

AUX send — put the arpeggio in a caveWeb Audio · live
0 % = dry voice only · slide up while it plays and the cave grows around it.
DSP · AX mixes voices Main RAM AUX buffer CPU runs the reverb send fetch wet buffer back fold into NEXT frame frame 214
The reverb detour, animated — note the frame counter: the wet signal always lands one frame late.
The AUX round trip. The DSP hands its side-mix to main RAM, the CPU applies the effect, and the processed buffer is folded into the next frame’s mix — a division of labour that costs one 5 ms frame of latency, which nobody can hear on a reverb tail.

The frame budget — why ~64 voices

Remember that AX works in frames: every 5 ms it must fetch, decode, resample, envelope, filter and mix every voice before the output buffer runs dry. That is a hard real-time deadline. Each voice costs cycles, so the frame is a budget — and when a busy scene pushes past it, frames miss the deadline, the buffer underruns, and the audio crackles and drops out. That's why the practical voice ceiling was ~64, and why sound designers spent their budget like money. The lab below simulates the deadline (your computer isn't actually struggling): each bar is one 5 ms frame, the dashed line is the budget, and red frames missed it — which mutes them, exactly the crackle a real overrun makes.

Frame budget — blow the 5 ms deadlinesimulation · live
budget ≈ 64 voices · slide past it and hear frames drop.
Key takeaways
  • A “voice” is one playing sound; AX walks a linked list of Parameter Blocks, one per voice.
  • It runs at a fixed 32 kHz in millisecond-sized frames.
  • Each voice = decode → resample → ramp → filter → mix — the entire Part I chain, in order.
  • It outputs stereo plus a separate encoded surround buffer.
  • Effects take the AUX detour: the DSP sends side-mixes to the CPU for reverb and folds them back in.
  • Every frame is a 5 ms deadline — overrun the voice budget and the audio drops out.
Module 12 · Part II

Dolby Pro Logic II — surround from two channels

GoalUnderstand matrix encoding: how four surround channels fold into an ordinary stereo pair, and how a decoder pulls them back out.

Here's a genuine puzzle. The GameCube's audio output carries only two channels — plain left and right. Yet games advertised surround sound with speakers behind you. How can four directions of sound (left, right, front-centre, rear) travel down just two wires? The answer is a clever trick called matrix encoding.

The idea: fold the four channels into one ordinary stereo pair in a specific, reversible way, so that a matching decoder on the other end can tease them back apart. Two rules do the work:

  • The centre channel (dialogue, usually) is mixed equally into both left and right. If a decoder sees identical sound in both channels, it knows that's centre, and steers it to a centre speaker — a “phantom centre.”
  • The surround (rear) channel is mixed in out of phase — added to one channel and subtracted from the other — and slightly delayed. A decoder spots the anti-matched part and sends it to the rear speakers.

So the decoder reconstructs the centre from what left and right share, and the surround from what's opposite between them. Nothing is truly “hidden” — the directions are encoded in the relationships between the two channels.

The original Dolby Surround / Pro Logic recovered a single mono rear. Pro Logic II goes further: with active steering it teases the surround back apart into a real stereo pair (left- and right-surround), so a decoder can pan a sound across the rears — which is why DPL2 titles like Rogue Leader place effects distinctly rear-left or rear-right. (Separation isn't as clean as discrete 5.1, but it's convincingly there.)

Analogy · 3-D glasses

A 3-D movie sends two overlapping images down to one screen; the glasses pull them apart into left-eye and right-eye views to create depth. Matrix encoding does the same for sound: extra channels are layered into the stereo pair, and a Dolby Pro Logic II decoder is the glasses that separate them again.

L R C S Matrix encode Lt Rt 2 wires only Pro Logic II decode L R C S 4 source channels 4 channels rebuilt C → both equally · S → out of phase
Matrix surround. Four channels are folded into an ordinary stereo pair — the centre added equally to both, the surround added out of phase — sent down just two wires, and a Pro Logic II decoder rebuilds the four directions from how left and right relate.

On the real console, this encoding happens inside the ucode. The Zelda ucode does the most literal version: every voice carries a Dolby position, and a quarter-wave sine table turns that X/Y position into left/right/front/back quadrant volumes (some ucodes even set a “make Dolby louder” flag). AX carries a matrix-encoded surround bus alongside the main stereo. Drag the source around the room below and watch the four channel meters respond; press play to actually hear the phase-inverted surround cue (headphones help).

Pro Logic II panner — steer a source around the roomWeb Audio · live
L R C S
L
R
C
S

Top-centre feeds a phantom centre; drag to the bottom and the source is sent phase-inverted to both channels — the cue a Pro Logic II decoder steers to the rears. This lab is a real end-to-end PLII chain: the puck is encoded into plain stereo using PLII's asymmetric surround coefficients (0.87 / 0.49) — that asymmetry is how rear-left vs rear-right survives the two-channel trip, and it's why games like Rogue Leader can pan across the rears. Tip: switch the source to 🗣 “can you hear me?” — human hearing localises speech far better than a steady tone.

🎧 Decode surround — on or off? The encoded stereo needs exactly one decoder; pick where it runs:

  • Old, hardware DPL2-capable gear (e.g. a Yamaha RX-V481 receiver), modern Dolby Surround-capable equipment (e.g. a Samsung HW-Q990 soundbar — say, over AirPlay), or a Smyth Realiser A16 (including the Speaker Edition): keep it off. The page sends the raw encoded stereo and your hardware's decoder does the work. Windows tip: if that raw stereo leaves a PC, set the sound panel's output configuration to stereo — configured as 5.1, Windows remixes the two channels across the speaker layout and corrupts the matrix on the way out.
  • AirPods (Pro / Max): keep it off here too — enable Apple's Convert to Spatial Audio from Control Center instead, and iOS renders the stereo (surround cues included) with head tracking. Running the page's decoder and Apple's spatialiser at once double-processes and smears the image.
  • Multi-channel but non-DPL2-capable gear (e.g. a PC driving standard analogue 5.1 speakers): turn it on — the page is then the only matrix decoder in the chain, and it sends the five decoded channels discretely to your speakers — real rears, no headphone processing.
  • A classic pair of stereo headphones (wired or non-spatial Bluetooth — including an iPhone with non-spatial headphones such as the classic white EarPods): turn it on — the five decoded channels are binauralised: each virtual speaker is rendered through an HRTF, so the fronts sit ahead of you and the surrounds behind. This is the ideal way to hear this lab.

(Plain stereo speakers aren't a surround endpoint at all: two fixed loudspeakers can't reproduce discrete rears, and the room mixes both channels into both ears, which destroys the binaural cues too. If that's your setup, grab headphones.)

With decode on the page runs its own active-steering decoder live: it finds the dominant direction, cancels crosstalk toward the other speakers, and renders five virtual speakers binaurally — drag to a rear corner and it follows; diffuse sound relaxes back to the passive matrix. (Approximations vs real Dolby: polarity + a short delay stand in for the 90° phase quadrature, and an analytic gating law replaces Dolby's exact coefficient tables.) With decode off you hear the raw encoded stereo — at the very bottom it largely cancels on a mono/phone speaker; that cancellation is the cue.

On playback, Dolphin re-expands that two-channel stream into real 5.1 surround with a bundled decoder called FreeSurround — but only under LLE, and only on supported backends. Why the LLE restriction? That's the first module of Part III.

Key takeaways
  • Surround is matrix-encoded into two channels: centre goes in equally, surround goes in out-of-phase.
  • The relationships between L and R carry the extra directions; a decoder reverses the encoding.
  • The Zelda ucode positions voices with a sine-table quadrant panner; AX carries a surround bus.
Part III

Emulating it in Dolphin

Finally: how does software running on your PC stand in for that physical chip? Dolphin offers two very different answers, decides which ucode a game loaded by fingerprinting it, and funnels everything through one resampling mixer to your speakers. These four modules connect the whole course to the actual settings you'll toggle in the emulator.

Modules 13–16Difficulty EmulationLabs 7 interactive
Module 13 · Part III

LLE vs HLE — two philosophies

GoalGrasp the single most important choice in audio emulation: actually run the chip, or just imitate what it produces.

To emulate the console's sound, you face one big fork in the road. You can either recreate the DSP so faithfully that you run the game's real ucode instruction by instruction — or you can skip the chip entirely and just write software that produces the same output. These two paths have names, and they're literally a dropdown in Dolphin's audio settings.

LLE is Low-Level Emulation: emulate the actual hardware down at the level of individual instructions. HLE is High-Level Emulation: work at the level of what the code accomplishes, reimplementing it directly. Neither is “more real” sound — both make audio — they just disagree about where the work happens.

Analogy · Translating a recipe

LLE is following a foreign recipe word-for-word through a dictionary, every step exactly as written — slow, but you get precisely the original dish. HLE is a chef who already knows that dish and just cooks it from memory — much faster, but only for dishes they've learned.

LLE Low-Level Emulation

Emulate the actual DSP. Load the real ucode into virtual IRAM and execute its instructions one by one.

  • Bit-exact: whatever the game's ucode did, you get.
  • Runs homebrew & obscure ucodes with no special-casing.
  • Enables true Dolby Pro Logic II output.
  • Needs a DSP ROM dump; costs more CPU.
  • Two engines: an interpreter (slow, for debugging) and an x64 recompiler / JIT (the practical choice).
Slower, but the reference for accuracy.

HLE High-Level Emulation

Skip the DSP. Recognise the ucode by a fingerprint of its code and run a hand-written C++ reimplementation of what it does.

  • Fast — no instruction-level emulation at all.
  • No DSP ROM required.
  • Only works for ucodes Dolphin has been taught (AX, AXWii, Zelda…).
  • An unknown ucode falls back to “force AX/AXWii” and may misbehave.
  • Historically lacks full Pro Logic II decoding.
Dolphin's default — “if unsure, select HLE.”
Game uploads a ucode LLE — run the chip Load into virtual IRAM Execute ~230 opcodes one instruction at a time Sound — accurate · needs ROM HLE — fake the chip Fingerprint (CRC) the code Run a C++ class AXUCode, ZeldaUCode … Sound — fast · no ROM
Two ways to run the same ucode. LLE loads the code into a virtual chip and executes its real instructions one by one — bit-exact, but it needs the DSP ROM and more CPU. HLE fingerprints the code and runs a C++ reimplementation of what it does — fast, and no ROM required.
Where does the work happen?toggle

LLE: the game ucode is loaded into virtual IRAM. Dolphin's DSP core fetches, decodes and executes each of the ~230 opcodes — arithmetic, the accelerator, the mailbox handshake — exactly as silicon would. The C++ never 'knows' it's mixing audio; it just runs a DSP.

Reading about “executing the real instructions” is one thing; doing it is another. Below is a tiny slice of DSP program — the inner loop of one voice being scaled by its volume, written in the chip’s real style of instructions. Press Step and you are performing LLE by hand: fetch one instruction, execute it, update the registers. What Dolphin’s LLE core does millions of times per second, you do at one press per instruction.

Be the emulator — run 4 samples through the DSP, one instruction at a timeinteractive
Press Step to fetch the first instruction.

In Dolphin the combo box maps to two flags — DSP HLE and DSP JIT. HLE = HLE on. LLE Recompiler = HLE off, JIT on. LLE Interpreter = both off. Pro Logic II is only offered when HLE is off.

What HLE-gone-wrong sounds like. Because HLE re-implements each ucode by hand, its failures are oddly specific: crystal-clear sound effects but missing music (one command list handled, another not), wrong instruments (a mis-read sample header), missing reverb (the AUX detour skipped) — or, the classic, a Factor 5 game booting to total silence. Under LLE those bugs are impossible by construction: there is no reimplementation to get wrong.

How Dolphin got its sound

The LLE/HLE fork isn’t an abstract design choice — it is twenty years of hard-won history:

2003
Silence. Dolphin begins life as a closed-source experiment. Audio is an afterthought — many games boot mute, and getting any sound often means a hack for that one game.
2008
Open source. The code is released and the DSP finally gets serious attention: the instruction set is reverse-engineered and an LLE interpreter runs genuine ucodes — accurate, but it devours an entire CPU core.
~2010
LLE goes fast. The DSP recompiler (the JIT in Module 13’s dropdown) translates DSP code into native x64 on the fly: the same bit-exact accuracy at playable speed.
2012
HLE done right. The “new-AX-HLE” effort rewrites HLE’s guesswork: instead of patching one game at a time, faithfully reimplement each ucode family — and whole libraries of titles get correct music, reverb and streaming at once.
today
Two engines, one answer. HLE is accurate enough to be the default; LLE remains the bit-exact reference — and the Pro Logic II path. They mostly agree, which is exactly how you know the reverse engineering is done.
Key takeaways
  • LLE runs the DSP's real instructions; HLE reimplements each ucode directly in C++.
  • HLE is faster and needs no ROM; LLE is more accurate and enables DPL2.
  • Unknown ucodes are LLE's strength and HLE's weakness.
Module 14 · Part III

Boot, mail & ucode dispatch

GoalFollow the exact handshake that loads a ucode onto the DSP, and how HLE picks which C++ implementation to run.

Let's watch a ucode actually get loaded — the mechanism behind Module 07. At power-on the DSP is running its built-in boot ROM ucode, waiting. The CPU kicks things off by sending an init message (0x8071FEED) through a mailbox, followed by a small descriptor: where in main memory the real ucode lives, how long it is, and where in the DSP's instruction RAM to put it. The boot ROM ucode reads that, DMAs the program into IRAM, and jumps to it. A single game swaps between audio, memory-card and GBA ucodes just by repeating this with different code.

All the back-and-forth rides those two 32-bit mailboxes using a small protocol of tagged messages — DSP_INIT, DSP_RESUME, DSP_YIELD, DSP_DONE, DSP_SYNC, DSP_FRAME_END, MAIL_NEW_UCODE. Nintendo's official SDK and the homebrew libogc handler share this same AX-derived protocol.

Init mail
CPU → DSP: 0x8071FEED + descriptor
DMA to IRAM
ROM ucode copies the program in
Hash (HLE)
Ector CRC of IRAM image
Dispatch
Match → AXUCode / ZeldaUCode / …

That third step is where HLE makes its decision: it fingerprints the freshly-loaded IRAM and looks the result up. A match instantiates the matching C++ class — AXUCode, ZeldaUCode, and so on.

Under HLE, an unrecognised fingerprint triggers a panic (“might be incompatible with DSP HLE… try LLE if this is homebrew”) and a best-effort fallback to AX. Under LLE there's no lookup at all — it just runs whatever code arrived.

That fingerprint deserves a closer look, because HLE’s whole strength and fragility hide inside it. Below are the first 32 bytes of a make-believe ucode, freshly DMA’d into IRAM. Dolphin hashes the image and looks the result up in its table of known ucodes. Try corrupting a single byte — then imagine you are a developer who merely recompiled your audio engine.

The fingerprint — one byte between music and silenceinteractive
hash of IRAM image →

Now watch it happen. The simulator below replays the handshake one mail at a time — the actual 32-bit values a GameCube pushes through the mailboxes at boot, with a translation beside each.

Mailbox simulator — boot a ucode, one mail at a timeinteractive
— mailboxes idle · the DSP is running its boot ROM ucode —
Press Step to send the first mail.
Key takeaways
  • The boot ROM ucode loads the real ucode via a mailbox handshake + a DMA copy.
  • Two mailboxes carry a tagged message protocol shared by Nintendo's SDK and libogc.
  • HLE dispatches by fingerprinting the loaded IRAM; LLE just executes whatever arrived.
Module 15 · Part III

The output path

GoalFollow the finished samples from the DSP, through Dolphin's mixer, and out to a real sound card.

No matter which side of Module 10 produced the samples — LLE or HLE — they all arrive at the same place: Dolphin's Mixer. Its job is to reconcile two audio streams that don't even run at the same speed, and then hand a single clean stream to your sound card.

Two streams merge here. The DSP's own audio runs at 32 kHz, while streaming music read straight off the game disc — called DTK (Disc Track) audio — runs at 48 kHz. Your sound card, meanwhile, probably wants 44.1 or 48 kHz. So the mixer must resample both streams to a common rate (Module 08, back again). It uses a windowed, granule-based 6-point Hermite interpolator — the practical cousin of Module 08's polyphase filter — then applies volume, dithers back down to 16-bit, and hands off stereo (or 6-channel float for surround) to the backend.

DSP / DMA audio
32 kHz
AX & Zelda output
Streaming (DTK)
48 kHz
Disc / ADPCM tracks
Resampler
6-pt Hermite
256-sample granules, 50% overlap
DPL2 block
512–4096
FreeSurround quality setting
DSP output
32 kHz stereo, big-endian R/L
Mixer FIFO
Granules + Hermite resample
FreeSurround
Optional DPL2 → 5.1 float
Backend
Cubeb / OpenAL / WASAPI…

That last box, the backend, is simply whichever audio system your operating system provides. Dolphin defaults to Cubeb, but can also use OpenAL, ALSA or PulseAudio (Linux), WASAPI (Windows), or OpenSL ES (Android). Each one does the same simple thing: repeatedly ask the mixer for the next block of samples and pass it to the hardware. Dolphin can also dump either stream to a WAV file if you want to inspect it.

Hear the Mixer’s job

Module 08 gave you the ladder of interpolation quality — nearest, linear, polyphase — but only on paper. Here it finally gets ears. The lab stores a bright sweep at the DSP’s 32 kHz and converts it to your device’s rate three different ways: with nearest (grab the closest sample — listen for the gritty, metallic edge as the sweep climbs), linear (straight lines between samples — duller artefacts), and a 4-point Hermite curve, the small cousin of the Mixer’s 6-point interpolator — clean to the top.

Resampler A/B — 32 kHz → your device, three waysWeb Audio · live

Hear the deadline being missed

One more thing deserves ears: the top row of Module 16’s troubleshooting table. When the emulated console can’t keep up, the Mixer’s buffer runs dry and the sound card gets handed silence — over and over, dozens of times a second. Play the same melody three ways: at full speed; from an emulator running at 70 %, where every gap is a buffer under-run; and rescued by audio stretching, which slows the sound itself down to match — continuous and pitch-correct, at the price of a slightly warbly texture.

Buffer starvation — full speed vs 70 % vs audio stretchingWeb Audio · live
Key takeaways
  • DSP audio (32 kHz) and streaming DTK audio (48 kHz) merge in one mixer.
  • A 6-point Hermite resampler converts everything to the sound card's rate.
  • Optional FreeSurround expands stereo to 5.1; the backend (Cubeb, WASAPI…) just pulls blocks.
Module 16 · Part III

Try it in Dolphin

GoalTurn the whole course into a few settings you can toggle and A/B for yourself.

You now have the full picture, so let's make it hands-on. Open Config → Audio in Dolphin and experiment — every option here is something you can now explain:

  • DSP Emulation Engine → HLE — the fast default (Module 13). Good for almost everything.
  • → LLE Recompiler — switch here if a game sounds wrong under HLE, if you're running homebrew with an unknown ucode, or if you want Dolby Pro Logic II. Requires a DSP ROM.
  • → LLE Interpreter — for DSP debugging only; very slow.
  • Dolby Pro Logic II decoder — appears only with LLE on a supported backend; pick a quality level to trade CPU for a bigger FreeSurround block.

What LLE actually needs. The two ROMs you met in Module 09: dsp_rom.bin (the boot IROM) and dsp_coef.bin (the resampling coefficient ROM), dropped into Dolphin’s User/GC folder and verified against the hashes from that module (irom 0x66f334fe / drom 0xf3b93527). No dump? Dolphin quietly falls back to its bundled clean-room replacement — audio still works, with minor differences in a few titles.

If something sounds wrong

SymptomLikely causeTry
Crackles & stutters everywhere The game isn’t running full speed, so the audio buffer starves — Module 11’s deadline, replayed on your PC Lower demanding settings; enable Audio Stretching
Music missing, or wrong instruments A quirk in HLE’s hand-written reimplementation of that ucode Switch to LLE Recompiler
“Unknown ucode” warning Homebrew, or an exotic engine (Module 10’s Factor 5 story) LLE Recompiler
No Pro Logic II option in the list HLE is selected, or the backend can’t deliver it LLE + a supported backend (e.g. Cubeb, WASAPI)
LLE makes the game itself stutter The emulated DSP is fighting the CPU thread for time Enable “DSP LLE on dedicated thread”
Config → Audio: pick an engine Sounds right under HLE and you don't need surround? yes HLE fast default no Want DPL2, homebrew, or maximum accuracy? yes LLE Recompiler needs DSP ROM just debugging the DSP? LLE Interpreter — very slow
Which engine? Stay on HLE unless you need Dolby Pro Logic II, are running homebrew with an unknown ucode, or want maximum accuracy — then switch to LLE (Recompiler for normal use; the Interpreter is debugging-only).

A great final exercise ties Parts II and III together: boot a Zelda-ucode title (Wind Waker, Twilight Princess) under HLE, then again under LLE with DPL2 turned on, and compare the surround field. The difference you hear is the matrix encoding from Module 12 coming alive — the whole course, audible in a single A/B test.

Final exam — six broken consoles

Six situations, straight out of real bug reports. You’re the one at the keyboard: pick the setting you’d reach for. Every answer points back at the module that taught it.

Diagnose it — what do you change?quiz · 0 / 6
Course complete ✦
  • You can read the signal path from a pressure wave all the way to your speakers.
  • You know why one small DSP plays audio, saves games, and boots a GBA.
  • You can choose LLE vs HLE — and DPL2 — and predict exactly what each will do.
  • Ready to make the DSP mix your sounds? The homebrew course teaches you to write and run your own software on this machine.

An interactive course built for the Dolphin project. Every sound here is generated on the fly with the Web Audio API to illustrate a concept — plus one short synthetic speech clip (“can you hear me?”) we generated with the open-source Piper TTS (voice: jenny_dioco). This page contains no game audio. Technical details are grounded in Dolphin's own source.

Source of truth · Source/Core/Core/DSP/ · Source/Core/Core/HW/DSPHLE/UCodes/ · Source/Core/AudioCommon/

Now playing
Nothing playing
Web Audio · synth
no game audio