Let's start with what sound physically is. When something vibrates — a guitar string, your vocal cords, a speaker cone — it pushes and pulls on the surrounding air, making tiny, fast changes in air pressure. Those changes ripple outward as a wave and reach your eardrum, which vibrates in step. That's all sound is: a wobble in air pressure over time.

To put that wobble in a computer we do something wonderfully simple. Many thousands of times per second we measure the pressure and write the number down. Each measurement is called a sample. String the samples together in order and you have the whole sound as a plain list of numbers; feed that list back to a speaker at the same speed and the sound reappears.

Three numbers describe any simple wave. Its frequency is how many times it repeats each second, measured in hertz (Hz) — 440 Hz means 440 cycles a second, and higher frequency means higher pitch. Its amplitude is how far the wave swings from rest — how loud it is. And its phase is where in the cycle it happens to start. Hold on to frequency and amplitude; they run through the whole course.

The basic shapes

A synthesiser like the one on this page builds everything from a few shapes. A sine is the purest tone — one frequency, no extra colour, like a tuning fork. Square and saw(tooth) are jagged and buzzy — those classic chiptune “bleeps” — because they're secretly a sine plus a pile of higher tones (Module 03). Triangle sits in between, mellow but a little richer than a sine. Noise is every frequency at once with no pitch — your “shh”: wind, static, a snare. Play with the scope: switch shapes, sweep the frequency, and watch the trace while you listen.

The trace is an oscilloscope: it draws amplitude (up/down) against time (left/right), so you're literally seeing the stream of sample values as a line.

Storing sound as numbers raises two questions. How often do we measure, and how precisely do we write each measurement down? The answers are the sample rate and the bit depth, and together they define any audio format — including the SNES's.

The sample rate is measured in samples per second, again in hertz. There's a hard rule about it, the Nyquist limit: to capture a frequency you must sample more than twice as fast. You've seen this rule with your eyes: in old films a fast-spinning wagon wheel can appear to crawl or even roll backwards, because the camera's frames sample the motion too slowly — the flipbook from Module 01 with too few pages. Sample sound too slowly and high frequencies do the same trick: they don't just vanish — they fold back down and reappear as false, lower tones. That folding is called aliasing, and it sounds like a metallic, warbly grunge riding on top of the sound.

Bit depth is the other knob: how many distinct values a sample can take. 16-bit gives 65 536 possible levels; drop to 8-bit and you have only 256, so every sample must snap to the nearest available rung. That rounding is quantisation, and the error it introduces is heard as a gritty hiss — quantisation noise. The lab below lets you strangle both knobs on a bright tone. Drop the rate to hear aliasing; drop the bits to hear the crunch. The reference button plays it back at the SNES's own 32 kHz / 16-bit.

In Module 01 we said a square or saw wave is “secretly a sine plus a stack of higher tones.” That wasn't a metaphor. It's one of the most important facts in audio, and we owe it to the mathematician Fourier: any sound, however complicated, is just a sum of simple sine waves at different frequencies and volumes. Add the right sines and you can build any waveform; pull a sound apart and you always find sines underneath.

The lowest sine is the fundamental — the pitch you actually hear. The sines stacked above it, at 2×, 3×, 4×… the fundamental's frequency, are the harmonics. Their pattern of loudnesses is what makes a flute sound different from a trumpet at the same pitch. A pure sine has no harmonics at all; a saw has every harmonic, each a bit quieter than the last; a square has only the odd ones.

Slide the first eight harmonics up and down and listen to the timbre change, or hit a preset to snap to a sine, a saw, a square or a reedy organ. This is additive synthesis — building sound by piling sines up. (The SNES doesn't synthesise this way; it plays recorded samples. But understanding harmonics is what lets you read why one sample sounds bright and another dull.)

The SNES has only 64 KB of audio RAM (Part II), so every sample must be tiny. Raw 16-bit audio is far too fat, so the chip stores everything in a compressed form. The trick is ADPCM — Adaptive Differential PCM. Instead of storing each sample's full value, ADPCM predicts the next sample from the previous ones and stores only the small correction, in just a few bits. Sound changes smoothly, so the corrections stay small, and a handful of bits is plenty.

The SNES's specific flavour is called BRR — Bit Rate Reduction — and it's beautifully rigid. Samples are packed into 9-byte blocks, each holding exactly 16 samples. The first byte is the header; the remaining eight bytes hold sixteen 4-bit signed nibbles, one per sample — each a tiny signed value spanning just −8…+7.

Decoding is where the “adaptive” and “predictive” parts live. Each 4-bit nibble is first scaled up by the header's shift (0–12). Then one of four filters reconstructs the real sample by adding a prediction from the previous one or two decoded outputs:

• Filter 0 — no prediction; the nibble stands alone. Used at the start of a sample so nothing leaks in from before.
• Filter 1 — adds about 0.94× the previous sample. A gentle slope-follower.
• Filters 2 and 3 — use the previous two samples for a stronger prediction, ideal for smooth, tonal material.

Let's decode one sample by hand, with real numbers. Say the header set shift = 8 and filter 1, the previous decoded sample came out as 1000, and the next nibble is −3. First the scaling, (nibble << shift) >> 1: shift −3 left by 8 places to get −768, then halve it → −384. Next the prediction: filter 1 contributes about 15⁄16 of the previous sample, ⌊1000 × 15⁄16⌋ = 937. Add them: 937 − 384 = 553. That's the reconstructed 16-bit sample — sixteen of those little sums and the block is done.

Two header flags finish the picture. END marks the last block of a sample; LOOP, when set on that final block, tells the hardware to jump back to a loop point instead of going silent (Module 10). One subtlety hides there: filters 1–3 lean on samples decoded before the block, so the block at the loop point should use filter 0 (or data crafted so the history matches), otherwise the first pass and every repeat through the loop decode slightly differently and the joint clicks — we'll see the loop machinery in Module 10. The whole scheme lands around a 3.5 : 1 squeeze — 32 bytes of PCM become 9 bytes of BRR.

The lab actually runs a BRR codec. It synthesises a short plucked waveform, encodes it to real 9-byte blocks (brute-forcing the best filter and shift per block, exactly as tools like BRRtools do), then decodes and plays it back. Compare the source with the decoded version — the loss is audible but small — and inspect the block headers.

A real note isn't a steady blast — its loudness changes from the moment it starts to the moment it dies away. That loudness-over-time curve is the envelope, and it matters as much as the waveform. The very same tone with a slow, swelling envelope becomes a string pad; with a sharp attack and quick decay it becomes a plucked harp. Envelope is character.

The classic way to describe an envelope has four stages, abbreviated ADSR:

• Attack — how quickly the note fades in to full volume when it starts.
• Decay — how quickly it then falls back to a held level.
• Sustain — the held level while the note is still on. (This is a loudness, not a time.)
• Release — how quickly it fades to silence once the note is let go.

Draw an envelope with the sliders below and hear it applied to a live tone. Watch how the same sawtooth becomes a pluck, a pad, or a stab depending only on the four numbers. In Part II you'll set these exact stages on the S-DSP's hardware envelope registers (Module 11).

A game can't store a separate recording for every note; there isn't the RAM. Instead it stores one sample and plays it back faster for a higher pitch or slower for a lower one. You already know this trick from tape decks: run the tape fast and a voice turns squeaky and chipmunk-like; run it slow and it drops to a growl. Changing a recording's playback speed changes its pitch. Reading a stored waveform at a new speed is called resampling, and it's the beating heart of a sample-based sound chip.

The catch: to play back faster you need sample values between the ones you stored — at fractional positions like “sample 4.63.” You have to make up those in-between values, and how you do it is called interpolation. The crudest method, nearest-neighbour, just grabs whichever stored sample is closest — cheap, but it produces a jagged, aliased buzz. Linear interpolation draws a straight line between the two neighbours — better, but corners in the signal still add grit.

The SNES uses something nicer: a 4-point Gaussian interpolation. For each fractional position it takes the four nearest stored samples and blends them with weights read from a fixed Gaussian-shaped table baked into the chip — a 512-entry table on the real silicon, which the lab below stands in for with a smooth Gaussian approximation (the same shortcut its code comments admit to). The result is smooth, and it gently rolls off the highest frequencies — which, along with the 32 kHz output, is the other half of why the SNES sounds so soft and round. Play the same bright sweep through nearest and Gaussian and the difference is immediate.

The last big idea before the hardware is space. A sound in a room reaches your ear directly, then again a moment later after bouncing off the walls, and again and again, each bounce quieter. Recreate that and a dry, flat sample suddenly sits in a cathedral. It's built from three simple parts.

The first is a delay line: a short memory that holds the sound and plays it back a set number of milliseconds later. One delay gives you a single echo. The second is feedback: route the delay's output back into its own input, scaled down, and the echo repeats — each pass quieter than the last — until it fades. Many overlapping, decaying echoes blur into reverb.

The third part is a filter. Real walls absorb high frequencies more than low ones, so each echo should come back a little duller than the last. A filter does exactly that. The most flexible kind is an FIR filter, which builds each output sample as a weighted sum of the last few input samples; the weights are called taps. Choose the taps and you choose the filter's colour. The SNES bakes exactly this into its echo hardware — an 8-tap FIR — which we'll design in Module 13.

“Weighted sum” sounds fancier than it is. Take a 3-tap FIR with taps 0.5, 0.3, 0.2, and suppose the three most recent input samples were 10, 20, 40 (newest first). The output sample is 0.5 × 10 + 0.3 × 20 + 0.2 × 40 = 5 + 6 + 8 = 19. That's the whole algorithm: multiply each recent sample by its tap, add, slide forward one sample, repeat — 32 000 times a second.

Here's the surprise that defines SNES audio: it isn't a peripheral, it's a whole second computer. Sony's S-SMP package — often just called the APU, the Audio Processing Unit — contains its own CPU, its own memory, its own boot ROM, and the sound mixer, all running on a separate crystal. The main SNES CPU can't reach into it directly at all.

Inside the S-SMP:

• The SPC700 — an 8-bit CPU running at roughly 1.024 MHz, with a small, quirky instruction set of its own.
• 64 KB of private audio RAM (ARAM) — the SPC700's entire world: driver code, the sample directory, all BRR samples, and the echo buffer share this one space.
• Three timers the driver reads to keep musical time — T0 and T1 tick at 8 kHz, T2 at a finer 64 kHz.
• A 64-byte IPL boot ROM that runs at power-on and knows how to receive an upload.
• The S-DSP (next module), which does the actual sample decoding and mixing.

Before the handshake, one thing is worth pinning down: what does “writing to $2140” physically mean? Not every address in a computer is RAM. Some addresses are wired straight to a chip instead — store a byte there and it lands in a register, a byte-sized cell of the hardware itself. This is memory-mapped I/O: the CPU talks to other chips with the same store instructions it uses for memory, just at special addresses. The four mailbox ports are latches — each simply holds the last byte one side wrote until the other side gets around to reading it. And the driver we keep mentioning is not hardware at all: it's a program, the piece of SPC700 code that reads those mailbox bytes and turns them into music. It gets a module of its own (Module 14).

All communication squeezes through those four mailbox ports. At power-on the main CPU can't just poke samples into ARAM; it has to hand them over through a handshake with the IPL boot ROM. The CPU uploads a sound driver plus all the BRR samples, tells the SPC700 where to start, and from then on the APU runs on its own — the game only sends short note events (“play sample 5 at this pitch”) whenever the music demands. Step through that boot handshake below.

The S-DSP is the part that actually makes sound. It runs eight independent voices, and every voice is a little assembly line: it reads a BRR sample from ARAM, resamples it to the wanted pitch (the Gaussian of Module 06), shapes it with an envelope (ADSR or GAIN), and scales it by a left and a right volume. All eight results are summed and pushed out as one 16-bit stereo pair, 32 000 times a second.

The S-DSP is controlled entirely through a 128-byte register file, addressed $00–$7F. It's laid out cleverly: the low nibble of the address picks a field, the high nibble picks the voice, so each voice gets a tidy 16-byte row. Per voice you get:

Those volumes being signed is not a technicality. A negative volume flips the waveform upside down — it inverts its phase (Module 01). Set a voice's left volume positive and its right volume negative and the two speakers push and pull in opposition, a trick games leaned on for wide, pseudo-surround effects.

The rest of the file is global: KON/KOFF key voices on and off (one bit each), FLG holds mute/reset, the noise clock and the echo-write-enable bit ECEN (remember it — it matters in Module 13), MVOLL/R and EVOLL/R are the master and echo volumes, PMON/NON/EON enable pitch-modulation, noise and echo per voice, DIR points at the sample directory (Module 10), ESA/EDL place the echo buffer and EFB ($0D) sets its feedback (Module 13), C0–C7 are the echo FIR taps, and the read-only ENDX ($7C) reports which voices have reached their sample's END block. Toggle the eight voices below and hear them sum.

A voice's VxSRCN register just says “play sample number 5.” How does the DSP turn that number into an address in ARAM? Through a lookup table called the sample directory. The global DIR register holds the high byte of where that table lives, and each entry is four bytes: a start pointer and a loop pointer, both 16-bit. VxSRCN is simply the index into it.

So keying a voice on does this: read VxSRCN, jump to DIR's table, pull out the start and loop addresses for that entry, and begin decoding BRR blocks from the start address. The hardware walks block after block until it hits one with the END flag set. What happens next depends on the LOOP flag in that same header:

• END set, LOOP clear — the voice goes silent. This is a one-shot: a drum hit, a footstep.
• END set, LOOP set — the voice jumps to the loop pointer and keeps decoding. This is how a half-second recording of a violin sustains for as long as the note is held.

That loop pointer usually lands partway into the sample — a short attack plays once, then a small steady-state section repeats forever. It's the single most important trick for fitting real instruments into 64 KB. The lab decodes a real looping BRR sample and lets you play it once or hold the loop.

Module 05 drew envelopes as smooth curves. On the S-DSP an envelope is a handful of bits, and there are two ways to make one. The first is hardware ADSR, packed into two registers per voice:

The rates aren't times in milliseconds — they're indices into a fixed rate table in the chip. A higher attack rate means a faster attack; the fastest is nearly instant, the slowest takes several seconds. A few real rows to calibrate your ear: attack rate 15 is effectively instantaneous and rate 14 takes about 6 ms, while attack rate 0 crawls up over ~4.1 s; the fastest decay (rate 7) falls in ~37 ms; sustain rate 31 fades in ~18 ms while sustain rate 1 takes a glacial ~38 s. Attack rises linearly; decay and the sustain phase fall exponentially, which is why SNES notes taper so naturally. Sustain rate 0 is special: it means “hold the sustain level forever,” used for organ-like tones.

One naming trap if you're mapping this back onto Module 05's ADSR: the sustain rate is not the release. It governs how the envelope keeps sagging from the sustain level while the key is still held. So where did the “R” go? It happens at key-off: when the driver clears the voice via KOF, the chip ignores the sustain rate entirely and applies one fixed release for every voice — a linear ramp down of 1⁄256 of full scale per output sample, so even a full-volume note reaches silence in about 8 ms. On this chip the release isn't programmable at all.

The alternative is GAIN mode (bit 7 of ADSR1 cleared). GAIN gives the driver hands-on control: a direct mode sets the level outright, and four ramp modes slide the level: linear increase, bent-line increase (rises linearly, then eases off for the final quarter), linear decrease, and exponential decrease. Drivers use GAIN for effects ADSR can't express, like a slow fade-in or a custom multi-stage envelope. Either way, the chip exposes the current envelope level in the read-only VxENVX register (0–127) — the ENVX readout the lab tracks live.

Back in Module 06 we heard why interpolation matters. Here's how the S-DSP actually does it. For each output sample the chip needs a value at a fractional position between stored samples. It takes the four nearest stored samples — two behind, two ahead — and multiplies each by a weight read from a fixed 512-entry Gaussian table, indexed by the fractional part of the voice's accumulating pitch counter — the running position that grows by VxPITCH every output sample. (VxPITCH is the step size; the counter's fraction is where the read head currently sits between two stored samples.) Sum the four weighted taps and that's the output.

Drag the position below and watch the four active taps slide along the stored samples, their weights (g0…g3) shifting as the read head moves. Because the Gaussian is broad and smooth, it quietly attenuates the highest frequencies — the mathematical root of the SNES's soft treble.

Two more features ride on the pitch machinery. Pitch modulation (PMON) lets voice N's output continuously bend the pitch of voice N+1 — a cheap way to make growling, vibrato-rich, or metallic sounds without extra samples. (Voice 0 is the exception: it has no lower-numbered neighbour to listen to, so its PMON bit does nothing.) And the noise generator (NON) replaces a voice's sample with pseudo-random noise, clocked at a rate set in the FLG register — the source of wind, surf, cymbals and explosions. Both are per-voice enable bits, so a game can mix tonal voices, a modulated voice and a noise voice in the same eight.

The lush reverb in so much SNES music comes from one dedicated block: the echo unit. It's the delay-feedback-filter machine of Module 07, built into the chip. The delay lives in a circular echo buffer inside ARAM: the ESA register sets its start page and EDL sets its length in 16 ms steps, from 0 to 15 — so the longest possible echo is 15 × 16 = 240 ms. (EDL = 0 is a special case: the buffer doesn't vanish but shrinks to a token 4 bytes.) The price is concrete: at 32 000 samples/s in 16-bit stereo, 240 ms of delay is 240 ms × 32 000 × 4 bytes ≈ 30 KB — nearly half of the 64 KB of ARAM, gone from the space the samples live in. Because the buffer sits in the same 64 KB as the samples, a long echo eats memory the samples could have used — a constant trade-off for SNES composers.

Each pass through the buffer is scaled by the feedback register EFB before being written back — that's what makes the echo repeat and decay. And crucially, on the way out the echo signal runs through an 8-tap FIR filter whose coefficients are the global registers C0–C7. Those eight signed taps let the driver shape the echo's tone: a gentle low-pass for a warm hall, or odd taps for a metallic ring. Design the filter below — set the eight taps, the delay and the feedback — and hear the reverb while the tap stems and the filter's frequency response plot update live.

Follow one sample through to the speaker. Each of the eight voices is scaled by its VxVOL left and right, summed, and multiplied by the master volume MVOLL/R. In parallel, the echo return is scaled by EVOLL/R and added in. The result is clamped to 16 bits and clocked out as the stereo pair, 32 000 times a second. That's the whole output stage — a handful of multiplies and adds, done in hardware.

But the S-DSP has no idea what a “song” is. It only knows registers. Turning sheet music into register writes is the job of a music driver — a program running on the SPC700 that the game uploaded at boot. Every musical tick it reads a sequence of note events and translates them: load VxSRCN and VxPITCH for the instrument and note, set the volumes, choose an envelope, then write a bit into KON to key the voice on. Timers keep the tempo.

Different studios wrote different drivers. Nintendo's own is often called the N-SPC / Kankichi engine (heard across first-party games); Rare, Konami, Square and others shipped their own, each with its own effects and sequence format. They all boil down to the same loop: read events → write DSP registers → wait for the timer. The lab is a tiny four-voice tracker doing exactly that — toggle steps and each active cell fires a KON write on the beat.

An emulator that wants SNES music to sound right can't fake the audio chip — it has to run it. That means two problems at once. First, there are two crystals: the main system runs at 21.477 MHz while the S-SMP runs off its own ~24.576 MHz crystal (divided down to the SPC700's ~1.024 MHz). A quick word on the unit hiding in “MHz”: a CPU cycle is a single tick of a chip's clock, the smallest step in which it can do any work, and megahertz just counts millions of those ticks per second. The two clocks here aren't in any tidy ratio, so the emulator must interleave them carefully; let them drift and the mailbox handshake from Module 08 breaks, and games hang waiting for an APU that answered a beat too early or too late.

Second is the CPU↔APU sync problem. Because games poll the mailbox in tight timing loops, an accurate emulator has to switch between emulating the main CPU and the SPC700 the instant either touches a shared port — sometimes cycle by cycle. Get it slightly wrong and you get missing music, stuck notes, or a boot that never completes.

On top of timing sit the fiddly details of the DSP itself: the exact 512-entry Gaussian table, the precise integer maths of the four BRR filters, and edge cases like what a shift value above 12 does or how the very first block behaves. Getting these bit-exact took years of reverse engineering. Much of it crystallised in blargg's SPC_DSP core — a compact, bit-accurate C++ implementation. snes9x uses that core directly, and bsnes' own DSP is derived from the same core and the research behind it — which is why the two agree down to the sample. The lab steps an SPC700 driver snippet as it writes a voice's volumes and keys it on — the fetch/execute cycle an emulator runs millions of times a second.

Reading the listing: in MOV X, #$4C the # means “this literal number” — copy the value $4C itself into register X. Without #, as in MOV $F2, X, the operand is an address: X's value is stored to location $F2. Everything after ; is a comment for humans.

Here's the elegant payoff of the two-computer design. Because the whole song lives inside the APU — driver, samples and all — you can capture a track by freezing the APU's entire state. That snapshot is the SPC file format, and it's small and rigidly fixed in size: a 256-byte header carrying the SPC700's CPU registers and the ID666 metadata tags (song title, game, composer, length — the same idea as an MP3's ID3 tags), then a full 64 KB dump of ARAM, the 128 bytes of DSP registers, and a few spare bytes: 66 048 bytes, every time. No rendered audio is stored — the ARAM dump does contain the BRR-compressed samples, but the finished waveform the console would output is nowhere in the file. It's the machine itself, paused mid-song.

An SPC player is therefore not really a player of audio; it's a mini SNES with only the sound half. It loads the 64 KB into an emulated ARAM, restores the DSP registers and SPC700 state, and simply lets the frozen driver keep running from exactly where it was captured. The music resumes because the driver picks up its own sequence data and carries on writing KONs — the same loop from Module 14, thawed. This is why a 66,048-byte file can play minutes of music: you're shipping the program and its instruments, not the performance.

One last thing: a quick exam. Six questions, each pointing back at the module that answers it.