How audio compression works

Audio2 min read

What a psychoacoustic model actually removes, why speech and music behave so differently, and what the artefacts sound like when it goes too far.

In short

Lossy audio codecs model what human hearing resolves and store only that: masked sounds and inaudible frequencies are discarded. How much you can remove before it is audible depends heavily on the material.

Two words get used for two different operations here, and confusing them causes real problems. Dynamic range compression makes quiet parts louder and loud parts quieter — a mixing tool. File compression makes the file smaller. This is about the second.

The idea: model the ear, not the sound

Uncompressed audio stores every measurement. Lossy codecs ask a different question: which of these measurements would a person actually notice if they were wrong? The answer comes from psychoacoustics, and it yields three big savings.

  • Frequency masking — a loud tone hides quieter tones at nearby frequencies, so those do not need storing accurately.
  • Temporal masking — for a few milliseconds after a loud sound, the ear cannot hear quiet ones. That window is cheap.
  • Hearing limits — most adults cannot hear much above 16–18 kHz, so the top of the range can go entirely.

What is left gets its precision allocated by importance: lots of bits where the ear is sensitive, few where it is not.

Why material matters so much

A single voice occupies a narrow band and has a lot of predictable structure, so it survives aggressive compression well — 64 kbps speech is perfectly clear. A dense mix with cymbals, reverb tails and wide stereo has far more that the ear does resolve, so the same bitrate falls apart.

What Dexta uses

Encoders, from src/lib/audio.ts
OutputEncoderBitrate setting
MP3libmp3lameApplies — 64k to 256k
AAC (.m4a)aacApplies
OGGlibvorbisApplies
WAVpcm_s16leIgnored — uncompressed
FLACflacIgnored — lossless

Mistakes worth avoiding

  • Compressing an already-compressed file. The second encoder cannot see what the first removed and discards a fresh layer on top.
  • Judging quality on laptop speakers. Compression artefacts are most audible on headphones.
  • Using a music bitrate for speech, or a speech bitrate for music. They are different problems.