An audio language model is a machine learning model that processes audio as a sequence of discrete or continuous tokens, applying the same probabilistic modelling principles used in text language models to the domain of sound. The model learns statistical patterns in audio data — pitch, rhythm, timbre, phoneme sequences, or ambient sound events — and can then generate, transcribe, classify, or transform audio conditioned on a prompt or context. Audio language models are the backbone of modern speech-to-text systems, text-to-speech synthesisers, and AI music generators.

Think of it like a language model that has learned to read and write in sound instead of words. Just as a text model predicts the next likely word given previous words, an audio model predicts the next audio frame or token given everything it has heard so far.

A common architecture encodes raw waveforms into a compact latent representation using a codec (such as EnCodec), converts those codes into discrete tokens, and then trains a Transformer on those token sequences. At inference time, the process reverses:

Text prompt → token embedding
→ Transformer generates audio token sequence
→ Codec decoder reconstructs waveform
→ Output: synthesised speech or music

Audio language models connect to the broader Multimodal family of systems, since many recent models (such as GPT-4o or Gemini) can accept both text and audio inputs within a single Context Window. They share training dynamics with text-based LLMs — Pretraining on large corpora, fine-tuning for specific tasks — and benefit from the same Tokenization insight that converting a continuous signal into discrete units makes generative modelling tractable.

You will encounter audio language models when working with speech APIs, voice assistants, AI music generation tools, and any multimodal pipeline where audio is one of the input or output modalities.