A diffusion model is a generative model that learns to produce data by being trained to reverse a gradual noise-corruption process. During training, clean examples are progressively destroyed over hundreds of timesteps by adding Gaussian noise until only noise remains; a neural network learns to predict and remove small amounts of noise at each step. At generation time, the model starts from pure noise and iteratively denoises it, guided by a text prompt or other conditioning signal, until a coherent output emerges.

Think of it like restoring a faded photograph through a series of small, careful touchups. Each pass clarifies a little more detail, and after hundreds of passes the image looks sharp — even though it began as pure static.

Generating an image with a Stable Diffusion pipeline in Diffusers looks like this:

from diffusers import StableDiffusionPipeline
import torch

pipe = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16
).to("cuda")

image = pipe("a photorealistic cat astronaut").images[0]
image.save("output.png")

The pipeline runs roughly 20-50 denoising steps, each calling the underlying U-Net conditioned on the text Embedding.

Diffusion models have largely supplanted GANs for high-quality image and video synthesis because they produce more diverse outputs and are more stable to train. The conditioning mechanism relies on Embedding and Attention to incorporate text or image guidance. The intermediate noisy representations live in a Latent Space when a variational autoencoder is used, as in Stable Diffusion's latent diffusion variant.

You will encounter diffusion models as the backbone of nearly every modern image generation product, including Stable Diffusion, DALL-E 3, Midjourney, and Sora's video generation system.