A foundation model is a large neural network trained on massive, broadly diverse data — text, images, code, or a combination — in a self-supervised fashion, producing a single general-purpose system that can then be adapted to an enormous range of downstream tasks. The defining characteristic is the scale of pretraining: foundation models typically contain billions of parameters and are exposed to data volumes in the trillions of tokens, giving them emergent capabilities that smaller, narrowly trained models lack. GPT-4, Claude, Gemini, and Llama 3 are all examples, as are vision encoders like CLIP and multimodal systems like GPT-4o.

Think of it like a medical school education. After years of broad training across every speciality, a newly graduated doctor can then do a short residency to become a cardiologist, a surgeon, or a psychiatrist — rather than starting a full education from scratch for each role. The residency is fine-tuning; medical school is pretraining on broad data.

In practice, a developer rarely trains a foundation model from scratch. Instead they take a base model and either prompt it directly or fine-tune it on domain-specific data:

# Fine-tune a foundation model on customer support data
from transformers import AutoModelForCausalLM, Trainer
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3-8B")
trainer = Trainer(model=model, train_dataset=support_dataset, ...)
trainer.train()

Foundation models connect to Pretraining (the initial broad-data training phase), Fine-tuning (task-specific adaptation), LoRA (parameter-efficient fine-tuning), RLHF (alignment after pretraining), and Mixture of Experts (an architectural approach used inside many frontier foundation models to scale efficiently).

You will encounter the term most often in research papers discussing scaling laws, in cloud AI product documentation listing which base model underlies a service, and in enterprise discussions about build-vs-buy decisions for AI systems.