Fine-Tuning Llama
From the course Meta Llama for Developers
Fine-tuning adapts Llama to your specific domain or task, dramatically improving performance compared to prompting alone.
Why Fine-Tune?
- Domain expertise — Teach Llama your industry's terminology and knowledge
- Style consistency — Match your brand's tone and writing style
- Task optimization — Improve accuracy on specific tasks (classification, extraction, etc.)
- Smaller models — A fine-tuned 8B can outperform a generic 70B on your specific use case
LoRA: The Practical Approach
Full fine-tuning requires enormous compute. LoRA (Low-Rank Adaptation) makes it practical:
from peft import LoraConfig, get_peft_model
lora_config = LoraConfig(
r=16,
lora_alpha=32,
target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
lora_dropout=0.05,
task_type="CAUSAL_LM"
)
model = get_peft_model(base_model, lora_config)
# Only ~0.1% of parameters are trained!
Preparing Training Data
Format your data as instruction-response pairs:
[
{
"instruction": "Summarize this customer complaint",
"input": "I ordered a blue shirt but received a red one...",
"output": "Customer received wrong color (red instead of blue). Requesting exchange."
}
]
Aim for 500-5,000 high-quality examples for meaningful improvement.
Training with Unsloth (Fastest)
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="unsloth/llama-3.1-8b-instruct-bnb-4bit",
max_seq_length=2048,
load_in_4bit=True
)
# 2x faster training with 60% less memory
Fine-Tuning Costs
- Local (8B + LoRA) — Free if you have a 24GB GPU, takes 2-4 hours
- Cloud (8B + LoRA) — ~$5-20 on Lambda Labs or RunPod
- Cloud (70B + LoRA) — ~$50-200 depending on dataset size
Data Quality Over Quantity
A common fine-tuning mistake is gathering 50,000 low-quality instruction-response pairs when 1,000 high-quality examples would work far better. Before scaling your dataset, fine-tune on 500 examples and evaluate. If the model is moving in the right direction, scale up. If not, the problem is data quality, not quantity. Each example should represent the exact type of input and output you want in production — avoid including edge cases in training data that you do not want the model to generalize from.
Merging LoRA Adapters for Deployment
After fine-tuning, merge the LoRA adapter back into the base model for efficient production serving. Serving a base model plus a separate adapter requires more memory and slightly higher latency than a single merged model.
from peft import PeftModel
merged = PeftModel.from_pretrained(base_model, adapter_path)
merged = merged.merge_and_unload()
merged.save_pretrained("./merged-model")
The merged model can then be quantized and served with vLLM or llama.cpp at full base-model speed.
Key Takeaways
- •Fine-tuning dramatically outperforms prompting for domain-specific tasks
- •LoRA trains only ~0.1% of parameters, making fine-tuning practical
- •500-5,000 high-quality examples is enough for meaningful improvement
- •Unsloth provides 2x faster training with 60% less memory
Progress is kept in this browser only.