24 min
Lesson 2 of 7
Training Infrastructure
From the course AI Fine-Tuning Pipelines
Set up scalable infrastructure for fine-tuning models efficiently.
Fine-Tuning Approaches
Full Fine-Tuning vs PEFT
Full Fine-Tuning:
- Updates all model weights
- Requires significant GPU memory
- Risk of catastrophic forgetting
- Best quality when done right
Parameter-Efficient Fine-Tuning (PEFT):
- LoRA: Low-rank adaptation matrices
- QLoRA: Quantized LoRA (4-bit)
- Adapters: Small trainable modules
- 10-100x less memory required
LoRA Training Setup
from peft import LoraConfig, get_peft_model
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
"meta-llama/Llama-2-7b-hf",
load_in_4bit=True,
device_map="auto"
)
lora_config = LoraConfig(
r=16,
lora_alpha=32,
target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM"
)
model = get_peft_model(model, lora_config)
# Trainable params: 4,194,304 || all params: 6,738,415,616 || 0.06%
Cloud Training Options
| Provider | Pros | Cons |
|---|---|---|
| OpenAI Fine-tuning | Simplest setup | Limited customization |
| AWS SageMaker | Full control | Complex setup |
| Together AI | Cost-effective | Fewer models |
| Modal | Serverless | Pay-per-use |
Key Takeaways
- •LoRA trains only 0.06% of parameters while achieving similar quality to full fine-tuning
- •QLoRA enables fine-tuning 7B models on a single consumer GPU (24GB)
- •DeepSpeed Zero Stage 2 enables distributed training with optimizer offloading
- •Cloud providers offer trade-offs between simplicity (OpenAI) and control (SageMaker)
0 of 7 lessons marked complete
Progress is kept in this browser only.