AI Model Wiki

    Skip to main content
    24 min
    Lesson 2 of 7

    Training Infrastructure

    From the course AI Fine-Tuning Pipelines

    Set up scalable infrastructure for fine-tuning models efficiently.

    Fine-Tuning Approaches

    Full Fine-Tuning vs PEFT

    Full Fine-Tuning:

    • Updates all model weights
    • Requires significant GPU memory
    • Risk of catastrophic forgetting
    • Best quality when done right

    Parameter-Efficient Fine-Tuning (PEFT):

    • LoRA: Low-rank adaptation matrices
    • QLoRA: Quantized LoRA (4-bit)
    • Adapters: Small trainable modules
    • 10-100x less memory required

    LoRA Training Setup

    from peft import LoraConfig, get_peft_model
    from transformers import AutoModelForCausalLM
    
    model = AutoModelForCausalLM.from_pretrained(
        "meta-llama/Llama-2-7b-hf",
        load_in_4bit=True,
        device_map="auto"
    )
    
    lora_config = LoraConfig(
        r=16,
        lora_alpha=32,
        target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
        lora_dropout=0.05,
        bias="none",
        task_type="CAUSAL_LM"
    )
    
    model = get_peft_model(model, lora_config)
    # Trainable params: 4,194,304 || all params: 6,738,415,616 || 0.06%
    

    Cloud Training Options

    ProviderProsCons
    OpenAI Fine-tuningSimplest setupLimited customization
    AWS SageMakerFull controlComplex setup
    Together AICost-effectiveFewer models
    ModalServerlessPay-per-use

    Key Takeaways

    • •LoRA trains only 0.06% of parameters while achieving similar quality to full fine-tuning
    • •QLoRA enables fine-tuning 7B models on a single consumer GPU (24GB)
    • •DeepSpeed Zero Stage 2 enables distributed training with optimizer offloading
    • •Cloud providers offer trade-offs between simplicity (OpenAI) and control (SageMaker)
    0 of 7 lessons marked complete

    Progress is kept in this browser only.