AI Model Wiki

    Skip to main content
    7 min read
    Lesson 4 of 6

    Fine-Tuning Llama

    From the course Meta Llama for Developers

    Fine-tuning adapts Llama to your specific domain or task, dramatically improving performance compared to prompting alone.

    Why Fine-Tune?

    • Domain expertise — Teach Llama your industry's terminology and knowledge
    • Style consistency — Match your brand's tone and writing style
    • Task optimization — Improve accuracy on specific tasks (classification, extraction, etc.)
    • Smaller models — A fine-tuned 8B can outperform a generic 70B on your specific use case

    LoRA: The Practical Approach

    Full fine-tuning requires enormous compute. LoRA (Low-Rank Adaptation) makes it practical:

    from peft import LoraConfig, get_peft_model
    
    lora_config = LoraConfig(
        r=16,
        lora_alpha=32,
        target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
        lora_dropout=0.05,
        task_type="CAUSAL_LM"
    )
    
    model = get_peft_model(base_model, lora_config)
    # Only ~0.1% of parameters are trained!
    

    Preparing Training Data

    Format your data as instruction-response pairs:

    [
      {
        "instruction": "Summarize this customer complaint",
        "input": "I ordered a blue shirt but received a red one...",
        "output": "Customer received wrong color (red instead of blue). Requesting exchange."
      }
    ]
    

    Aim for 500-5,000 high-quality examples for meaningful improvement.

    Training with Unsloth (Fastest)

    from unsloth import FastLanguageModel
    
    model, tokenizer = FastLanguageModel.from_pretrained(
        model_name="unsloth/llama-3.1-8b-instruct-bnb-4bit",
        max_seq_length=2048,
        load_in_4bit=True
    )
    
    # 2x faster training with 60% less memory
    

    Fine-Tuning Costs

    • Local (8B + LoRA) — Free if you have a 24GB GPU, takes 2-4 hours
    • Cloud (8B + LoRA) — ~$5-20 on Lambda Labs or RunPod
    • Cloud (70B + LoRA) — ~$50-200 depending on dataset size

    Data Quality Over Quantity

    A common fine-tuning mistake is gathering 50,000 low-quality instruction-response pairs when 1,000 high-quality examples would work far better. Before scaling your dataset, fine-tune on 500 examples and evaluate. If the model is moving in the right direction, scale up. If not, the problem is data quality, not quantity. Each example should represent the exact type of input and output you want in production — avoid including edge cases in training data that you do not want the model to generalize from.

    Merging LoRA Adapters for Deployment

    After fine-tuning, merge the LoRA adapter back into the base model for efficient production serving. Serving a base model plus a separate adapter requires more memory and slightly higher latency than a single merged model.

    from peft import PeftModel
    merged = PeftModel.from_pretrained(base_model, adapter_path)
    merged = merged.merge_and_unload()
    merged.save_pretrained("./merged-model")
    

    The merged model can then be quantized and served with vLLM or llama.cpp at full base-model speed.

    Key Takeaways

    • •Fine-tuning dramatically outperforms prompting for domain-specific tasks
    • •LoRA trains only ~0.1% of parameters, making fine-tuning practical
    • •500-5,000 high-quality examples is enough for meaningful improvement
    • •Unsloth provides 2x faster training with 60% less memory
    0 of 6 lessons marked complete

    Progress is kept in this browser only.