AI Model Wiki

    Skip to main content
    25 min
    Lesson 4 of 7

    Hands-On Tutorial: End-to-End Fine-Tuning

    From the course AI Fine-Tuning Pipelines

    Build a complete fine-tuning pipeline from scratch with runnable code.

    Project: Customer Support Model

    We'll fine-tune Llama-2-7b to answer customer support questions for an e-commerce store.

    Step 1: Install Dependencies

    pip install transformers peft datasets trl accelerate bitsandbytes
    pip install wandb  # for experiment tracking
    

    Step 2: Prepare Your Dataset

    from datasets import Dataset
    import json
    
    # Load your JSONL training data
    def load_support_dataset(filepath):
        examples = []
        with open(filepath) as f:
            for line in f:
                item = json.loads(line)
                examples.append({
                    "messages": [
                        {"role": "system", "content": "You are a helpful customer support agent for ShopEasy."},
                        {"role": "user", "content": item["question"]},
                        {"role": "assistant", "content": item["answer"]}
                    ]
                })
        return Dataset.from_list(examples)
    
    dataset = load_support_dataset("support_data.jsonl")
    dataset = dataset.train_test_split(test_size=0.1)
    

    Step 3: Load Base Model with QLoRA

    from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
    from peft import LoraConfig, get_peft_model
    import torch
    
    bnb_config = BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_quant_type="nf4",
        bnb_4bit_compute_dtype=torch.bfloat16
    )
    
    model = AutoModelForCausalLM.from_pretrained(
        "meta-llama/Llama-2-7b-hf",
        quantization_config=bnb_config,
        device_map="auto"
    )
    tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-hf")
    tokenizer.pad_token = tokenizer.eos_token
    
    lora_config = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05, bias="none", task_type="CAUSAL_LM")
    model = get_peft_model(model, lora_config)
    

    Step 4: Train with TRL's SFTTrainer

    from trl import SFTTrainer
    from transformers import TrainingArguments
    
    training_args = TrainingArguments(
        output_dir="./results",
        num_train_epochs=3,
        per_device_train_batch_size=4,
        gradient_accumulation_steps=4,
        learning_rate=2e-4,
        fp16=True,
        save_steps=100,
        logging_steps=25,
        report_to="wandb"
    )
    
    trainer = SFTTrainer(
        model=model,
        args=training_args,
        train_dataset=dataset["train"],
        eval_dataset=dataset["test"],
        tokenizer=tokenizer,
        max_seq_length=2048
    )
    
    trainer.train()
    trainer.save_model("./fine-tuned-support-model")
    

    Step 5: Run Inference

    from peft import PeftModel
    
    base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf", device_map="auto")
    model = PeftModel.from_pretrained(base, "./fine-tuned-support-model")
    
    def answer_question(question: str) -> str:
        prompt = f"[INST] <<SYS>>You are a helpful customer support agent.<</SYS>>
    {question} [/INST]"
        inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
        outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
        return tokenizer.decode(outputs[0], skip_special_tokens=True)
    
    print(answer_question("Where is my order #12345?"))
    

    Key Takeaways

    • •QLoRA with 4-bit quantization lets you fine-tune a 7B model on a single 24GB GPU
    • •TRL's SFTTrainer handles chat-format datasets natively — no custom data collator needed
    • •Gradient accumulation (steps=4) simulates larger batch sizes on limited hardware
    • •Always split 10% of data for evaluation and log metrics to Weights & Biases
    0 of 7 lessons marked complete

    Progress is kept in this browser only.