QLoRA (Quantized Low-Rank Adaptation) is a memory-efficient Fine-tuning technique that combines Quantization of the frozen base model with LoRA adapter training in higher numerical precision, enabling practitioners to fine-tune models with tens of billions of parameters on a single consumer GPU. The base model is loaded in 4-bit NormalFloat (NF4) format, slashing its VRAM footprint by roughly 75 % compared to 32-bit; the lightweight LoRA adapters are added on top and trained in BFloat16, where the precision matters for gradient quality. Introduced by Dettmers et al. (2023), QLoRA made fine-tuning a 65-billion-parameter model feasible on a single 48 GB GPU, democratising large-model customisation.

Think of it like storing a giant reference encyclopedia in compressed microfilm to save shelf space, while keeping only the few pages you are actively annotating in full-size legible print - you get the knowledge of the full book without needing a warehouse to hold it.

Fine-tuning a 7B model with QLoRA using Hugging Face PEFT and BitsAndBytes:

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
from peft import get_peft_model, LoraConfig
import torch

bnb_config = BitsAndBytesConfig(load_in_4bit=True,
                                 bnb_4bit_quant_type="nf4",
                                 bnb_4bit_compute_dtype=torch.bfloat16)
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3-8b", quantization_config=bnb_config, device_map="auto"
)
lora_config = LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj","v_proj"])
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()  # ~0.1 % trainable

QLoRA sits at the intersection of Quantization and LoRA and inherits the trade-offs of both: some accuracy is sacrificed by 4-bit storage, and adapter expressivity is limited by rank. It competes with full fine-tuning and Knowledge Distillation as a compression strategy, and connects to Hyperparameter choices like LoRA rank and quantisation bit-width.

You will encounter QLoRA whenever a team wants to adapt a large Foundation Model to a domain-specific task without access to a multi-GPU cluster - it is now the default starting point for open-source fine-tuning workflows and is supported natively in most major training frameworks.