AI Model Wiki

    Skip to main content
    45 min
    Lesson 1 of 5

    Containerizing AI Models

    From the course AI Deployment with Docker

    Why Docker for AI?

    Docker provides consistent, reproducible environments for AI model serving — eliminating "works on my machine" issues with complex ML dependencies.

    Basic AI Dockerfile

    FROM python:3.11-slim
    
    WORKDIR /app
    
    # Install system dependencies
    RUN apt-get update && apt-get install -y --no-install-recommends \
        build-essential && rm -rf /var/lib/apt/lists/*
    
    # Install Python dependencies
    COPY requirements.txt .
    RUN pip install --no-cache-dir -r requirements.txt
    
    # Copy application code
    COPY . .
    
    # Download model at build time (cached in layer)
    RUN python -c "from transformers import AutoModel; AutoModel.from_pretrained('bert-base-uncased')"
    
    EXPOSE 8000
    CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]
    

    Multi-Stage Builds

    Reduce image size by separating build and runtime:

    # Build stage
    FROM python:3.11 AS builder
    COPY requirements.txt .
    RUN pip install --user --no-cache-dir -r requirements.txt
    
    # Runtime stage
    FROM python:3.11-slim
    COPY --from=builder /root/.local /root/.local
    COPY . /app
    WORKDIR /app
    ENV PATH=/root/.local/bin:$PATH
    CMD ["uvicorn", "app:app", "--host", "0.0.0.0"]
    

    Image Size Optimization

    • Use slim or alpine base images
    • Install only production dependencies
    • Remove build tools after compilation
    • Use .dockerignore to exclude unnecessary files
    • Cache model weights in a separate layer

    Common Pitfall: Model Downloads at Container Startup

    A frequent mistake is downloading model weights at runtime inside the container entrypoint script rather than baking them into the image. This causes two problems: startup latency of 30–120 seconds, and failures when the container has no internet access (common in air-gapped production environments). Download models during the Docker build step using a RUN python -c "..."; layer so the weights are cached in the image.

    Best Practice: Pin Your Dependencies

    Always pin both the Python package versions and the base image tag. Using FROM python:3.11 or pip install torch without version pins means builds become non-reproducible as upstream packages change. A model that works with PyTorch 2.3 may break silently with PyTorch 2.5. Use a lock file (pip freeze > requirements.lock) or a tool like pip-tools to generate deterministic dependency sets.

    FROM python:3.11.9-slim
    
    # Pin exact versions
    COPY requirements.lock .
    RUN pip install --no-cache-dir -r requirements.lock
    

    Named Product Reference: NVIDIA CUDA Base Images

    NVIDIA publishes official CUDA base images at nvcr.io/nvidia/cuda and on Docker Hub under nvidia/cuda. The image variants follow a naming convention: 12.2.0-cudnn8-runtime-ubuntu22.04. Choose runtime (not devel) for production deployments — the devel image includes compilers and headers that add ~2 GB to the image with no runtime benefit. The cudnn8 variant includes cuDNN, which is required for most deep learning frameworks.

    Key Takeaways

    • •Docker ensures consistent AI model serving environments
    • •Multi-stage builds reduce image size by 50-80%
    • •Cache model downloads in Docker layers to speed up rebuilds
    • •Use .dockerignore and slim images for minimal container size
    0 of 5 lessons marked complete

    Progress is kept in this browser only.