Containerizing AI Models
From the course AI Deployment with Docker
Why Docker for AI?
Docker provides consistent, reproducible environments for AI model serving — eliminating "works on my machine" issues with complex ML dependencies.
Basic AI Dockerfile
FROM python:3.11-slim
WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential && rm -rf /var/lib/apt/lists/*
# Install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy application code
COPY . .
# Download model at build time (cached in layer)
RUN python -c "from transformers import AutoModel; AutoModel.from_pretrained('bert-base-uncased')"
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]
Multi-Stage Builds
Reduce image size by separating build and runtime:
# Build stage
FROM python:3.11 AS builder
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt
# Runtime stage
FROM python:3.11-slim
COPY --from=builder /root/.local /root/.local
COPY . /app
WORKDIR /app
ENV PATH=/root/.local/bin:$PATH
CMD ["uvicorn", "app:app", "--host", "0.0.0.0"]
Image Size Optimization
- Use slim or alpine base images
- Install only production dependencies
- Remove build tools after compilation
- Use .dockerignore to exclude unnecessary files
- Cache model weights in a separate layer
Common Pitfall: Model Downloads at Container Startup
A frequent mistake is downloading model weights at runtime inside the container entrypoint script rather than baking them into the image. This causes two problems: startup latency of 30–120 seconds, and failures when the container has no internet access (common in air-gapped production environments). Download models during the Docker build step using a RUN python -c "..."; layer so the weights are cached in the image.
Best Practice: Pin Your Dependencies
Always pin both the Python package versions and the base image tag. Using FROM python:3.11 or pip install torch without version pins means builds become non-reproducible as upstream packages change. A model that works with PyTorch 2.3 may break silently with PyTorch 2.5. Use a lock file (pip freeze > requirements.lock) or a tool like pip-tools to generate deterministic dependency sets.
FROM python:3.11.9-slim
# Pin exact versions
COPY requirements.lock .
RUN pip install --no-cache-dir -r requirements.lock
Named Product Reference: NVIDIA CUDA Base Images
NVIDIA publishes official CUDA base images at nvcr.io/nvidia/cuda and on Docker Hub under nvidia/cuda. The image variants follow a naming convention: 12.2.0-cudnn8-runtime-ubuntu22.04. Choose runtime (not devel) for production deployments — the devel image includes compilers and headers that add ~2 GB to the image with no runtime benefit. The cudnn8 variant includes cuDNN, which is required for most deep learning frameworks.
Key Takeaways
- •Docker ensures consistent AI model serving environments
- •Multi-stage builds reduce image size by 50-80%
- •Cache model downloads in Docker layers to speed up rebuilds
- •Use .dockerignore and slim images for minimal container size
Progress is kept in this browser only.