AI Model Wiki

    Skip to main content
    10 min read
    Lesson 4 of 5

    Running DeepSeek Locally with Ollama

    From the course DeepSeek AI: Open-Source Reasoning Power

    Running DeepSeek on your own machine gives you complete privacy, zero API costs, and offline access. Here's how to set it up.

    Why Run Locally?

    • Privacy — Data never leaves your machine
    • Cost — Zero per-token cost after hardware investment
    • Speed — No network latency for responses
    • Offline — Works without internet
    • Customization — Fine-tune for your specific needs

    Hardware Requirements

    ModelVRAMRAMSpeed
    R1-Distill 7B (Q4)5 GB8 GBFast
    R1-Distill 14B (Q4)10 GB16 GBGood
    R1-Distill 32B (Q4)20 GB32 GBModerate
    V3 (full, quantized)40+ GB64 GBSlow
    R1 (full)300+ GBMulti-GPUResearch only

    Setup with Ollama (Easiest)

    # Install Ollama
    curl -fsSL https://ollama.com/install.sh | sh
    
    # Download and run DeepSeek R1 (distilled 7B)
    ollama run deepseek-r1:7b
    
    # For better quality (needs 10+ GB VRAM)
    ollama run deepseek-r1:14b
    
    # For best local quality (needs 20+ GB VRAM)
    ollama run deepseek-r1:32b
    
    # DeepSeek V3 for general chat
    ollama run deepseek-v3
    

    Using with a Chat UI

    Pair Ollama with a local chat interface:

    • Open WebUI — Beautiful ChatGPT-like interface
    docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway \
      -v open-webui:/app/backend/data \
      --name open-webui --restart always \
      ghcr.io/open-webui/open-webui:main
    
    • Jan — Desktop app with built-in model management
    • LM Studio — Visual interface with one-click model downloads

    API Access for Local Models

    Ollama exposes an OpenAI-compatible API locally:

    const response = await fetch('http://localhost:11434/v1/chat/completions', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        model: 'deepseek-r1:14b',
        messages: [{ role: 'user', content: 'Hello!' }],
      }),
    });
    

    Performance Optimization

    • Quantization — Use Q4_K_M for best quality/speed balance
    • GPU offloading — Ensure your GPU is being used (ollama ps to check)
    • Context length — Reduce context window if you don't need full 64K
    • Batch size — Adjust for throughput vs latency trade-offs

    Practical Local Use Cases

    • Code assistant — Private Copilot alternative with Cursor or Continue.dev
    • Document analysis — Process sensitive documents without cloud exposure
    • Personal journal AI — Completely private AI companion
    • Development testing — Test AI features without API costs during development
    • Offline research — AI assistance without internet dependency

    Key Takeaways

    • •Run DeepSeek locally with Ollama for zero cost and complete data privacy
    • •R1-Distill 7B runs on most modern laptops; 14B and 32B need dedicated GPUs
    • •Pair with Open WebUI or Jan for a ChatGPT-like local experience
    • •Ollama exposes an OpenAI-compatible local API for app integration
    • •Best local use cases: private coding, sensitive documents, development testing
    0 of 5 lessons marked complete

    Progress is kept in this browser only.