10 min read
Lesson 4 of 5
Running DeepSeek Locally with Ollama
From the course DeepSeek AI: Open-Source Reasoning Power
Running DeepSeek on your own machine gives you complete privacy, zero API costs, and offline access. Here's how to set it up.
Why Run Locally?
- Privacy — Data never leaves your machine
- Cost — Zero per-token cost after hardware investment
- Speed — No network latency for responses
- Offline — Works without internet
- Customization — Fine-tune for your specific needs
Hardware Requirements
| Model | VRAM | RAM | Speed |
|---|---|---|---|
| R1-Distill 7B (Q4) | 5 GB | 8 GB | Fast |
| R1-Distill 14B (Q4) | 10 GB | 16 GB | Good |
| R1-Distill 32B (Q4) | 20 GB | 32 GB | Moderate |
| V3 (full, quantized) | 40+ GB | 64 GB | Slow |
| R1 (full) | 300+ GB | Multi-GPU | Research only |
Setup with Ollama (Easiest)
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh
# Download and run DeepSeek R1 (distilled 7B)
ollama run deepseek-r1:7b
# For better quality (needs 10+ GB VRAM)
ollama run deepseek-r1:14b
# For best local quality (needs 20+ GB VRAM)
ollama run deepseek-r1:32b
# DeepSeek V3 for general chat
ollama run deepseek-v3
Using with a Chat UI
Pair Ollama with a local chat interface:
- Open WebUI — Beautiful ChatGPT-like interface
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway \
-v open-webui:/app/backend/data \
--name open-webui --restart always \
ghcr.io/open-webui/open-webui:main
- Jan — Desktop app with built-in model management
- LM Studio — Visual interface with one-click model downloads
API Access for Local Models
Ollama exposes an OpenAI-compatible API locally:
const response = await fetch('http://localhost:11434/v1/chat/completions', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model: 'deepseek-r1:14b',
messages: [{ role: 'user', content: 'Hello!' }],
}),
});
Performance Optimization
- Quantization — Use Q4_K_M for best quality/speed balance
- GPU offloading — Ensure your GPU is being used (
ollama psto check) - Context length — Reduce context window if you don't need full 64K
- Batch size — Adjust for throughput vs latency trade-offs
Practical Local Use Cases
- Code assistant — Private Copilot alternative with Cursor or Continue.dev
- Document analysis — Process sensitive documents without cloud exposure
- Personal journal AI — Completely private AI companion
- Development testing — Test AI features without API costs during development
- Offline research — AI assistance without internet dependency
Key Takeaways
- •Run DeepSeek locally with Ollama for zero cost and complete data privacy
- •R1-Distill 7B runs on most modern laptops; 14B and 32B need dedicated GPUs
- •Pair with Open WebUI or Jan for a ChatGPT-like local experience
- •Ollama exposes an OpenAI-compatible local API for app integration
- •Best local use cases: private coding, sensitive documents, development testing
0 of 5 lessons marked complete
Progress is kept in this browser only.