Vector Search & Pinecone Setup
From the course Vector Search with Pinecone
Introduction to Vector Search
Vector search finds similar items by comparing high-dimensional embeddings rather than matching keywords. It powers semantic search, recommendations, and RAG systems.
How Vector Search Works
- Embed — Convert text/images into numerical vectors using an embedding model
- Index — Store vectors in a specialized database optimized for similarity search
- Query — Convert the search query into a vector and find nearest neighbors
- Return — Retrieve the most similar items with similarity scores
Pinecone Setup
import { Pinecone } from "@pinecone-database/pinecone";
const pc = new Pinecone({ apiKey: process.env.PINECONE_API_KEY });
// Create an index
await pc.createIndex({
name: "my-index",
dimension: 1536, // Must match embedding model dimension
metric: "cosine",
spec: { serverless: { cloud: "aws", region: "us-east-1" } },
});
const index = pc.index("my-index");
Choosing Dimensions & Metrics
| Embedding Model | Dimensions | Best Metric |
|---|---|---|
| OpenAI text-embedding-3-small | 1536 | cosine |
| OpenAI text-embedding-3-large | 3072 | cosine |
| Cohere embed-v3 | 1024 | cosine |
| Sentence Transformers | 384-768 | cosine |
Index Types
- Serverless: Auto-scales, pay per usage — best for most use cases
- Pod-based: Dedicated resources, predictable performance — for high-throughput
Common Pitfall: Index Dimension Mismatch
The most common Pinecone error is attempting to upsert vectors with a different dimension than the index was created with. If you create an index with dimension: 1536 (for text-embedding-3-small) and later switch to text-embedding-3-large (dimension 3072), every upsert will fail. Create a new index when switching embedding models — there is no resize operation.
// Always verify dimensions before upserting
const indexStats = await index.describeIndexStats();
const expectedDim = indexStats.dimension;
if (vectors[0].values.length !== expectedDim) {
throw new Error(`Dimension mismatch: got ${vectors[0].values.length}, index expects ${expectedDim}`);
}
Best Practice: Use Cosine Metric and Normalize Embeddings
For text embeddings, always use cosine similarity as the distance metric. OpenAI's embedding models return already-normalized vectors, but if you use a custom embedding model, normalize explicitly before upserting:
function normalize(v: number[]): number[] {
const norm = Math.sqrt(v.reduce((sum, x) => sum + x * x, 0));
return v.map(x => x / norm);
}
With normalized vectors, cosine similarity equals dot product, which Pinecone computes faster than full cosine calculation.
Named Product Reference: Pinecone Assistant
Pinecone Assistant is a managed RAG product built on Pinecone that handles chunking, embedding, indexing, and retrieval for uploaded documents. For teams that want a working document Q&A system without building the ingestion pipeline, Pinecone Assistant is a production-ready starting point. The same Pinecone API keys work for both the vector database and the Assistant.
Key Takeaways
- •Vector search finds semantically similar items using embeddings
- •Index dimensions must match your embedding model's output size
- •Cosine similarity is the most common metric for text embeddings
- •Serverless indexes auto-scale and are best for most use cases
Progress is kept in this browser only.