Reranking for Better Results

What Is Reranking?

A reranker takes the initial search results and re-scores them using a more powerful model, improving relevance at the cost of added latency.

Two-Stage Retrieval

Stage 1: Fast retrieval (vector search) → Top 50-100 candidates
Stage 2: Reranking (cross-encoder) → Top 5-10 results

Using a Reranker

from cohere import Client

co = Client(api_key="...")

results = co.rerank(
    model="rerank-v3.5",
    query="How do I deploy to production?",
    documents=[doc.text for doc in initial_results],
    top_n=5,
)

for result in results.results:
    print(f"Score: {result.relevance_score:.3f} — {result.document.text[:100]}")

Reranker Options

Query Expansion

Improve recall by expanding the original query:

• Synonym expansion: Add related terms
• LLM rewriting: Ask an LLM to rephrase the query in multiple ways
• HyDE: Generate a hypothetical answer, embed that instead of the query

Common Pitfall: Using the Same Model for Queries and Documents

Some embedding models are asymmetric — they produce better results when queries and documents are embedded differently. BGE models from BAAI expect query text to be prefixed with "Represent this sentence for searching relevant passages:" while document text is embedded without a prefix. Using the wrong prefix (or no prefix for queries) significantly degrades retrieval quality.

Best Practice: Log All Reranking Score Changes

When you add a reranker to an existing pipeline, the score changes can be dramatic. Log both the pre-rerank and post-rerank order for every query during your initial rollout to verify the reranker is improving (not degrading) results:

pre_rerank = [r["id"] for r in initial_results[:5]]
reranked = co.rerank(model="rerank-v3.5", query=query, documents=docs, top_n=5)
post_rerank = [r.index for r in reranked.results]

if pre_rerank != post_rerank:
    logger.info(f"Reranker changed order for query: '{query}'")
    logger.info(f"Before: {pre_rerank} | After: {post_rerank}")

Named Product Reference: Cohere Rerank v3.5

Cohere's rerank-v3.5 model is widely regarded as the best commercial reranker available. It is a cross-encoder model that processes query-document pairs jointly (unlike bi-encoder embedding models that process them independently), enabling much more precise relevance scoring. At ~$0.001 per 1000 document-query pairs, it is cost-effective for most production workloads.