AI Model Wiki

    Skip to main content
    40 min
    Lesson 3 of 5

    Metadata Filtering & Namespaces

    From the course Vector Search with Pinecone

    Advanced Query Features

    Metadata Filtering

    Combine vector similarity with metadata filters:

    const results = await index.query({
      vector: queryVector,
      topK: 10,
      filter: {
        category: { $eq: "science" },
        year: { $gte: 2023 },
        tags: { $in: ["biology", "chemistry"] },
      },
      includeMetadata: true,
    });
    

    Filter Operators

    OperatorMeaningExample
    $eqEquals{ status: { $eq: "active" } }
    $neNot equals{ type: { $ne: "draft" } }
    $gt / $gteGreater than{ score: { $gt: 0.8 } }
    $lt / $lteLess than{ price: { $lt: 100 } }
    $inIn array{ tag: { $in: ["a", "b"] } }
    $ninNot in array{ status: { $nin: ["deleted"] } }

    Namespaces

    Partition your index into logical segments:

    // Upsert into a namespace
    await index.namespace("user-123").upsert(vectors);
    
    // Query within a namespace
    const results = await index.namespace("user-123").query({
      vector: queryVector,
      topK: 5,
    });
    

    Namespace Strategies

    • Per-tenant: Each customer gets their own namespace for data isolation
    • Per-collection: Group documents by type (articles, products, FAQs)
    • Per-language: Separate indexes by language for multilingual search
    • Per-version: Support A/B testing different embedding approaches

    Common Pitfall: Querying Across Namespaces

    Pinecone queries are namespace-scoped — a query against namespace("user-123") will not return results from namespace("user-456"). If you need to search across multiple tenants simultaneously (e.g., a global admin search), you must query each namespace separately and merge results client-side. This can get expensive at scale; consider a separate "global" namespace for cross-tenant queries with deduplicated content.

    Best Practice: Track Vector Count per Namespace

    Monitor the number of vectors per namespace to detect runaway ingestion or orphaned data:

    const stats = await index.describeIndexStats();
    const namespaceSizes = stats.namespaces;
    for (const [ns, info] of Object.entries(namespaceSizes)) {
      console.log(`Namespace ${ns}: ${info.vectorCount} vectors`);
      if (info.vectorCount > MAX_VECTORS_PER_TENANT) {
        await alertOps(`Tenant ${ns} exceeds vector limit`);
      }
    }
    

    Named Product Reference: Pinecone Inference API

    Pinecone launched its own Inference API that generates embeddings using multilingual-e5-large. Generating embeddings and upserting in the same API call eliminates the round-trip to a separate embedding provider. This simplifies the ingestion pipeline and reduces the number of external API dependencies in production.

    Key Takeaways

    • •Metadata filters narrow results by exact values, ranges, and sets
    • •Namespaces partition indexes for multi-tenant and organized data
    • •Combine vector similarity with filters for precise retrieval
    • •Use per-tenant namespaces for data isolation in SaaS applications
    0 of 5 lessons marked complete

    Progress is kept in this browser only.