Semantic chunking is a preprocessing strategy for retrieval systems that splits a long document into segments based on meaning rather than on fixed character counts or paragraph breaks. Rather than cutting every 512 tokens regardless of content, semantic chunking detects topic shifts — often by embedding consecutive sentences and measuring cosine similarity — and places boundaries where meaning changes substantially. The output is a set of chunks that each represent a coherent idea, which improves retrieval precision because a retrieved chunk is more likely to contain a complete, self-sufficient answer to a query.

Think of it like cutting a novel into scenes rather than into equal-length pages. A scene chunk gives the retriever a meaningful, self-contained unit; a fixed-length page may start mid-dialogue and end mid-action, stranding the context the reader actually needs.

from langchain.text_splitter import SemanticChunker
from langchain_openai import OpenAIEmbeddings

splitter = SemanticChunker(
    OpenAIEmbeddings(),
    breakpoint_threshold_type="percentile",
    breakpoint_threshold_amount=90,
)
chunks = splitter.split_text(long_document)

Semantic chunking sits upstream of Embedding and Vector Database indexing in a RAG pipeline; the quality of chunks directly affects what the LLM sees during Inference. It is conceptually linked to Tokenization in that both involve decisions about how text is segmented before further processing.

You will encounter semantic chunking when building or optimising RAG applications — it is one of the highest-leverage tuning knobs for improving answer quality without changing the underlying model.