Why prompts need a home

A prompt that lives as a string literal inside a function behaves like any other piece of production logic, yet it rarely gets the same care. Nobody can say which wording was live last Tuesday, a quick edit ships without review, and a bad change cannot be undone without digging through commit history. Prompt management is the practice of giving those strings an owner, a history and a controlled path into production.

Prompt management is a systematic approach to storing, versioning, and then retrieving prompts for an LLM application.

Instead of hardcoding prompts in application code, you manage them centrally in Langfuse.

Moving the text out of the code is only the first step. The store that holds it also needs rules about who may change what, how each change is recorded, and how a known-good version comes back when a new one misbehaves. Those rules are what separate a registry from a folder of text files.

Decoupling prompt changes from releases

Product managers and domain experts iterate on prompts, whereas engineers manage deployments.

Notice who appears in that sentence: the people closest to the product wording are not the people who control releases.

With prompts in code, a simple text change requires engineering involvement, then code review, and then a full deployment cycle.

Those two sentences describe a bottleneck rather than a technical limit. When the people who know the wording best must wait for a release train, they stop experimenting, and the prompt drifts away from what users actually need.

When prompts live in Langfuse, non-technical team members can update them directly in the UI, and the application automatically fetches the latest version.

Treat the next two sentences as vendor statements to test, not as measurements to rely on.

Prompt updates deploy instantly, without needing to involve engineering or to trigger a deployment.

Prompts are cached client-side by the SDK, which makes retrieving them as fast as reading from memory.

Whatever tool you adopt, check such statements in your own environment. Time the latency of a fetch on a cold start, on a warm cache, and while the registry is unreachable. A fetch that can fail needs a fallback, for example the last version the application loaded successfully, so a registry outage never becomes a product outage.

Versions, labels and aliases

A version is a frozen snapshot. Once you accept that, edits stop being edits: every change creates a new version, and the history becomes an audit trail you can trust.

Prompt versions in MLflow are immutable, providing strong guarantees for reproducibility.

Frozen versions are awkward to deploy if every application must hard-code a version number. The usual remedy is a movable pointer, a label or alias that names a role such as staging or production and points at exactly one version at a time.

Aliasing lets you isolate prompt versions from main application code.

Aliasing also lets you perform tasks such as A/B testing and roll-backs.

Langfuse's overview lists two further practices, and the first one concerns tracing.

You can link prompts to traces to analyze performance by prompt version.

That link is what lets a team answer the question that matters after a release: did the new wording actually do better, or did it only feel better?

The second Langfuse practice concerns environments.

Use version control and labels to manage deployments across environments.

Promotion then becomes a pointer move. A candidate is registered as a new version, evaluated, labelled for staging, and only later labelled for production. A rollback is the same move in reverse, so it should take seconds and need no code change at all.

A few fields turn a bare history into something a team can reason about: the author, a short reason for the change, the model and settings the prompt was written for, and the evaluation result that justified promotion. The sketch below keeps the first two and leaves room for the rest.

from dataclasses import dataclass
from datetime import datetime, timezone


@dataclass(frozen=True)
class PromptVersion:
    name: str
    version: int
    template: str
    author: str
    note: str
    created_at: str


class PromptRegistry:
    def __init__(self):
        self._versions = {}  # name -> list of PromptVersion
        self._labels = {}    # (name, label) -> version number

    def register(self, name, template, author, note=""):
        history = self._versions.setdefault(name, [])
        created = datetime.now(timezone.utc).isoformat()
        item = PromptVersion(name, len(history) + 1, template, author, note, created)
        history.append(item)
        return item

    def label(self, name, label, version):
        if not 1 <= version <= len(self._versions.get(name, [])):
            raise ValueError(f"{name} has no version {version}")
        self._labels[(name, label)] = version

    def get(self, name, label="production"):
        version = self._labels[(name, label)]
        return self._versions[name][version - 1]


registry = PromptRegistry()
registry.register("summarizer", "Summarize the text in {n} sentences.", "ana", "first draft")
registry.register("summarizer", "Summarize the text for a busy reader in {n} sentences.", "ben", "audience")
registry.label("summarizer", "production", 1)
registry.label("summarizer", "staging", 2)
print(registry.get("summarizer").version)  # 1: production still points at the first version
registry.label("summarizer", "production", 2)  # promote; pointing back at 1 is the rollback
print(registry.get("summarizer").note)

Three properties of this sketch matter more than its length. Versions are frozen dataclasses, so nothing edits them after creation. Labels sit in a separate table, so moving production never rewrites history. And asking for a version that does not exist raises an error instead of quietly returning a default.

Anatomy of a registered prompt

Whatever the product, a registered prompt usually carries a handful of attributes. In the registry documented by MLflow the pieces look like this.

Version is a sequential number representing the revision of the prompt.

Text prompts are strings containing text with variables in {{variable}} format.

Chat prompts use a list of dictionaries that represent chat messages, with role and content keys.

The split matters because the two shapes behave differently. A text prompt is one block of instruction, while a chat prompt keeps each message and its role separate, so the roles stay explicit in the stored prompt.

Templates and variables

Prompt template text can contain variables in {{variable}} format.

Think of each variable as a named gap in the instruction, with the surrounding words fixed and reviewed once.

These variables can then be filled with dynamic content when using the prompt in an LLM application or AI agent.

Variables are the reason a prompt can be shared across requests, but they are also the main place where things go wrong. A missing variable should stop the request rather than send a half-empty instruction to the model, and user-supplied text belongs inside a clearly marked variable rather than spliced into the instruction itself.

MLflow supports Jinja2 templates with conditionals, loops, and filters for advanced templating needs.

Templates with only {{ variable }} syntax are treated as text prompts, which use simple string substitution.

The distinction is useful because it tells you, from the template alone, how much machinery will run when it is rendered.

Templates containing {% %} control flow syntax are treated as Jinja2 prompts.

Rendering text that people can edit is a security surface, because a template is effectively a small program.

The next statement is specific to the format() method of MLflow prompts; it is not a statement about Jinja2 in general.

In format(), Jinja2 rendering uses SandboxedEnvironment by default for security.

The snippet below shows why strict rendering is worth the extra line of setup, and it uses Jinja2's sandboxed environment because the templates are text that other people edit. It builds a template with a conditional block and a loop, and it refuses to render when a variable is missing.

from jinja2 import StrictUndefined
from jinja2.sandbox import SandboxedEnvironment

env = SandboxedEnvironment(undefined=StrictUndefined)
template = env.from_string(
    "You are a {{ role }} assistant.\n"
    "{% if examples %}Examples:\n"
    "{% for ex in examples %}- {{ ex.input }} -> {{ ex.output }}\n{% endfor %}"
    "{% endif %}Answer the question: {{ question }}"
)

print(template.render(role="support", question="How do I reset my password?", examples=[]))

try:
    template.render(role="support")
except Exception as err:
    print(type(err).__name__)  # UndefinedError: the missing variable stops the render

In practice, test every template with the awkward inputs first: an empty list, a very long value, and text that itself contains braces. Store those cases next to the prompt, because they are exactly what the evaluation lesson later in this course will replay on each new version. For background on the discipline as a whole, see prompt engineering in the glossary.

Working with a registry from code

To create a new prompt using the Python API, use the mlflow.genai.register_prompt() API.

Updating an existing prompt with a new version also calls mlflow.genai.register_prompt, with the existing prompt name.

The detail worth copying is not the function name but the shape of the workflow: a new version is created by registering again under the existing name. Whatever registry you choose, confirm with a throwaway prompt that it keeps the earlier version intact before you rely on it, then record in your team notes which label production traffic reads from, since that single fact decides what a rollback has to change.