What you will build

This capstone ties the three earlier lessons together. You will take one prompt that is currently a string in application code and run it through a full life cycle: move it into a registry, give it variables, record two versions, test both against a set of awkward inputs, promote one with a label, and rehearse the rollback. Pick a prompt you really maintain, such as a support reply, a summary or a classifier instruction. A real prompt will surface problems that a toy never does.

Each step ends with a small deliverable. Keep them in one folder, because the final write-up is assembled from them.

Step 1: move the prompt out of the code

Find where your prompt lives today and copy it into a file, untouched. Resist the urge to improve it yet. The first version in a registry should be the wording that is in production now, so that every later change is measured against something real.

Instead of hardcoding prompts in application code, you manage them centrally in Langfuse.

That sentence describes the goal in one product's words, and the same goal applies to a home-grown registry. The application should ask a central place for the prompt by name and label, instead of carrying the text itself. Some tools let you start from either direction.

You can create a prompt from scratch in the UI, or import the existing prompts from your application.

Deliverable: the current wording stored as version one, plus a note listing every place in the code that used to hold a copy.

Step 2: add variables

A prompt becomes reusable when the parts that change are named. Replace the request-specific text with variables, and leave the standing instruction as fixed words.

Use double curly braces for variables in the template.

Name each variable after the role it plays, such as message or customer, rather than after an example value. Decide what must happen when one is missing. The safest answer is that rendering fails, because an instruction that silently loses a variable reaches the model half-written.

Deliverable: a template with at least two variables and a one-line statement of what each one holds.

Step 3: register, tag and compare versions

Write a second version that changes one thing, for example a stricter length limit or an added audience. Changing one thing at a time keeps the comparison readable.

To update an existing prompt with a new version, use the register prompt API with the existing prompt name.

Registering under the same name adds a version and leaves the first untouched. Record why you made the change, because a version history without reasons is nearly useless six weeks later.

Tags can be applied at the prompt level, or to individual prompt versions.

Use tags for facts that are true of a whole prompt or of one version, such as the owning team or the evaluation date. Then look at what actually differs between your two versions.

With multiple versions of a prompt, you can compare them in order to understand the changes between versions.

Where a registry offers a visual comparison, use it.

To compare prompt versions in the MLflow UI, you click on the Compare tab in the prompt details page.

If your registry has no visual comparison, a plain text diff of the two templates does the same job. Deliverable: two versions, a tag on each, and a diff you can read in under a minute.

Step 4: write the cases that will judge them

Before running anything, write the cases. Aim for at least six, and make half of them uncomfortable: an empty message, a very long one, a message in another language, one that contains template braces, one that asks for something the prompt should refuse. Tag each case with the behaviour it protects.

Pass or fail is not the only thing to count. The testing guide you met earlier lists measures that suit this step.

Edge case analysis measures the percentage of edge cases handled without errors.

User feedback can be implicit, with measures such as task completion rates.

Edge-case coverage is something you can compute today from your suite. Implicit feedback only exists once real users are involved, so note it down as a metric to add after launch. Deliverable: the case list with one line per case saying what it protects.

Step 5: label, load and run

Langfuse's overview also covers environments.

Use version control and labels to manage deployments across environments.

Point a production label at the first version and a staging label at the second. The application then loads whatever each label points to, and neither the code nor the test harness mentions a version number.

To use a prompt in an LLM application, you can load it using the mlflow.genai.load_prompt() API, and then fill in the variables using the format() method of the prompt object.

The compact script below does the same thing with nothing but the standard library. It keeps versions in a list, resolves a label to a version, refuses to render when a variable is missing, and runs the cases against both labels. Notice that a message containing braces is inserted as plain text and is not expanded a second time.

import re

PROMPTS = {}  # name -> list of templates; position + 1 is the version number
LABELS = {}   # (name, label) -> version number
PLACEHOLDER = re.compile(r"\{\{\s*(\w+)\s*\}\}")


def register(name, template):
    PROMPTS.setdefault(name, []).append(template)
    return len(PROMPTS[name])


def render(name, label, **variables):
    template = PROMPTS[name][LABELS[(name, label)] - 1]
    missing = set(PLACEHOLDER.findall(template)) - variables.keys()
    if missing:
        raise KeyError(f"missing variables: {sorted(missing)}")
    return PLACEHOLDER.sub(lambda m: str(variables[m.group(1)]), template)


register("reply", "Reply politely to this message: {{ message }}")
register("reply", "Reply politely and briefly to this message from {{ customer }}: {{ message }}")
LABELS[("reply", "production")] = 1
LABELS[("reply", "staging")] = 2

CASES = {
    "greeting": {"customer": "Sam", "message": "hello"},
    "empty": {"customer": "Sam", "message": ""},
    "braces": {"customer": "Sam", "message": "{{ not_a_variable }}"},
    "no-customer": {"message": "hello"},
}

for label in ("production", "staging"):
    for case_id, variables in CASES.items():
        try:
            print(label, case_id, "->", render("reply", label, **variables))
        except KeyError as err:
            print(label, case_id, "-> error:", err)

Read the output carefully. The case without a customer renders fine against the first version and fails against the second, which is precisely the kind of difference a promotion decision should hinge on. Replace the printing with real checks, send the rendered prompts to your model, and score the results the way the evaluation lesson showed.

Deliverable: a results table with one row per case and one column per label.

Step 6: promote, then rehearse the rollback

If the staging version holds up, move the production label to it. Then, before you walk away, practise undoing it. Move the label back, run the suite again, and confirm that the results match what you recorded for the first version. A rollback you have never tried is a hope, not a procedure.

Write down, in your own words, the exact steps and how long they took. If the answer involves a code change or a deployment, the prompt is not yet managed the way this course describes.

Write-up

Finish with a page that a teammate could act on. It should say what the prompt does, what changed between versions and why, which cases protect it, what the suite showed, which label serves production now, and what you will monitor after launch. Add one paragraph on what surprised you, and one on the case you would add next. Link the glossary entry on prompt engineering if you share it with someone new to the field, and keep the latency of the registry lookup on your monitoring list.