Prompt Template Libraries — LangChain, Guidance

Variables and late binding

Every template starts with a problem: you write a prompt for one case, then need to reuse it for another with different inputs. The naive move is to string-format or f-string the values in. The scaling move is to use variables—placeholders in the template that bind at runtime.

Given the user query {user_query}, retrieve documents from {source}.

Context:
{retrieved_docs}

Answer the user query concisely.

Variables defer the binding until render-time. The same template works with different queries, different doc sources, different context windows, and different models—you do not rewrite the prompt for each. That flexibility is the reason template systems exist: {variable} lets you write once and parameterize infinitely.

Two key ideas. First, late binding means you can version the prompt independently of the data. You change the template, check it in, run evals against it, and all downstream systems pick up the new version automatically if they pull from a central store. Second, variable names act as a contract: if a template requires {retrieved_docs} and you pass {source_docs} instead, you find out at render time—earlier is better for catching mismatches.

Template composition × system + few-shot + user

Most prompts are not monolithic. A production system has a system prompt (stable role and instructions), a few-shot section (examples that steer the model), and a user prompt (the actual query). Concatenating them as strings is fragile; small changes to the middle break everything downstream.

Template systems let you compose these as separate units:

# system.txt
You are a customer support agent. Be concise and helpful.

# few_shot.txt
Example: Customer asks about refunds → explain policy.
Example: Customer asks about shipping → check tracking.

# user.txt
Customer: {user_message}

Each file is version-controlled separately. You update the system role, push v2.1, and every prompt that composes these three picks up the change. You A/B test two few-shot strategies by swapping the file. You add a new variable to the user template and no other section breaks. Composition is the reason large teams do not rewrite prompts every sprint.

LangChain PromptTemplate: familiar and verbose

LangChain's PromptTemplate is the most widely deployed library in production systems. It is familiar, handles chat message formats, and plays well with the rest of the LangChain ecosystem.

from langchain.prompts import PromptTemplate

template = PromptTemplate(
    input_variables=['user_query', 'context'],
    template="Given {user_query}, use {context} to answer."
)

filled = template.format(user_query="What is X?", context="Doc Y")
# Output: "Given What is X?, use Doc Y to answer."

Strengths: you declare variables upfront, LangChain validates at instantiation, and the API is straightforward. Weaknesses: the DSL is thin—no loops, no conditionals, no filters. For complex prompts, you write Python around it. For chat-based systems, LangChain also provides ChatPromptTemplate which handles message roles (system, user, assistant).

from langchain.prompts import ChatPromptTemplate

chat_template = ChatPromptTemplate.from_messages([
    ('system', 'You are a helpful assistant.'),
    ('human', '{user_query}'),
    ('ai', '{tool_context}')
])

messages = chat_template.format_messages(
    user_query="What is X?",
    tool_context="Available tools: Y, Z"
)

Use LangChain when: you need a lightweight, battle-tested system that hooks into the rest of LangChain (chains, agents, retrievers). Do not use it for prompts that need conditionals or complex formatting.

Guidance: constraint propagation and structured output

Guidance is the newer, tighter-coupled approach. It models prompt generation as a stateful program that interleaves template directives with the model's tokens. That sounds dense, but it solves a real problem: steering the model to produce structured outputs without losing quality.

import guidance

@guidance
def extract_fields(lm, text):
    lm += f"""Extract from: {text}
"""
    lm += """Name: {{name}}
"""
    lm += """Age: {{age}}
"""
    return lm

lm = guidance.models.OpenAI("gpt-4")
result = extract_fields(lm, "John Doe, 30 years old")

Guidance's {{variable}} syntax tells the model: the output must bind to this variable. The model still generates freely up to that point, but once it hits the variable, Guidance constrains the token stream to match a schema or a grammar you provide. It is like static type-checking for prompts: you get JSON that parses, not JSON-ish strings you hope parse.

Strengths: tight constraint on output structure, no post-hoc parsing, natural blend of template and logic. Weaknesses: steeper learning curve, tied to specific model APIs, less ecosystem support than LangChain.

Jinja2 and why it works for complex templates

For prompts that need loops, conditionals, and filters—think dynamic few-shot selection, multi-step reasoning scaffolds, or conditional tool exposure—reach for Jinja2, the battle-tested templating engine from Flask and web development.

You are a support agent.

{% if expert_mode %}
Expert instructions:
- Escalate high-severity issues immediately.
- Use advanced troubleshooting steps.
{% endif %}

{% for example in examples %}
Example: {{ example.input }} → {{ example.output }}
{% endfor %}

Customer: {{ user_query }}

Jinja2 gives you: conditionals ({% if %}), loops ({% for %}), filters ({{ text | truncate(50) }}), and macros (reusable fragments). You can pass Python lists, dicts, or objects, and compose them all in the template.

from jinja2 import Template

prompt_text = Template(template_string).render(
    user_query="What is X?",
    examples=[
        {'input': 'A', 'output': 'B'},
        {'input': 'C', 'output': 'D'},
    ],
    expert_mode=True
)

print(prompt_text)

Strengths: full programming power (conditionals, loops, macros), zero lock-in (pure Python library, no external service). Weaknesses: very permissive (you can write bugs in templates), no built-in schema validation, less model-specific (you still parse the output yourself).

Prompt loaders and centralized stores

Once you stop pasting prompts into code, you need a place to store them: files, databases, or a dedicated service. A prompt loader fetches the current version at runtime.

class PromptStore:
    def load(self, name: str, version: str = 'latest'):
        # Load from S3, database, or local files
        template_text = self._fetch(name, version)
        return PromptTemplate(
            template=template_text,
            input_variables=self._extract_vars(template_text)
        )

store = PromptStore()
template = store.load('customer_support')
prompt = template.format(user_query="I have a problem")

Benefits: prompts live outside the codebase, you version them independently, you can roll back a broken prompt instantly, you A/B test by pointing two code paths at different versions. Real teams often use a database with a metadata table (name, version, created_at, model, eval_score) so you can audit which version served which request.

Best practices: variable naming, escaping, and safety

Variables seem simple until they go wrong. Three practices save you.

1. Use clear, scoped variable names. {query} is ambiguous; {user_query} is not. {context} could be anything; {retrieved_docs} says what it is. Use prefixes if you compose templates: {system_instructions}, {fewshot_examples}, {user_input} makes it obvious where each belongs.

2. Escape user input. If a variable comes from user input or untrusted sources, always escape it before rendering. LangChain and Jinja2 handle HTML escaping by default; if you use raw string formatting, do it manually.

import html

unsafe = ""
safe = html.escape(unsafe)
template.format(user_input=safe)

3. Validate variable presence. If a template requires five variables and you forget one, better to fail at render-time with a clear error than silently render a malformed prompt. LangChain does this by default; Jinja2 requires you to set undefined=jinja2.StrictUndefined to raise on missing vars.

Trade-offs: complexity, performance, and maintainability

No template system is free. Here is how they trade off.

Pick LangChain if: you want simplicity, you are already using LangChain, your prompts are mostly static. Pick Guidance if: you need bulletproof structured output and are willing to learn new syntax. Pick Jinja2 if: your prompts have conditional sections or dynamic lists and you do not want to write Python orchestration.

Patterns: versioning, A/B testing, and rollback

Once templates are in a central store, you unlock workflows that scale.

Versioning: tag each prompt update with a semantic version (v1.2.3) or a timestamp. Log which version served which request so you can replay and debug. When an eval fails, you know which prompt version to blame.

A/B testing: point different traffic fractions at different prompt versions. Measure success rate, cost, latency. The winner becomes the default; the loser is archived. This is how teams improve—iteratively, on real data, not hunches.

import random

user_id = get_user_id()
variant = 'control' if random.random() < 0.5 else 'treatment'
template = store.load('support', version=variant)
response = llm(template.format(**context))
log_metric('variant', variant, success=response.success)

Rollback: if a new prompt degrades performance, you do not redeploy code—you point the loader back at the previous version. Instant fix. This is why templates belong outside the codebase.

When to use templates and when to hardcode

Templates are powerful, but they add a layer. Three questions decide if you need them.

1. Do you have more than one model call? One-off prototypes can hardcode. Multi-call systems should template. 2. Will the prompt change after launch? If yes, template. If it is frozen forever, hardcoding saves setup cost. 3. Do multiple services share the same prompt? Shared prompts must be templated; duplicating them is how you end up with inconsistent behavior.

A mental model: templates are infrastructure. They cost upfront but pay off the moment you have more than one codebase, model, or environment using the same prompts. Teams with one model in one service might skip them. Teams with five models across three services cannot.

Key takeaway

Use variables and late binding to parameterize prompts and reuse them across contexts. Compose your prompts from separate system, few-shot, and user sections so you can version and test each independently. Reach for LangChain PromptTemplate for simplicity and ecosystem fit, Guidance for bulletproof structured output, or Jinja2 for conditionals and loops. Store templates centrally so you can version, A/B test, and roll back without code changes. The discipline of not pasting prompts into code is what lets prompts scale past prototypes.