Pydantic AI in Production: Type-Safe Enterprise Architecture for Reliable LLM Applications

The Problem Pydantic AI Is Solving

Every team building on LLMs eventually meets the same class of production incident. The model returned a JSON blob that looked right until it wasn't — a missing field, a numeric string where a number should be, a nested object with an extra key that nobody caught. Downstream code either crashed at the wrong layer, or silently misinterpreted the payload. If you build enough of these systems, you learn to defend at the boundary: validate everything the model produces before it enters your program.

That is Pydantic's territory. In the FastAPI / SQLModel world, request bodies and DB rows have Pydantic models, and the framework rejects anything that does not conform. Pydantic AI applies exactly that idea to the LLM boundary. Prompts have typed inputs. Tools have typed arguments. Agent outputs have typed schemas. The model has to produce something that parses, and the framework retries or fails loudly when it does not.

The libraries this replaces — hand-rolled json.loads + try/except around model output, ad-hoc schema-validation via Zod-in-Python knock-offs, or the output_type=SomeBaseModel sugar in newer OpenAI/Anthropic SDKs — all solve pieces of this. Pydantic AI's contribution is treating typed agent boundaries as the *whole runtime*, not a feature you opt into on one call site.

What Pydantic AI Actually Is

Pydantic AI is a small, model-agnostic Python agent framework maintained by the Pydantic team. It gives you three things and deliberately not much else.

Agents are the primary object. An agent has a model, an instruction, a set of tools, and — importantly — a typed input and a typed output. Agent[Deps, Output] says explicitly what dependencies this agent needs at run time and what shape of answer it produces. The Python type-checker (and your IDE) understand this, so a downstream caller who tries to treat the output like a string when it is a TicketDecision gets a red squiggle, not a runtime surprise.

Tools are decorated Python functions with typed parameters. When the model asks to call a tool, Pydantic AI validates the arguments against the function's type signature before invoking it. If the model produces {"order_id": "ord_42"} and your tool expects order_id: int, the framework retries the call (up to a configurable limit) with the validation error fed back to the model, which usually corrects itself. This retry loop, done well, is the difference between a working agent and one that half-works in ways nobody can debug.

Dependency injection. Every agent run receives a typed Deps object — a database handle, a tenant context, a caller identity, whatever. Tools access it through the run context. This is the pattern that keeps agent code testable: swap the real deps for a fake one in tests, and the same agent code runs against both. If you have written FastAPI apps with Depends(...), the mental model transfers directly.

On top of these three you get streaming with the same typed guarantees, structured output validation with automatic model-side retry on failure, and native support for the standard model providers — OpenAI, Anthropic, Google Gemini, Groq, Ollama, Cohere, Bedrock — through a small provider abstraction. There is no vendored LangChain-style tool zoo and no chain builder. That is the entire point.

How Pydantic AI Compares to the Other Serious Options

Where Pydantic AI Fits in an Enterprise Stack

The pattern we deploy keeps Pydantic AI focused on the agent-runtime layer and lets other components handle everything else.

At the ingress you have a normal FastAPI or Starlette service — auth, rate limiting, tenant resolution. The request body is a Pydantic model. That model — or a projection of it — becomes the typed input to the agent.

Model calls route through an AI gateway rather than directly to a provider. Pydantic AI's provider abstraction lets you point at a LiteLLM proxy with one line; every subsequent budget, key-management, and cost decision then lives in one layer, not in every agent's config.

Tools that touch enterprise systems — a database, an internal API, a search backend — get exposed through Pydantic AI's tool decorator with dependency injection. If the tool is broadly reusable across agents, wrap it as an MCP server behind a thin Pydantic AI adapter; if it is agent-specific, keep it in-process.

Every agent run gets instrumented and forwarded to your LLM observability layer — Pydantic AI has native Logfire integration (also from the Pydantic team) and OpenTelemetry hooks that adapt to Langfuse or LangSmith without effort. This is where the typed outputs pay their second dividend: the trace records the exact schema, so you can query 'show me every run where the refund_amount field exceeded X' as a first-class operation.

Governance policy — what tools are allowed for which callers, what data classes require confirmation, what must be blocked outright — sits in a small policy service consulted from a pre-tool hook. Pydantic AI does not ship a hooks system as elaborate as the Claude Agent SDK; you achieve the same effect by wrapping tools in a decorator that consults the policy layer before dispatching.

A Production-Shaped Agent Setup

# agents.py — a typed enterprise agent with Pydantic AI

from dataclasses import dataclass
from pydantic import BaseModel, Field
from pydantic_ai import Agent, RunContext

# ---- Dependencies injected per run ----
@dataclass
class SupportDeps:
    orders: OrderService          # your service, real or fake in tests
    refunds: RefundService
    tenant_id: str
    user_id: str

# ---- Typed output the downstream code depends on ----
class SupportDecision(BaseModel):
    action: str = Field(pattern=r'^(answer|route_billing|route_returns|escalate)$')
    rationale: str
    refund_amount_usd: float | None = None
    confidence: float = Field(ge=0.0, le=1.0)

# ---- The agent itself ----
support_agent = Agent(
    'anthropic:claude-sonnet-4-5',   # or 'openai:gpt-5', or 'litellm:...' via gateway
    deps_type=SupportDeps,
    output_type=SupportDecision,
    system_prompt=(
        'You are the first-line triage for a SaaS support desk. '
        'You may look up orders, propose refunds up to $200, or escalate.'
    ),
    retries=2,   # retry on validation failure with model-side error feedback
)

# ---- Tools with typed args + dependency access ----
@support_agent.tool
async def lookup_order(ctx: RunContext[SupportDeps], order_id: str) -> dict:
    """Look up an order by ID. Read-only."""
    return await ctx.deps.orders.get(order_id, tenant=ctx.deps.tenant_id)

@support_agent.tool
async def propose_refund(
    ctx: RunContext[SupportDeps],
    order_id: str,
    amount_usd: float,
    reason: str,
) -> dict:
    """Propose a refund. Requires supervisor approval before executing."""
    if amount_usd > 200:
        raise ValueError('Refunds over $200 must be escalated to a human.')
    return await ctx.deps.refunds.create_pending(
        order_id=order_id, amount_usd=amount_usd, reason=reason,
        proposed_by=f'agent:{ctx.deps.user_id}',
    )

# ---- Usage ----
async def handle_ticket(ticket_text: str, deps: SupportDeps) -> SupportDecision:
    result = await support_agent.run(ticket_text, deps=deps)
    return result.output   # already validated as SupportDecision

Typed Outputs as an Operational Asset

The obvious benefit of typed outputs is correctness. The less obvious benefit is what they let your ops team do afterwards.

Because SupportDecision is a Pydantic model with named fields, every trace your observability platform captures has a queryable structure. 'Show me every agent run today where action was escalate and confidence was below 0.6' is one filter, not a regex over free-text output. Alerting on that same query is a couple of lines in your alerting stack. Regression analysis after a prompt change becomes 'did the distribution of action values shift' rather than a manual eyeball of transcripts.

The pattern that ages well: user-facing chat surfaces return prose, and every internal agent returns a schema. Even conversational agents can return prose plus structured metadata — the visible answer, plus {intent, sources, next_action, confidence} — and Pydantic AI's output type supports that shape cleanly.

Testing Pydantic AI Agents

The dependency-injection pattern is what makes agents testable. Pydantic AI ships with a TestModel that returns deterministic outputs matching the declared output type, so you can drive the agent through its tool calls and assert on the trace without a real LLM call. For higher-fidelity tests, FunctionModel lets you supply a callable that decides what the model 'says' given the message history — useful for testing branch coverage of the tool graph.

The pattern we use in enterprise projects: a small suite of TestModel-driven tests that assert the tool call graph is correct for a given input, plus a slower CI job that runs a curated golden set against a real (cheap) model to catch prompt regressions. Combined with typed outputs, this gives you a test surface that most agent frameworks either do not expose or bolt on as an afterthought.

Where Pydantic AI Tends To Be the Right Choice

Where It Is Not the Right Choice

If your team is not on Python, Pydantic AI is not for you — it is deliberately a Python-first library. If your workflow is a rigid state machine with the LLM as one node among many, a graph orchestrator like LangGraph is the closer fit and Pydantic AI ends up wrapped inside its nodes. If you need the wide LangChain component zoo — 200 built-in tools, memory backends, retrievers — Pydantic AI does not ship any of that, and adopting it means writing the ones you need. And if the value of the agent is *not* structured output but free-form conversation with a customer, typed-output-first framing adds ceremony to a problem that does not require it.

Seven Decisions Worth Making Before Your First Deploy

• Output type first. Design the output_type before the prompt. If you cannot write the Pydantic model that captures every valid answer, the agent's job is not defined tightly enough to build yet.
• One agent, one decision. Resist making one agent that does five things. Each with a narrow output type, chained by application code, is easier to test and easier to swap.
• Dependencies over globals. Every service the agent's tools need goes through the injected Deps — never a module-level import. This is the difference between agents that are testable and ones that pretend to be.
• Retry budget. Two retries is a good default. Higher than that hides a prompt problem behind a cost problem.
• Tool granularity. Fewer coarse tools with clear docstrings beat many tiny ones. The model uses docstrings to plan; write them like API reference, not code comments.
• Route through a gateway. Point the provider at your AI gateway, not the vendor's endpoint. Keys, budgets, and provider swaps then live in one layer.
• Instrument day one. Wire Logfire or your existing observability platform from the first commit. Typed outputs make the traces queryable in ways free-text agents will never be.

How To Approach a Pydantic AI Pilot

The pilot that produces a real answer looks like this. Pick one narrow decision your product has to make from an LLM — categorise this ticket, extract this invoice, propose this refund. Build one Pydantic AI agent with a tight output_type, two or three tools, and injected dependencies for anything that touches real systems. Point the model provider at your gateway. Wire traces to your observability layer. Run it in shadow mode for a week — the agent runs, its decisions are logged, but humans decide what actually happens. Then move to suggested-action mode where the agent proposes and a human accepts. Only then, and only for the safest subset of decisions, move to autonomous.

By the end you have concrete numbers on decision quality (accuracy against the human ground truth), retry rate (what fraction of runs needed re-prompts to produce valid output), cost per decision, and the specific failure modes you actually see. That is the input you need to decide whether to widen the agent's tool set, tighten its output type, or hand it to a different framework entirely.

If you would like help with the architecture — designing the output types, connecting to your existing gateway and observability layer, integrating with an existing governance framework, or picking between Pydantic AI and one of its alternatives for your specific workload — talk to us. We have deployed this pattern across support triage, document extraction, and internal ops agents, and the shape of the reference architecture is far more repeatable than most first builds assume.