Guide Published Last updated 7 min read

Building Production-Ready AI Agents: Architecture, Evals and Cost

The map for taking an AI agent to production: tools behind an MCP boundary, a person before every irreversible write, evals from real traffic, and cost measured per request. Each section links to the deep guide.

Tayyab JavedFreelance AI agent developer
Building Production-Ready AI Agents: Architecture, Evals and Cost
On this page 10 sections
  1. What "production-ready" means for an agent
  2. Architecture: the loop, the tools, and the boundary
  3. Writes: a person before anything irreversible
  4. Evals: prove a change did not make it worse
  5. Cost: measure per request, then cut
  6. A short pre-launch checklist
  7. In practice
  8. Who should build it
  9. Frequently Asked Questions
  10. Conclusion

A production-ready AI agent is a model in a loop with tools, plus four things a demo skips: a defined boundary for each tool, a person on every risky write, an eval set that runs before each change, and a cost per request you actually measure. This page is the map. Each section is a short answer and a link to the post that goes deep.

Quick answer

  • Architecture: one agent loop, tools behind a protocol boundary (MCP), state in a durable checkpointer, and a pause before any write that matters.
  • Evals: a labelled set from real traffic, scored on the path the agent took and the result, run before every prompt or model change.
  • Cost: measure cost per request per route, then route easy traffic to a cheap model, cache, and cap output. List prices are the start of the estimate, not the bill.
  • Order: ship one job with one tool, add evals, then add tools. The how to build an AI agent guide is the starting point.

What "production-ready" means for an agent

A demo answers the happy path once. A production agent answers the same request a thousand times, on messy inputs, with tools that time out, and nobody watching each run. In my experience the gap is not the model. It is four questions: what can each tool touch, what happens before a write, how do you know a change did not make it worse, and what does one request cost. The rest of this page takes those in order.

If you have not built the loop yet, start with how to build an AI agent: the five parts every agent has (model, tools, memory, control loop, guardrails) and when a plain workflow is the better choice.

Architecture: the loop, the tools, and the boundary

Keep the agent loop small and put the tools behind a boundary you can test on its own. The Model Context Protocol is that boundary for most new builds: a host (Claude Desktop, Cursor, or your own agent) opens a client per server, and the server exposes tools, resources and prompts. The pieces and how they connect are in MCP server architecture.

The MCP specification defines two standard transports: stdio, for a subprocess the client launches, and Streamable HTTP, where each message is an HTTP POST to a single endpoint and the reply is JSON or a request-scoped SSE stream (MCP transports, checked November 14, 2026). Use stdio on one machine and Streamable HTTP for anything other systems call.

  • Build one server: how to build an MCP server in TypeScript or Python, tested with MCP Inspector before a model sees it.
  • Lock it down: the MCP authorization spec is optional, applies to HTTP transports, and is based on OAuth 2.1; a stdio server should take credentials from its environment instead (MCP authorization, checked November 14, 2026). The audience check and the 401/403 responses are in MCP server authentication.
  • Run it: how to deploy an MCP server covers the bind address, TLS in front, Host and Origin checks, and where to host a long-lived process.

Writes: a person before anything irreversible

Reads can run on their own. Refunds, record changes and anything you cannot undo should wait for a person. In LangGraph that pause is the interrupt() function: the graph saves its state with a checkpointer, waits, and resumes when you send a Command(resume=...) on the same thread ID. The docs say a durable checkpointer should back this in production, and that the node restarts from its beginning on resume, so any side effect before the interrupt must be safe to repeat (LangGraph interrupts, checked November 14, 2026).

The full pattern, with a Postgres checkpointer and a resume flow, is the LangGraph human-in-the-loop tutorial, which uses interrupt(). The docs describe the older interrupt_before and interrupt_after options as static breakpoints for debugging, not for approvals.

Evals: prove a change did not make it worse

An agent eval scores the path as well as the answer: which tools it called, in what order, with what arguments, and whether the final result was right. Build the set from real, messy traffic, not from examples that look like your prompt. Run it before every prompt, model or tool change, and sample live traffic after launch.

The four layers (unit, trajectory, outcome, online) and a comparison of LangSmith, Langfuse and Braintrust are in AI agent evaluation. Evals are also what make the cost work below safe: you cannot route traffic to a cheaper model without a set that tells you whether the answers held.

Cost: measure per request, then cut

Start with the arithmetic. Input tokens times the input price plus output tokens times the output price, per request, per route. Current list prices for Claude, GPT, Gemini and Grok, with cache and batch rates and a worked cost per conversation, are in the LLM API pricing comparison. Read the result as a floor: history grows, tools add round trips, and an agent that loops can spend a month's budget on one bad ticket.

  • The fixes, in order: how to reduce LLM costs ranks nine tactics by effort and payoff, from output caps to caching.
  • The biggest lever: LLM routing sends easy requests to a small model and hard ones to a strong model. Only do it with an eval set in place.

A short pre-launch checklist

  1. Every tool has a written scope: what it can read, what it can write, and what it must refuse.
  2. Every irreversible write pauses for a person, and the agent reads the record back after the write instead of trusting the tool's success message.
  3. State lives in a durable checkpointer, so a restart does not lose a paused run.
  4. An eval set built from real requests runs before every change, with a pass bar written down.
  5. Cost per request is logged per route, with an alert before the monthly budget, not after.
  6. Remote tool servers sit behind TLS and token checks; nothing on stdio is exposed to the network.

In practice

The ERP operations agent is the build I hold this checklist against. Reads run without a pause. A write stops until a person approves it, and the agent then reads the records back rather than trusting the tool's success response. Routing precision on 211 hand-labelled prompts moved from 80.3% to 95.2%, and that number only exists because the labelled set existed first. On cost, the cost-optimization case study is a measured bill, not list-price arithmetic: caching, prompt compression and routing cut a SaaS platform's OpenAI spend by 65%. When I scope a new agent, I ask for the eval set and the write list before I ask about the model.

Who should build it

Whoever you hire should be able to show the three things on this page: a tool boundary, a paused write, and an eval set that moved a number. How to hire AI engineers has the screening interview and published rates. If you want me to build it, that is AI agent development; if an agent is already live and misbehaving, the production audit is the starting point.

Frequently Asked Questions

What makes an AI agent production-ready?

A scoped set of tools, a person approving irreversible writes, state in a durable checkpointer, an eval set from real traffic that runs before every change, and cost per request measured per route. A demo that works once has none of these by default.

Do I need MCP to build a production agent?

No, but it gives tools a boundary you can test, version and secure on their own. The MCP spec defines stdio for a local subprocess and Streamable HTTP for a remote server, and its authorization spec, based on OAuth 2.1, applies to the HTTP transport.

How do I add human approval to a LangGraph agent?

Call interrupt() inside the node that needs approval, compile the graph with a durable checkpointer, and resume with Command(resume=...) on the same thread ID. The node restarts from the beginning on resume, so keep side effects after the interrupt or make them idempotent.

What should an AI agent eval measure?

The trajectory (which tools, in what order, with what arguments) and the outcome (was the final result correct), on requests taken from real traffic. Run it before every prompt, model or tool change, and sample live traffic after launch.

How do I keep an agent's LLM cost under control?

Log cost per request per route first. Then cap output, cache repeated context, and route easy requests to a cheaper model, checking each change against your eval set. Current per-token prices are in the LLM API pricing comparison.

Conclusion

Production-ready is a short list: a boundary around each tool, a person before each irreversible write, an eval set that runs before each change, and a measured cost per request. Build the loop first, then add those four, in that order. Each link above goes one level deeper.

Taking an Agent to Production?

Free 30-minute scoping call. Bring the job you want the agent to do and the systems it touches. If it is not a fit, I will say so.

Book a Scoping Call
Share