Run AI Agents Reliably: Patterns, Tests & Production Checklist

Why this matters

AI agents promise automation across software workflows, but many teams report brittle behavior, unexpected costs, and operational failures when agents leave the lab. This guide distills patterns and concrete techniques to make agents observable, testable, and maintainable in production.

High-level checklist

Design for determinism where possible: make decisions reproducible and log inputs/outputs.
Use an orchestration layer that enforces timeouts, retries, and circuit breakers.
Isolate tool access and apply capability-based permissions and sandboxes.
Implement record-and-replay testing and strong mocks for CI.
Monitor cost, latency, success rates, hallucination signals, and business KPIs.

Architecture patterns

1) Agent controller and worker separation

Keep a lightweight controller that validates plans, enforces policies, and schedules work. Worker processes execute plan steps with explicit tool interfaces. This reduces blast radius and enables independent scaling.

2) Idempotent, composable tools

Wrap each external operation (database write, API call) in an idempotent function with a clear input/output contract. Prefer side-effect-free dry runs and prepare/commit phases if the tool supports them.

3) Deterministic replay and state checkpoints

Record the agent's entire decision trace (prompts, inputs, model responses, chosen actions). Use these traces for deterministic replay in tests and postmortems.

Practical implementation: a minimal agent runner

Below is a compact reference implementation pattern (pseudo-Python) showing a supervised agent loop with timeouts, retries, and a replay log. Adapt this pattern to your stack and enforce strict input validation for tool calls.

from time import time, sleep
import json
 
MAX_STEP_DURATION = 10  # seconds
MAX_RETRIES = 2
 
class ReplayLog:
    def __init__(self, file_path):
        self.fp = open(file_path, 'a')
 
    def record(self, entry):
        self.fp.write(json.dumps(entry) + '\n')
        self.fp.flush()
 
class ToolError(Exception):
    pass
 
def call_tool(tool_fn, payload, timeout):
    # enforce timeout at call site or via subprocess
    start = time()
    result = tool_fn(payload)
    if time() - start > timeout:
        raise ToolError('tool timeout')
    return result
 
def safe_execute_step(step, tools, replay):
    # step contains: { 'prompt': ..., 'action': 'call_tool_name', 'args': {...} }
    replay.record({'type': 'step_start', 'step': step, 'ts': time()})
 
    tool_name = step.get('action')
    if tool_name not in tools:
        raise ToolError('unknown tool: ' + str(tool_name))
 
    for attempt in range(0, MAX_RETRIES+1):
        try:
            out = call_tool(tools[tool_name], step.get('args', {}), timeout=MAX_STEP_DURATION)
            replay.record({'type': 'step_success', 'step': step, 'out': out, 'ts': time()})
            return out
        except ToolError as e:
            replay.record({'type': 'step_retry', 'step': step, 'err': str(e), 'attempt': attempt, 'ts': time()})
            if attempt == MAX_RETRIES:
                replay.record({'type': 'step_fail', 'step': step, 'err': str(e), 'ts': time()})
                raise
            sleep(0.5)  # backoff
 
# Example tool: a safe HTTP POST wrapper that validates responses
import requests
 
def http_post_tool(args):
    url = args['url']
    body = args.get('body', {})
    resp = requests.post(url, json=body, timeout=5)
    if resp.status_code != 200:
        raise ToolError(f'upstream {resp.status_code}')
    return resp.json()
 
# Wire it together
if __name__ == '__main__':
    replay = ReplayLog('agent.replay.log')
    tools = {'http_post': http_post_tool}
 
    plan = [
        {'prompt': 'create ticket', 'action': 'http_post', 'args': {'url': 'https://api.example/tickets', 'body': {'title': 'issue'}}}
    ]
 
    for step in plan:
        try:
            out = safe_execute_step(step, tools, replay)
            print('step out', out)
        except Exception as e:
            print('fatal step error', e)
            break

Notes on the example:

Keep tool wrappers small and focused; they validate inputs and responses.
Replay logs use JSON lines to simplify parsing and deterministic replay in tests.
Replace sleep backoffs with jittered exponential backoff for production.

Testing and CI: record-and-replay and mocked models

Production-quality agents need tests that run in CI without calling real model endpoints or side-effecting tools.

Record representative traces from staging with controlled inputs.
In CI, replay traces against the agent code and assert the sequence of tool calls and outputs.
Provide a stable mock model that returns the recorded model outputs, not regenerated text.

Example: deterministic replay test

def load_trace(path):
    with open(path) as f:
        return [json.loads(l) for l in f]
 
# In tests: replace model call with a function that returns the recorded response
mock_model_responses = {
    'prompt id 1': 'model output text'
}
 
def mock_model(prompt):
    return mock_model_responses[prompt]
 
# Assert that replaying the trace results in the same tool sequence
trace = load_trace('tests/fixtures/example_trace.json')
# run agent in replay mode which uses mock_model and a test tool harness
# assert tool calls == trace tool calls

Observability and metrics

Instrument these signals:

Per-step latency, retries, and error rates.
Model failure signals: unexpected token patterns, low-confidence indicators, or mismatched schema.
Business KPIs: task completion rate, human override rate, cost per task.

Push metrics to a monitoring system and set alerts on regression thresholds (e.g., rising hallucination indicators or sudden cost spikes).

Safety, permissions, and sandboxing

Limit agent capabilities by default (least privilege). Tools should require explicit enablement.
Use sandboxed runtimes for code execution tools (containers, seccomp, language sandboxes).
Implement human-in-the-loop gates for high-risk actions (financial transfers, destructive infra changes).

Common failure modes and mitigations

Non-deterministic outputs: use schema validators and confidence checks; fall back to human review.
Escalating cost: add budget-aware admission control and per-call cost accounting.
Long tail latency: enforce per-step timeouts and queue backpressure.
Untrusted tool results: verify with checksums, idempotency tokens, or secondary validations.

Tradeoffs

Safety vs. agility: stricter controls reduce incidents but slow iteration and may increase manual work.
Determinism vs. model improvement: recording and replay helps reliability but can mask model drift; refresh recorded traces regularly.
Cost vs. fidelity: expensive grounding (tool calls, retrieval, fine-tuning) improves correctness but raises unit cost.

Conclusion

AI agents can automate complex workflows, but production reliability requires engineering discipline: small, testable tools, careful orchestration, deterministic replay for tests, robust observability, and safety-first permissions. Start small, instrument everything, and run regular replay-based regression tests to catch failures before they affect users.

Design Patterns and Practical Steps to Run AI Agents Reliably in Production