Architecture

Reference Architecture for Enterprise AI Agents

A common way to describe how AI agents mature is a staircase: LLM Call, Agent Loop, Agent Framework, Agent Harness, Long-Running Agent, Governed Agentic System. It is a useful story about growing capability, but it treats security as an end state rather than a property every stage needs from the start — and it has no named place for a gateway, a control plane, risk assessment, or evidence collection.

This page summarizes our working alternative: the same capability staircase, extended with the infrastructure and assurance layers that enterprise deployment actually requires.

Build layer (sequential capability growth)

1. LLM. The reasoning engine. Stateless, no memory, no tools.
2. Agent. Model plus a harness loop: reasoning, tool calls, and a stop condition.
3. Agent Framework. Reusable orchestration and coordination primitives, e.g. LangGraph, CrewAI, OpenAI Agents SDK.
4. Agent Harness. The logical control layer: tools, context, guardrails, permissions, error handling.
5. Runtime. The operational hosting layer: compute, scaling, isolation, agent identity, networking.

Mediation and control layer (cross-cutting, over all running agents)

1. Gateway. The data plane for LLM, MCP, and agent-to-agent traffic: rate limiting, inline guardrails, cost control, tracing.
2. Control Plane. The control layer: agent registry, identity, policy-as-code, lifecycle management, kill switch.

Governance and assurance layer (enclosing)

1. Risk Assessment. Per-agent autonomy tier, data reach, and blast radius, determining which controls must be active.
2. Continuous Controls. Ongoing enforcement: runtime monitoring, behavioral baselines, anomaly detection, continuous evaluation.
3. Evidence Collection. Tamper-evident logs and audit trails, mapped to frameworks such as the EU AI Act, ISO 42001, and NIST AI RMF.

An emerging, still-immature final capability sits on top of this stack:Autonomous Security — agentic security controls that detect and respond at machine speed, under mandatory human oversight.

What common maturity models miss

• Security is treated as a final stage rather than a property that applies to every stage.
• No dedicated gateway layer for agent-to-tool, agent-to-LLM, and agent-to-agent traffic.
• No dedicated control plane for agent registration, identity, and lifecycle.
• "Harness" overlaps ambiguously with "framework," "scaffold," and "runtime."
• No named place for risk assessment, continuous controls, or evidence collection.

Cutting across every layer

• Identity (human and non-human)
• Secrets management
• Human-in-the-loop approval
• Policy

Security is not one box on the staircase. It is the walls of the building.