The safety conversation in enterprise AI has focused predominantly on two things: whether AI systems want to do the right thing, which is the alignment problem, and whether they can be prevented from saying the wrong thing, which is the output filtering problem. Constraint architecture is a different question entirely. Regardless of what an AI system wants or outputs, what can it actually do?

For most current enterprise AI deployments, this question has a reassuring answer. An AI assistant that can only respond to text cannot order stock. A model with no database access cannot modify records. A system constrained to a specific tool set operates within the limits that tool set imposes. The safety is architectural, implicit in the design, and largely unplanned.

This article was originally published August 2024, and revised August 2026. The argument remains the same. What's changed is that the protocols that now carry tools to models, and agents to each other, did not exist at the time. I've extended two sections to note them against the constraint taxonomy.

Agentic systems change this. Agents are designed to act. They are given tools because the value of an agent lies in its capacity to take actions that would otherwise require human effort. Giving an agent tools removes the implicit constraints that made earlier AI deployments safe(r) by default. The safety architecture needs to become explicit, because the architectural safety of impotence is no longer available.

Types of Constraints

Not all constraints are equivalent. Understanding the difference is essential to building constraint architecture that provides real assurance rather than paying it lip service.

Hard constraints are actions the system cannot take regardless of instructions, context, or apparent good reason. They are implemented at the tool level, not the model level. A payment tool that rejects transactions above a defined threshold regardless of the instruction it receives is a hard constraint. It cannot be overridden by an instruction in the model's context. It cannot be bypassed by a prompt injection attack. It cannot be disabled by a reasoning chain that concludes the override is justified. It simply does not execute the disallowed action.

Soft constraints are actions the system is instructed not to take. They are implemented as rules in the model's context, in the system prompt, or in the model's fine-tuning. They are meaningfully better than no constraints. They are also bypassable, either by adversarial input that constructs a context in which the constraint appears not to apply, by model error in edge cases the constraint did not anticipate, or by prompt injection that overwrites or neutralises the constraint.

Approval gates are actions the system can propose but cannot take without confirmation from a human or a designated authority. They are appropriate for high-consequence, low-frequency decisions: actions that may be correct and should be executed, but that carry enough potential for harm that human review before execution is warranted. They preserve operational efficiency while maintaining human oversight at the consequential decision points.

A hard constraint cannot be overridden by an instruction, bypassed by a prompt injection attack, or disabled by a reasoning chain that concludes the override is justified. It simply does not execute the disallowed action.

Using the Wrong Constraint Type

Most current enterprise agentic deployments rely primarily on soft constraints. The system prompt includes instructions about what the agent should and should not do. These instructions are drafted carefully, reviewed by the legal and governance teams, and treated as the primary safety control for the system.

This is a category error. Soft constraints are a model-level control. They depend on the model interpreting the constraint correctly, applying it consistently across a large input distribution, and not being manipulated into reasoning its way around it. None of these properties can be guaranteed. The constraint is as reliable as the model, which means it is as unreliable as the model in precisely the circumstances where constraint reliability matters most: adversarial inputs, edge cases, and novel situations the constraint authors did not anticipate.

Hard constraints are a tool-level control. They do not depend on model behaviour. Their reliability is determined by the implementation of the tool, which is deterministic and auditable in ways that model behaviour is not. This means if your primary safety controls are at the model level, your safety case rests on the reliability of a probabilistic system. If your primary safety controls are at the tool level, your safety case rests on the reliability of deterministic code that can be verified and tested.

The Implementation Challenge

Moving primary safety controls from the model level to the tool level requires that the governance function be involved in tool design. This is where most current AI governance frameworks fall short. The frameworks focus on model evaluation: what the model can and cannot do, how it performs against safety benchmarks, whether it has been red-teamed. The tools the model has access to are treated as an engineering concern.

This is backwards. The model's output is probabilistic and imperfect. The tools are deterministic. Implementing hard constraints at the tool level gives you guarantees that implementing them at the model level cannot. A governance programme that evaluates the model extensively and does not evaluate the tool access constraints is securing the wrong thing.

Practically, this means the governance function needs to participate in defining the action space of each agent: what tools it has access to, what those tools can do at their limit, and what the hard constraint layer looks like. This is a design conversation, not a post-hoc review. It needs to happen before the tools are built, not after the agent is deployed.

The Model Context Protocol, published a few months after this piece first appeared, has since become the common way tools reach models. It standardises the interface: how a tool describes itself, how it is invoked, and how access to it is authorised. That standardisation helps, as it gives the hard constraint layer a defined place to live. The boundary between the model and the tool is now an explicit, documented surface rather than whatever each integration happened to improvise.

What the protocol does not do is decide what belongs on either side of that boundary. The specification says as much itself: it cannot enforce its security principles at the protocol level, and leaves consent and authorisation flows to the people building implementations. A server can therefore expose a payment tool with no ceiling as easily as one that refuses transactions above a threshold. The constraint remains a design decision taken by whoever builds the server, and nothing in the specification requires that decision to be taken deliberately.

And to urge caution taking claims at face value, more than most governance guidance provides, it treats a tool's own account of its behaviour, including its annotations, as untrusted unless the server supplying it is itself trusted. A description stating that a tool will not exceed a threshold is a claim the model has no means of checking, and a model that has read the description is no better informed about the limit than one that has not. Only the implementation settles what the tool does.

Adopting MCP therefore changes where the governance conversation needs to happen without changing whether it needs to happen at all.

Multi-Agent Constraint Propagation

Multi-agent systems introduce a specific constraint problem that single-agent architectures do not face. An agent can instruct another agent to take an action that the first agent could not take directly. If Agent A cannot transfer funds but can instruct Agent B, which can, the hard constraint on Agent A is not a constraint on the system. It is a constraint on one path through the system that can be routed around.

Constraint architecture for multi-agent systems therefore requires analysis of what each agent can accomplish indirectly, through instruction of other agents, not just directly through its own tool access. The relevant question is not "what can Agent A do?" but "what outcomes can Agent A achieve, accounting for all the agents it can instruct and all the tools those agents have access to?"

This analysis is more complex than single-agent constraint design, but it is tractable. It needs a map of the agent interaction graph, the tool access of each agent, and the composition of actions that could lead to outcomes outside the intended operational envelope. Performing this analysis before deployment is feasible. Doing so after a constraint violation has occurred in production is much more difficult.

Delegation between agents was mostly bespoke when this was written. It now has a protocol of its own in A2A, which gives an agent a standard way to advertise what it can do and to hand work to another agent that does something else. Discovery is part of the design, so an agent can establish what its counterpart offers before delegating to it.

That makes the propagation problem easier to analyse and harder to ignore. The interaction graph no longer has to be inferred from a set of point-to-point integrations, because it can be read from what each agent publishes about itself. But an agent advertising a capability is describing what it will do, not what it can be stopped from doing. The routing problem survives the protocol: if Agent B exposes a funds transfer and Agent A can reach Agent B, the hard constraint on Agent A still governs one path rather than the system. Standardised delegation makes that path easier to find, and it makes the path easier to build.

Decision Traceability as a Constraint Audit Mechanism

Constraint architecture does not eliminate the need for audit trails. It makes them more tractable. If you have defined the action space of each agent and implemented hard constraints at the tool level, you have also defined in advance what a complete audit trail needs to capture: the sequence of agent actions, the tool calls made, the inputs and outputs of each call, and any instances where a proposed action was blocked by a hard constraint or held at an approval gate.

An audit trail designed around a constrained action space is a more useful governance artefact than an audit trail designed to capture everything a system did. It has a defined scope. It has a defined vocabulary. And it makes it possible to ask: did the system behave within its constraint architecture, and if not, where and why?

What Boards Should Require

Before any agentic system reaches production, boards and governance functions should be asking for four things. Evidence of a documented action space: what can each agent in this system do, both directly and through instruction of other agents? An explicit constraint architecture: which actions require hard constraints, which soft constraints, and which approval gates, and how was that determination made? Evidence that hard constraints have been tested against adversarial inputs and cannot be bypassed through instruction alone. And a defined audit trail structure that captures constraint compliance as a first-class concern.

This is not onerous governance. It is the minimum viable for systems designed to act autonomously in consequential domains. Organisations that establish these requirements before deployment avoid the far more difficult problem of establishing them after something has gone wrong.