From Chatbots to Agent Gateways: How to Control What AI Agents Can Touch

For most of the last few years, the risk with a language model was a sentence. A chatbot could say something wrong, biased, or embarrassing, but it could not reach into your systems and change anything. That made safety mostly a question of what the model said. Agents broke that assumption. The moment a model can call tools, write files, hit APIs, and chain those actions together toward a goal, the question stops being “what did it say” and becomes “what is it allowed to do.”

That is the gap an agent gateway fills. It is worth separating from a related idea: a runtime firewall asks whether a request looks malicious and blocks the bad ones. A gateway asks a different question, which is what the agent is permitted to do in the first place, even when the request is perfectly legitimate. One is detection, the other is authorization. This post is about the authorization layer: what an agent gateway is, what it controls, why a system prompt is not a substitute, and a starter policy you can apply before you connect an agent to anything real.

What is an agent gateway?

An agent gateway is a control plane that sits between an autonomous agent and the systems it can act on. Every tool call the agent wants to make passes through it, and the gateway decides whether that call is allowed, under which credentials, in which environment, and whether it needs a human in the loop before it runs.

The closest familiar analogy is an API gateway combined with an identity and access management system. An API gateway fronts your services and enforces authentication, routing, rate limits, and logging. An IAM system decides who can do what. An agent gateway applies both ideas to a new kind of caller: a non-deterministic model whose next request you cannot predict. Because you cannot anticipate what the agent will try, the gateway works from the opposite assumption, which is that the agent may attempt anything its available tools permit, so the safe set of tools and actions has to be defined explicitly rather than left open.

How is a gateway different from a firewall?

The two are easy to conflate because both sit in the request path and both can block things. The distinction is the question each one answers. A firewall is looking for adversarial intent. A gateway is enforcing policy. A request can be completely benign and still be denied by the gateway because the agent simply does not have permission, and a request can look clean to the gateway while a firewall flags it as an injection attempt.

They are complementary layers in a defense-in-depth stack. The firewall reduces the odds that the agent is manipulated. The gateway makes sure that even a manipulated agent can only reach the small set of things you explicitly allowed. Run both and a successful injection still cannot turn into a destructive action that was never in scope.

What does an agent gateway control?

A complete gateway governs five surfaces. You can implement them as a vendor platform or as a thin wrapper around your own tool calls, but the surfaces are the same either way.

• Identity. The agent has its own identity and credentials, separate from the human who started it. Without this, you cannot tell which agent did what, scope permissions per agent, or revoke a single misbehaving one without taking down the rest.
• Tool registry and scoping. The gateway exposes an explicit list of tools the agent may call, and nothing else. New tools are opt-in, not discovered at runtime. This is the difference between “the agent can use these four tools” and “the agent can use whatever it can reach.”
• Environment isolation. Tool calls run against scoped resources: a sandbox or staging environment, a database role that can only see the rows it needs, credentials that expire. The agent operates inside a blast door, not on the whole house.
• Action approval. Before a side-effecting action runs, the gateway classifies it by blast radius and reversibility. Read-only and easily reversible actions pass through. Destructive or irreversible ones (deleting records, issuing payments, sending external messages) queue for human approval or are blocked outright.
• Observability and audit. Every tool call, verdict, and approval is logged with enough context to reconstruct what happened. When an agent does something surprising, you need to answer “what did it call, with what arguments, and why was that allowed” after the fact.

Notice that none of these depend on the model behaving well. That is the whole design goal. The gateway is the part of the system that stays correct when the model does not.

Why a system prompt is not a permission boundary

The most common mistake is to express permissions in the prompt. “You are a support agent. Never issue refunds over $100. Never touch the production database.” That reads like a rule, but it is only an instruction, and the model is free to ignore it the moment something convinces it to. A poisoned tool response, an indirect injection buried in a document the agent reads, or a clever user message can all override prompt-level guidance, because the model has no mechanism to enforce its own constraints.

The reframing is the same one that makes a firewall useful: behavior you ask for is a property of the model, but permissions are a property of the deployed system. If a refund over a threshold must never happen, the limit belongs in the code path that issues refunds, where it holds regardless of what the model decides. The prompt can still describe the intended behavior, and it should, but it is a usability feature, not a security control. Anything you would be unwilling to let the agent do on its worst day has to be enforced at the gateway.

How do you scope what an agent can touch?

The governing principle is least privilege, applied per use case rather than per agent platform. An agent should start with no access and earn each capability for a specific job. In practice that means scoping along three axes at once.

• By tool. Grant the smallest set of tools that completes the task. A summarization agent does not need a write tool. A triage agent that routes tickets does not need the tool that closes them.
• By environment. The same tool can be safe or dangerous depending on what it points at. Wire tools to staging and read replicas first, and promote to production resources deliberately, not by default.
• By action severity. Within an allowed tool, separate the reversible operations from the irreversible ones, and put the irreversible ones behind approval. “Draft an email” and “send an email to a customer” are not the same permission.

A useful mental model is capability tiers. A bronze tier reads and drafts and cannot change anything. A silver tier can make reversible changes inside a sandbox. A gold tier can act on production resources, and only after the lower tiers have proven the agent behaves. Default everything to deny, and treat each promotion as an explicit decision with an owner.

Where does evaluation fit in?

Scoping answers what an agent is allowed to do. Evaluation answers whether it has earned that allowance. This is where the recent wave of agent benchmarks becomes practical rather than academic. Harnesses like Harbor and EvoSkill connect agents to containerized benchmarks such as SWE-Bench Verified and Terminal-Bench, which run the agent through realistic, long-horizon tasks in a sandbox and measure whether it actually completes them without breaking things. Domain-specific suites are emerging in the same shape, for example CHI-Bench for healthcare agents, where the cost of a wrong action is high and the evaluation has to reflect that.

The connection to the gateway is direct. A benchmark proves a capability in a controlled environment. The gateway is the promotion gate that turns that proof into access. An agent that has passed the relevant evaluation can be granted the next tier of tools or the production environment, and an agent that regresses can have that scope revoked. Without a gateway, a good benchmark result is just a number. With one, it becomes the condition under which real permissions are granted and removed.

A starter policy before you connect an agent to real systems

You do not need a platform to start. Before wiring an agent into a CRM, a ticketing system, or anything that handles money, work through this sequence. Each step is a control you can implement as code around your tool calls.

1. Inventory tools by blast radius. List every tool the agent could call and label each one read-only, reversible, or irreversible. This list is the thing you are actually governing.
2. Default to deny. Start with an empty allowed set and add only the tools a specific task needs. Anything not on the list is unreachable.
3. Separate environments. Give the agent staging credentials and a scoped database role to begin with. Production access is a later, separate decision.
4. Require approval on irreversible actions. Anything you labelled irreversible queues for a human, or is blocked, until you have evidence it can be trusted.
5. Log every call. Record the tool, the arguments, the verdict, and the outcome, so you can reconstruct any incident and review what the agent has been doing.
6. Set quotas and rate limits. Cap how many actions the agent can take in a window. A runaway loop is contained by a limit, not by hoping it stops on its own.
7. Tie promotion to evaluation. Define what an agent has to demonstrate before it moves up a capability tier, and make widening its scope an explicit decision rather than a default.

Done in order, these turn an open-ended agent into one whose worst case you have already bounded. You are not trying to predict every mistake. You are making sure the mistakes that can happen are small and recoverable.

What are the limits?

A gateway is a strong layer, not a complete answer. Worth being honest about the tradeoffs.

• It is authorization, not detection. A gateway limits what an agent can do, but it does not tell you whether a request was an injection attempt. That is the firewall's job, which is why the two belong together rather than one replacing the other.
• It adds friction and ops overhead. Approval queues, scoped credentials, and logging are work to build and maintain. The payoff is concentrated on the actions where a mistake is expensive, so scale the friction to the blast radius.
• Permissions drift. Scopes that were tight at launch loosen over time as people grant access to unblock something and forget to revoke it. Permissions need periodic review the same way any access control does.
• Over-restriction kills utility. Lock an agent down too hard and it cannot do anything useful, and people route around it. The goal is the smallest set of permissions that still lets the agent do its job, not zero permissions.

Frequently asked questions