Skip to content
Olivier Reuland

Autonomy Is the Threat Model: Why the LLM Top 10 Wasn't Enough

Once an LLM can plan, choose tools, and act, autonomy becomes the threat model. The right response is least agency, narrow tool boundaries, and architecture that limits blast radius.

The LLM Top 10 was built for systems that mainly answer questions. Agentic systems introduce a different problem: they plan, choose tools, and act, sometimes across several steps before a human notices.

That is why the OWASP Top 10 for Agentic Applications for 2026 matters. It is not just an update to the old list. It is a reminder that autonomy changes the threat model.

The most useful idea in it, in my view, is not a category name but a design principle: least agency.

Autonomy should be earned, not assumed. Give an agent the minimum decision-making power and the minimum access it needs for the task in front of it, not the maximum you think might be convenient later.

How teams quietly build this problem

That sounds obvious. It is also the exact opposite of how many agents are being built.

A team starts with a harmless assistant. It summarises documents. Then it gets access to a ticketing system. Then an internal wiki. Then email. Then a database because someone wants it to "close the loop". At every step, the permissions make sense in isolation. By the end, the organisation has created an operator with broad access, vague boundaries, and no strong reason to stay inside them.

That is not just a prompt-injection problem. That is an autonomy problem.

The older LLM framing assumed a human was still close to the action. A user asks. The model answers. Maybe there is a tool call in the middle. Agentic systems break that assumption. The model is no longer only generating text. It is now choosing steps that affect the real world.

This is where the OWASP Securing Agentic Applications Guide 1.0 becomes useful alongside the top 10. The top 10 tells you what kinds of failure matter. The lifecycle guide tells you where those failures get invited into the architecture.

And architecture is the part too many teams still treat as ordinary engineering housekeeping.

Which model provider you use. Whether tools call systems directly or through a constrained service layer. Whether memory is shared or isolated. Whether the agent can choose parameters freely or only fill narrow slots. Whether a kill switch exists outside the environment the agent can influence. Those are not implementation details. That is the blast radius.

You can see it most clearly at the seams: tool definitions that are too broad, agent memory shared across tasks that should have stayed separate, fallback logic that quietly expands capability under failure, API keys with far more scope than the task requires, and observability that records the tool call but not the context that made the tool call seem reasonable. Those seams are where autonomy turns a normal defect into a security incident.

Least agency is the discipline that pushes back. An agent that summarises documents probably does not need to execute code. An agent that reviews invoices probably does not need access to every customer record in the environment. An agent that drafts outbound emails definitely does not need the same permissions as an agent that approves payments.

Yet broad access keeps showing up because it is easier to grant it once than to reason through the narrow path. That is understandable. It is also how teams accidentally build an agent with authority they would never hand to a human staff member on day one.

What least agency looks like in practice

The practical response is straightforward:

  1. Design narrow tools.
  2. Put sensitive systems behind constrained APIs rather than raw direct access.
  3. Separate planners from executors when the task matters.
  4. Make permissions task-scoped where possible.
  5. Keep memory and context boundaries tighter than feels convenient.
  6. Use canaries, staged rollouts, and kill paths you can trust.

Most of all, test the awkward paths, not just the happy path. What happens when the tool returns an error and the agent tries to compensate? What happens when instructions conflict? What happens when the agent is given a broader objective than the tool boundaries were designed for?

That is where the real security work lives.

The LLM Top 10 was not wrong. It was built for an earlier centre of gravity.

Once you add planning, tool choice, and real-world action, the question changes. It is no longer only "how do we stop the model saying something unsafe?" It becomes "how much freedom did we quietly give this system, and what happens when it uses that freedom badly?"

That is why autonomy is the threat model. If you do not design for least agency up front, the agent will usually inherit more power than you meant to give it, and correcting that later is rarely cheap.

Related

Olivier Reuland