An agent's tool interface changes its success rate as much as the underlying model does, SWE-agent's own numbers prove it, and the three capabilities that make an agent useful are exactly what Simon Willison calls the lethal trifecta

The previous post in this series was about the loop that turns a base model into something that follows instructions and prefers good answers to bad ones. An agent is what happens when you take that already-aligned model and put it in a second loop, one against a real environment instead of a reward model: observe something, decide what to do, take an action with a real side effect, observe what actually happened, decide again. Every term in that sentence sounds like a reasoning problem, and most public writing about agents treats it as one, better prompts, better chain-of-thought, a cleverer planning strategy. The real, disclosed evidence points somewhere less flattering to the model itself: Princeton’s SWE-agent paper got a bigger, better-documented performance jump from redesigning the interface between the agent and its environment than most papers get from swapping the underlying model, and the specific combination of capabilities that makes an agent worth deploying at all, access to private data, exposure to content it didn’t choose, and a way to communicate outward, is the exact combination Simon Willison named the “lethal trifecta” for a structural reason no amount of prompt engineering fixes. This post is about both of those facts, and about the real, disclosed systems that already had to solve for them.

Workflow or agent? Anthropic’s own distinction, and why “just add a loop” is usually the wrong first move#

Before any of the mechanics below, Anthropic’s own December 2024 engineering post on this exact question draws a distinction worth taking seriously rather than treating as marketing language, because it’s the thing most agent postmortems trace back to skipping. A workflow orchestrates LLMs and tools through a predefined, fixed path someone designed in advance: prompt chaining, routing to a specialist sub-prompt, running steps in parallel, an orchestrator decomposing work for workers, an evaluator-optimizer loop checking and refining output. An agent, properly, is a system where the model itself decides the path: what tool to call, in what order, how many times, when the task is actually done. The post’s own stated recommendation is blunt and worth quoting directly: “find the simplest solution possible, and only increase complexity when needed.” A fixed workflow is more predictable, more debuggable, and fails in ways you can enumerate in advance. A true agent is more capable of handling a task you didn’t fully anticipate, and correspondingly harder to bound, test, or predict, because the whole point is that you didn’t write down the path it’s going to take. Nearly everything that goes wrong in the rest of this post is a consequence of that tradeoff, not of a bad model.

The ReAct loop, and what it trades away against planning first#

Yao et al.’s 2022 ReAct paper is the pattern underneath almost every agent framework built since: interleave reasoning and acting in the same generation, rather than treating them as two separate stages. Concretely, at each step the model produces a Thought (reasoning about what to do next), an Action (a tool call), receives an Observation (the tool’s real output), and repeats.

def react_loop(model, tools, task, max_steps=15):
    trace = [{"role": "user", "content": task}]
    for step in range(max_steps):
        response = model.generate(trace, tool_schema=tools.schema())
        trace.append({"role": "assistant", "content": response})
        if response.is_final_answer:
            return response.answer, trace
        tool_name, args = response.action, response.action_input
        try:
            observation = tools.execute(tool_name, args)
        except ToolError as e:
            observation = f"Error: {e}"          # the failure mode below starts here
        trace.append({"role": "tool", "content": observation})
    return None, trace   # exhausted max_steps without a final answer

ReAct’s own reported results (HotPotQA, FEVER, ALFWorld, WebShop) show the reasoning trace doing real work beyond just picking actions: it lets the model track a multi-step plan, notice when an observation contradicts its assumption, and revise, in the same context, without a separate planning pass. The alternative, plan-and-execute (decompose the whole task into a fixed sub-task list first, then execute each one), is the workflow-flavored version of the same idea, and the real tradeoff is worth stating plainly rather than treating one as strictly better: a full upfront plan is cheaper in tokens and easier to audit before anything runs, but it’s brittle exactly when an early tool call returns something the plan didn’t anticipate, a search that comes back empty, a file that doesn’t exist at the expected path, because the plan was fixed before that information existed. ReAct pays for adaptability with more tokens and more turns; plan-and-execute pays for efficiency with fragility to surprise. Neither is a free lunch, and picking between them is a real, first-order design decision, not a preference.

The interface is not free: SWE-agent’s real numbers#

Here is the result worth sitting with longer than the ReAct loop itself. SWE-agent (Princeton, NeurIPS 2024) didn’t improve on prior coding-agent results primarily by using a better model. Its central contribution was the Agent-Computer Interface (ACI): a purpose-built, constrained set of commands, view, edit, search, run, replacing raw shell access, designed around two explicit principles the paper states directly. Simplicity: each command takes few options and has concise, unambiguous documentation, because a command an agent can misuse in many ways gets misused in many ways. Efficiency: operations that matter, like navigating to a specific file location and editing it, are consolidated into as few actions as possible, so one step makes real progress instead of three steps groping toward it. The real repository is public, and the real, reported number attached to this specific design choice: 12.5% pass@1 on SWE-bench, state-of-the-art at publication, and 87.7% on HumanEvalFix, using the same underlying model class other, worse-performing scaffolds had access to.

That’s the whole point worth internalizing before writing a single line of agent code: giving a capable model unrestricted raw bash access is not the maximally capable configuration, it’s frequently the worse one, because an interface with too many degrees of freedom is exactly what produces the malformed, half-correct actions that burn a turn without making progress. The design lesson generalizes past coding agents specifically: every tool you hand an agent is an interface design problem with the same two axes, how easy is this for a language model specifically to use correctly, and how much real progress does one successful call actually buy.

Memory: three real mechanisms, and two genuinely different ways to learn from failure#

An agent’s memory splits into three mechanisms with three different cost profiles, not one generic “memory” feature. Short-term memory is just the context window, exact and complete, but bounded, and every tool observation appended to it is competing for the same finite space as the original task and everything that’s happened since. Long-term memory, typically a vector database plus retrieval, scales past the context window, but trades exactness for approximation: embedding similarity is a proxy for relevance, not relevance itself, and a retrieval miss is silent, the agent simply never sees the fact it needed. Episodic memory summarizes past task attempts into compact text, bounding cost, at the price of specific detail that summarization necessarily discards.

The genuinely interesting question isn’t which of these three to use, it’s what an agent does with a failure once it’s noticed one, and here there are two real, substantially different answers already deployed at real scale. Reflexion (Shinn et al., NeurIPS 2023) is the cheap one: after a failed attempt, a language model generates a natural-language self-critique, “the previous attempt failed because X, try Y instead,” which gets prepended to context on the next attempt. No gradient update happens anywhere. Learning is entirely mediated by what fits in the prompt, which is also its real limit, the lesson lives only as long as that context does, and evaporates the moment the episode ends or the buffer rolls over, unless something else persists it. The real number attached to this approach is worth having exactly because it’s surprising for something this cheap: 91% pass@1 on HumanEval, surpassing GPT-4’s zero-shot 80% on the same benchmark, purely from verbal self-reflection with no fine-tuning at all.

Poolside’s RLCEF (Reinforcement Learning from Code Execution Feedback) is the expensive, permanent answer to the same underlying problem. Instead of a self-critique living in a prompt, real execution outcomes, does the code actually run, does it pass the tests, become a reward signal that updates the model’s weights via the exact actor-learner reinforcement learning infrastructure derived in full in the previous post: an actor generating attempts, execution nodes scoring them, a training loop consuming that signal. This is disclosed as running at real, substantial scale, roughly one million container images on AWS, sustaining on the order of 10,000 code executions per minute, specifically so the resulting model improvement is permanent and doesn’t need to be re-derived by every fresh agent session the way a Reflexion-style critique does. The tradeoff is exactly what you’d expect once you’ve seen the memory-and-compute cost of RL infrastructure once already in this series: Reflexion costs nothing beyond an extra generation call and forgets everything outside its context; RLCEF costs a real production RL cluster and never forgets, because the lesson gets baked into the weights instead of the prompt.

Standardizing the interface: MCP, and the cost of standardizing it#

Every agent needs some way to describe available tools to the model, and until recently that was a bespoke JSON schema per framework, per company, sometimes per project. Anthropic released the Model Context Protocol (MCP) on November 25, 2024: a standard way for an application to expose tools, data, and prompts to any MCP-compatible model, instead of every integration being reinvented per framework. The real adoption curve is unusually fast for an open protocol: roughly 100,000 SDK downloads in November 2024, over 8 million by April 2025, and OpenAI and Google DeepMind both adopted it as native support during 2025. By December 2025, Anthropic reported over 97 million monthly SDK downloads across all languages and donated MCP’s governance to a newly formed Agentic AI Foundation under the Linux Foundation, the kind of move a company only makes once a protocol has stopped being a competitive asset and started being shared infrastructure.

Standardization won for the reason standards usually win: an MCP server written once works with every MCP-compatible agent, instead of once per framework. But a shared protocol is also a shared attack surface, and that cost shows up directly in the next section, in an incident that specifically used MCP as its delivery mechanism.

The lethal trifecta: why this failure mode can’t be prompted away#

Simon Willison’s framing, from June 2025, names the exact combination of capabilities that turns prompt injection from an annoyance into a real security failure: access to private data, exposure to untrusted content, and the ability to communicate externally. Any one or two of these alone are usually fine. A read-only agent with no external communication path can be manipulated but can’t exfiltrate anything. An agent that only reads trusted, curated content has no injection vector in the first place. It’s the combination of all three at once that’s dangerous, and the danger is structural, not a bug: language itself is the attack surface, because there’s no clean, reliable way for a model to distinguish “the user’s actual instruction” from “text that happens to look like an instruction, sitting inside a document the model was asked to read.”

The concrete, real incident that made this vivid: a GitHub MCP integration combined all three capabilities in a single tool, it could read public issues (untrusted content an attacker fully controls, since anyone can file a public issue), access private repository contents (the private data), and create pull requests (the external communication channel). An attacker files a public issue containing text crafted to look like an instruction; an agent using that MCP server to triage issues reads it, and the poisoned content persuades the agent to open a pull request that leaks private repository data, no vulnerability in any traditional sense, no exploit against a parser, just an agent doing exactly what its instructions said to do, except one of those “instructions” came from an attacker instead of the user.

Anthropic’s own release of Computer Use, in public beta on October 22, 2024, states the same risk directly rather than burying it: content on a webpage or inside an image can override the user’s actual instructions, and Claude will sometimes follow it. Their own recommended mitigation is architectural, not a prompting trick: run the agent in a VM or container with minimal privileges, and isolate it from sensitive data and destructive actions rather than trusting the model to always resist injected instructions. That’s the honest shape of the fix for a lethal-trifecta risk: you don’t train it away or prompt it away, because the vulnerability lives in the combination of capabilities you granted, not in a specific flaw in the model’s judgment. You remove one leg of the trifecta, usually by scoping down what private data or what external channel a given agent instance actually has access to, or you accept the residual risk explicitly.

When the demo isn’t the product: Devin’s real numbers#

Cognition Labs’ Devin launch in March 2024 reported 13.86% resolved on SWE-bench, against a prior best of 1.96% unassisted and 4.80% assisted, and the launch demo, shipping a benchmark, debugging a model, completing a real Upwork job, helped the company raise more than $175 million. An independent review by Answer.AI in October 2024 tested Devin against 20 real-world tasks and reported it fully completed 3, with several runs producing malformed pull requests. Cognition’s own later release, Devin 2.0, reports a considerably more conservative and specifically-qualified 45.8% on SWE-bench Verified. Stating this plainly rather than picking a side: a striking launch number and a much rougher independent, real-task review can both be real at once, describing different things, a curated benchmark result and unscripted real-world task completion are not the same measurement, and the same lesson this series already derived for eval methodology in general applies directly here: a single headline number, however real, is never a substitute for someone else running the same claim against tasks they chose, not you.

Approach	Learns from failure via	Cost	Persists across sessions?
ReAct	In-context reasoning, same episode	One extra generation per step	No
Reflexion	Verbal self-critique, episodic memory	One extra generation per failed attempt	Only within the memory buffer’s lifetime
Poolside RLCEF	RL weight update from execution reward	Actor-learner cluster, real production infra	Yes, permanently, baked into weights
Raw shell access	N/A (interface, not a learning method)	Cheapest to build, most misuse per step	N/A
SWE-agent ACI	N/A (interface, not a learning method)	More design work upfront	N/A

Common mistakes#

Assuming a better underlying model is the highest-leverage fix for a struggling agent: SWE-agent’s own real numbers show interface design, simplicity and efficiency of the command set specifically, produced a bigger, better-documented jump than most model upgrades do, on the same model class competitors had access to.

Treating Reflexion and RL-based approaches like RLCEF as competing solutions to the same problem: one is a cheap, session-scoped patch that costs a generation call and forgets; the other is an expensive, permanent capability upgrade that costs real actor-learner infrastructure. Picking between them is a cost-versus-permanence decision, not a quality ranking.

Believing prompt injection is a training problem that better instruction-following will eventually solve: the lethal trifecta is a structural property of which capabilities an agent has simultaneously, not a flaw in how well it follows instructions, and Anthropic’s own Computer Use documentation says exactly this about their own model.

Try it yourself#

Beginner. Using the react_loop function above, trace through what happens when tools.execute raises a ToolError on three consecutive steps for the same tool call with unchanged arguments. Identify the exact point at which this becomes the “unrecovered tool error” loop-and-timeout failure mode, and write the one-line fix.

Intermediate. A prompt injection scenario: an agent has read access to a company’s private Slack, reads web pages to answer questions, and can post messages back to Slack. Identify which of the three lethal-trifecta capabilities are present, and propose the smallest change (removing or scoping one capability) that breaks the trifecta while preserving as much of the agent’s usefulness as possible.

Advanced. SWE-agent’s ACI reports 12.5% pass@1 on SWE-bench with a constrained, purpose-built command interface. Design an experiment that would isolate how much of that number comes from the interface versus the underlying model. What’s the confound if you only compare against a raw-bash baseline using the same model, and how would you rule it out using the eval-rigor practices (confidence intervals, sample size) already derived in this series?

What this takes to be frontier-job-ready#

The technical axis is stated directly in OpenAI’s own listing for Researcher, Agentic Post-Training: “Own end-to-end research and engineering projects that improve the final post-training of OpenAI’s agentic models. Develop horizontal model improvements across factuality, instruction following, tool/function calling, multi-agent behavior, reasoning-effort calibration.” Every one of those nouns, tool/function calling, multi-agent behavior, has a literal referent somewhere in this post. OpenAI’s separate Frontier Evals & Environments listing names “coding agents, tool-using agents” explicitly as valid hands-on experience, on equal footing with more traditional RL and post-training work, not as a niche specialization.

The research-depth axis shows up at DeepMind, whose Research Engineer, Multi-Agent Learning role requires a PhD and “deep expertise in multi-agent reinforcement learning, algorithmic game theory, or computational economics,” a meaningfully higher theoretical bar than the single-agent tool-use work this post covers, worth knowing as a distinct track rather than assuming all “agent” roles are the same job.

The production-scale axis is Poolside’s own stated mission, listed verbatim in their materials as “AGI through software development + RLCEF”, not an abstract aspiration but a specific, named technique run at the real, disclosed scale described above. Kimi K2’s own technical positioning names “300-step tool calling” as a supported capability, a concrete number for how far “agent” now means “sustains a coherent, correct multi-turn tool-use trajectory,” not “calls one API successfully.”

---

The one-sentence version: an agent is a model in a loop, but SWE-agent’s own real numbers show the interface you build around that loop, not the reasoning strategy inside it, is where a large, measurable share of the performance actually comes from; Reflexion’s free-but-forgetting verbal self-critique and Poolside’s expensive-but-permanent RLCEF are two real, different answers to how an agent improves after failure, at two completely different points on the cost-versus-persistence curve; and the same three capabilities that make an agent worth deploying, private data, untrusted content, external communication, are exactly the combination Simon Willison named as unfixable by prompting, proven out in a real GitHub MCP exploit rather than a hypothetical. None of this replaces the eval discipline this series already built: Devin’s own real numbers are the clearest reminder available that a launch demo and an independent review can both be true and still describe two different things.