I measured Code Mode on my MCP server: in Claude Code it made twice the model requests and spent 44% less than calling the tools one by one; across 20 chained calls on the API, 4.7× less.
In Claude Code, with my MCP server, google-docs-mcp, Code Mode made more than twice as many requests to the model (13 versus 5) and still cost 44% less than calling the tools one by one. The reason is that tool results stay in a script variable and never enter the context.
When an agent calls the tools of an MCP server one by one, every result goes back to the model in full, even if it only needs to count three things in it. Code Mode changes that: the agent writes a script that calls the tools, filters the results in code and returns only what matters. This week Claude Code shipped it as an experiment, and the headlines of “MCP is dead” and “saves 98% of tokens” are back. Instead of repeating those figures, I measured them.
The short version:
- With 20 chained calls, it wins by a lot. 4.7 times cheaper than classic mode on the API. In Claude Code, with three calls to my real server, 44% less than calling the tools one by one.
- With a single call, it doesn’t pay off. It takes twice as long as classic mode (14 s versus 7 s) and tool search is cheaper ($0.006 versus $0.012).
- With hundreds of tools, it isn’t what helps. What helps is tool search.
- Tokens are not money. 118,000 tokens of definitions cost $0.016 with a warm cache.
Everything is reproducible: the benchmark is in yeyo11/mcp-code-mode-bench, with the server, the tasks, the logs of every run and the table.
What is Code Mode, and why are there three?
Instead of giving the model one tool per operation and waiting for it to call them one at a time, you give it a single tool that runs code, and your MCP tools become functions that code can call. Intermediate results stay in script variables and never pass through the model’s context. The idea predates the name: the CodeAct paper (February 2024) already used executable code as the agent’s action, and smolagents’ CodeAgent brought it to open source in December 2024. Cloudflare coined the name in September 2025.
Today “Code Mode” means three different things, and it’s worth not mixing them up:
- Cloudflare Code Mode, on the server. It turns MCP tools into a TypeScript API and runs the model’s code in a V8 isolate. In February it applied it to its whole API: more than 2,500 endpoints behind two tools,
searchandexecute. - Programmatic tool calling, in the Claude API. Claude writes Python in the code execution container and calls your tools from there. It went into beta on November 24, 2025 and has needed no beta header since February 17, 2026. Anthropic explains the pattern in Code execution with MCP.
- Claude Code’s Code Mode, on the client. Claude writes JavaScript that calls Bash, Read, Edit and your MCP tools through a tool called
REPL. It arrived as an experiment in version 2.1.323, on November 12, 2026; that’s why it’s being talked about now.
What do the circulating numbers mean, in context?
Before measuring, I checked where each number being repeated these days comes from:
- From 150,000 to 2,000 tokens (98.7%) is from Anthropic, and it refers to loading tool definitions on demand, not to the Google Drive to Salesforce example in the same article. They don’t publish how they measured it.
- From 1.17 million to about 1,000 tokens is from Cloudflare. The starting point is hypothetical: a server with one tool per endpoint that never existed.
- The “independent” 78% is an AIMultiple test with 2 queries, GPT-4.1 and Bright Data’s MCP server (63 tools). The page itself states that Bright Data subscribes to their benchmarks. And most of the saving comes from not loading 63 definitions, not from filtering in code.
- The 37% is programmatic tool calling on the API (from 43,588 to 27,297 tokens on average), not Claude Code’s Code Mode. Anthropic has not published Code Mode figures for Claude Code.
How did I measure it?
For the API I built a mock MCP server with the same 6 tools as google-docs-mcp (list_documents, list_comments, get_comments_summary, get_comment, get_document, get_auth_status), the same schemas and the same output format. It answers with made-up data: 20 documents in the “Q4 planning” folder of a fictional team, with 300 comments, reply threads, reopened and deleted comments. There is no login, and “today” is always November 20, 2026, so the correct answer is computed in advance and each run is graded by code, not by me.
Three tasks:
| Task | What it asks | Shape |
|---|---|---|
| T1 | Across all documents, which open comments have gone more than 7 days without a reply, grouped by document and counted by author | 20 chained calls to list_comments, one per document, with large results and a small answer (21 counting the initial list_documents) |
| T2 | How many open comments the document “Budget 2027” has | One call: the case where Code Mode shouldn’t pay off |
| T3 | T2 with 200 filler tools in the catalog (206 in total) | What the definitions alone cost |
And three modes, with the same model (Sonnet 5.5), the same effort (medium) and the same prompt:
- A. Classic: every result goes back to the model.
- B. Classic with tool search: all tools with
defer_loading: trueexceptlist_documents, and the model searches for the ones it needs. - C. Programmatic tool calling:
code_execution_20260120andallowed_callerson every tool.
For each run I add up the tokens of all the requests in the loop (uncached input, cache writes and cache reads), the output tokens, the number of requests, the time and the cost using Anthropic’s price table. Five repetitions per task and mode, 45 runs in total, all correct. They cost me $5.54.
Then I repeated the comparison in Claude Code with my real server, classic against Code Mode.
What were the results on the API?
Medians of 5 runs with a warm cache:
| Task | A. Classic | B. Tool search | C. Programmatic tool calling |
|---|---|---|---|
| T1, 20 chained calls | $0.281 · 77k input · 11.3k output · 79 s | $0.283 · 80k input · 75 s | $0.060 · 44k input · 3.8k output · 57 s |
| T2, one call | $0.018 · 7 s | $0.006 · 7 s | $0.012 · 14 s |
| T3, one call, 206 tools | $0.016 · 118k input | $0.005 · 11k input | $0.029 · 161k input |

With a cold cache (one run of each): in T1, $0.288 for classic versus $0.083 with code; in T3, $0.118 for classic, $0.020 with tool search and $0.110 with code.
A T1 detail that surprises: classic mode makes only 3 requests to the model, because the model launches the 20 list_comments calls in parallel in a single turn. Programmatic tool calling makes 24, because every call the script makes goes back to your loop for you to execute it. It still costs almost five times less than classic: those results don’t enter the context or get billed as input, and the model writes 3,800 tokens instead of 11,300.
What happens with my real server in Claude Code?
The task, over three public design documents with comments (two from Kubernetes and one from OpenTelemetry): in each document, which open threads have a reply from someone else as the last message, that is, where the comment’s author owes a reply. The correct answer is 4, 8 and 0, and every run got it.
Claude Code 2.1.330, Sonnet 5.5, interactive sessions, 3 repetitions of each mode interleaved:
| Mode | Model requests | Input tokens (written to cache) | Output | Time | Estimated cost |
|---|---|---|---|---|---|
| Classic | 5 (4–6) | 256k (about 38k) | 2,251 | 21 s | $0.191 |
| Code Mode | 13 (12–14) | 170k (about 18k) | 1,739 | 21 s | $0.107 |
Medians. The cost is an estimate at API prices: I ran it on my subscription, where you don’t pay per run but spend quota.
Compared with classic mode, Code Mode makes more than twice as many requests to the model, takes the same time and still comes out 44% cheaper. The difference is in the cache: Claude Code rereads its system prompt on every request, and those reads are cheap. What costs is writing to the cache, and in Code Mode the results of the three MCP calls never reach the context, so half as much is written.
With Ctrl+O you can see the code Claude writes. This is the script that does the work (I formatted it and shortened the identifiers):
const ids = ["12F1YXsv…", "1q4a8uB_…", "1y63W66E…"];
const out = {};
for (const id of ids) {
out[id] = await tools["mcp__google-docs__list_comments"]({
document: id, status: "open", format: "json", max_results: 100,
});
}
globalThis.out = out;

The three responses stay in globalThis.out, and Claude only sees the length of each one. In the following scripts it filters in code the threads whose last reply isn’t from the author and returns the lists of identifiers. The model never reads the comments.
To turn it on you need Claude Code 2.1.323 or later, an interactive terminal session and this variable:
CLAUDE_CODE_EXPERIMENTAL_CODE_MODE=1 claude
The first time it asks whether you want to enable it; you answer yes and restart. It doesn’t work with claude -p: the REPL tool doesn’t appear. And if you launch claude from inside another Claude Code session, the process inherits CLAUDE_* variables and tells you Code Mode isn’t available in that kind of session; start it without them.
What do the numbers say?
It wins when there are many chained calls whose results only need filtering. Not by making fewer requests to the model, because it makes more, but because the intermediate results don’t enter the context and the model writes much less. In T1, with 20 chained calls, 4.7 times cheaper than classic mode on the API; with my server in Claude Code, 44% less than calling the tools one by one.
With a single call, it doesn’t pay off. In T2 it takes twice as long as classic mode (14 s versus 7 s) and tool search is cheaper ($0.006 versus $0.012). Writing the script and starting the container has a fixed cost that a small call doesn’t recover.
With hundreds of tools, what works is tool search. Programmatic tool calling, as the API offers it, still loads all the definitions: in T3 it reads 161k tokens. Tool search brings that down to 11k and is 6 times cheaper than classic mode with a cold cache and 3 with a warm one. Cloudflare’s two tools, search and execute, work because they combine both ideas: search for what to call, then call it from code. Saying Code Mode solves 2,500 endpoints mixes them up. Claude Code, by the way, already defers MCP tool definitions behind its own search, so you have that part covered in classic mode too.
Tokens are not money. In T3 classic mode reads 118,000 tokens of definitions and costs $0.016, because almost all of it is cache reads, which on Sonnet 5.5 cost 5% of the input price. The bill depends on what’s written to the cache and what the model generates, not on the total token count.
One run proves nothing. The same classic prompt in Claude Code went from 4 to 6 requests to the model and from 189k to 327k input tokens between repetitions. With claude -p, two identical runs gave 5 and 11 turns. If you compare modes, repeat.
What does this measurement cover?
A mock server with made-up data, so anyone can reproduce it, and a comparison with my real server. One model (Sonnet 5.5) at one effort (medium). Five repetitions per cell on the API and three per mode in Claude Code: enough to see the direction, not to argue over decimals. The mock server answers with no latency; with a real one, which waits on Google’s API, every extra round trip weighs more on time, in all modes. Cloudflare Code Mode is left out: it’s a different stack and deploying it requires the paid Workers plan.
On security, in Claude Code every call still goes through your deny and ask rules and through the PreToolUse hooks. In auto mode, however, the classifier reviews the code and not each call, and it doesn’t see the values computed when it runs. And according to the documentation, the JavaScript and its file, web and MCP calls run outside the sandbox; only the shell commands it launches run inside.
How do I repeat it myself?
Node 20 or later. The API modes need ANTHROPIC_API_KEY; the rest is free.
git clone https://github.com/yeyo11/mcp-code-mode-bench
cd mcp-code-mode-bench
npm install
npm test
# The exact request a mode would send, without sending it or needing a key
npm run bench -- api --dry-run --mode C --task T2
# The three API modes interleaved, 5 repetitions, with a spending cap
npm run bench -- api --mode A,B,C --task T1,T2,T3 --reps 5 --cap 9
npm run report
The cap (--cap, in dollars) stops the whole run as soon as it’s reached. --model changes the model and --padding the size of the catalog. The steps to measure Claude Code’s Code Mode from an interactive session are in docs/claude-code-code-mode.md.
If you want to hear about the next experiment of this kind, subscribe to the newsletter below.
Your turn
This is what I’m left with, and I’m more interested in your experience than mine:
- Where should Code Mode live, in the client or in the MCP server? Cloudflare puts it in the server; Claude Code, in the client. In the client, each server doesn’t have to build its own sandbox; in the server, you don’t depend on your agent supporting it.
- Is it worth approving a program instead of each call? Especially in auto mode, where nobody sees the values computed when it runs.
- From how many tools or how many calls does it pay off for you? In my tasks, not with one call; with 20 chained, yes.
And if you have another model at hand, run the benchmark with --model and post your numbers in an issue on the repository. If yours differ from mine, even better: that’s exactly what I want to see.

