One question, 437,000 tokens: what real agents found in our MCP server

One question. 437,000 input tokens. Not a hard question either. An agent connected to our MCP server, asked something a support engineer answers in a sentence, and worked its way there through twenty tool calls, each one dragging every earlier answer along behind it. Nothing was broken while that happened. The server answered initialize correctly, spoke the 2025-03-26 revision, returned valid JSON-RPC to everything we threw at it. All of which turned out to be beside the point. So we pointed real agents at production and watched. 18 scenarios, two vendors, a 5 dollar budget that we topped up once. This is the long version with the traces in it. There is a shorter one on our blog if you only want the conclusions. What the server is Briefly, because it shapes everything below. FoxNose stores content as collections: schema-defined records with typed fields, some of them vector indexed. The MCP server is generated from that schema and served from the same URL prefix as the REST API . Fixed catalog of seven tools regardless of how many collections exist, five read and two optional write. Two of those properties matter below. Collections are what an agent chooses between, so a badly described collection is effectively invisible. And the agent inherits exactly the rights of the API key it connects with, so there is no second allowed-tools list drifting out of sync with the first. What the harness actually is A scenario is a question in plain English, a set of tools, and a check. The checks are where we made the most mistakes, so start there. They do not look at the answer text. Model output moves between runs, and a suite that asserts on wording is a suite you quietly stop trusting. They look at the trace: which tools ran, in what order, with what arguments, which errors came back, how many tokens the whole thing burned. A check is a small predicate over the run: any_of ( no_tool_errors (), recovered_after ( " unknown_resource " , then = " search_records " ), ) That second