The Context Window Is a Budget You Are Already Overspending

The blended price of a million tokens decreased by about two-thirds from the beginning of 2025 to the beginning of 2026. During that same time, the FinOps Foundation determined that 73% of enterprises exceeded their AI budget projections. That is a number that should bother you.

Prices are down two-thirds, budgets are blown regardless. That only happens when consumption grows faster than anyone modeled, and with agents, it did. Gartner estimates agentic workloads to consume 5 to 30 times the tokens of a single-turn chatbot interaction. The chatbot you piloted and the agent you shipped are not the same product, financially speaking.

Over the past couple of years, I have built and secured agent platforms. My repeated conclusion has been uncomfortable for those who prefer buying their way out of problems. Cheaper models do not fix it. The context window has to be treated as an engineered resource, the way we eventually learned to treat memory, bandwidth, and cloud spend. Most teams have never researched what is truly held in the agents' context windows. When they do, the reaction is usually some version of "wait, what is all this?"

What is actually in there

The prompt you typed is a rounding error. Most of the agents' requests consist of multiple layers that arrived without anyone deciding they should.

There are definitions of tools. When your agent uses MCP (Model Context Protocol), every connected server injects the schema for every tool it offers, on every request, even if the agent never calls them. There are standing instructions, system prompts, context documents of the repositories, policy documents, the AGENTS.md file that has not been accessed in the last six months, and chat records. The history of tool use silently accumulates, recording each invocation failure, each retry of an invocation, every read of a stale file, and every paragraph generated by the model three turns ago. From my experience, tool output is the heaviest and least scrutinized layer: unprocessed logs, complete JSON dumps with the same field repeated hundreds of times, and complete documents when only ten lines are needed.

Production agent's five layers of context windows, and where each leaks.

Every turn, the model has to reprocess every window. This comes with a cost, but it is not only a budget problem. Chroma's Context Rot research revealed that all major models, regardless of advertised window size, exhibit decreased performance as the volume of input data increases. An overstuffed context window is not a passive container with some waste in it. It actively increases the likelihood that the model will be ineffective at completing the assigned task, which drives retries and more token usage. This explains the budget graph with a consistent increase along the x and y axes, even if the cost per token drops.

Now, the focus should not be answering the question 'how do I make my prompt shorter.' Instead, it should be focused on identifying the layer generating ongoing costs and determining whether that layer is justifying its investment through task completion. Layer by layer, this is what the existing data supports.

Tool schemas are the new dependency bloat

The clearest numbers in this whole area come from the interface you pick. Scalekit compared, with the same model, MCP versus standard CLI for the same task 75 times. MCP takes between 4 and 32 times more tokens per operation. For the simple task of identifying the programming language in a repository, CLI costs 1,365 tokens while MCP costs 44,026 tokens. Almost all the difference in costs is attributed to the schema, with the difference being dozens of tool definitions that get auto-injected into the conversation so the agent can use one of them.

This does not mean that MCP is bad. Typed schemas do help mitigate the issue of poorly formed tool calls, and MCP is the appropriate choice if you have specific needs around auth, governance, multi-tenancy, or remote structure. What I am saying is that the tool surface is a dependency and we should evaluate it as such. In the same benchmark, an 800-token cheat sheet on CLI usage reduced the agent's tool use and latency by almost a third. 800 tokens of insight appears to offer more value than tens of thousands of schema tokens. The approach that continues to dominate is progressive disclosure, where the model presents a brief description and fully loads the complete workflow only when it is necessary. The MCP ecosystem is also approaching deferred tool loading.

My default now is the narrowest interface that reliably does the job. CLI if the model already knows the tool. A minimal skill file if the workflow is procedural, and repeatable. MCP if remote structuring, server-side filtering, or governance is needed. And an actual audit, quarterly, of which connected tools have gone unused for weeks.

Your context file is probably hurting you

I understand how hard this is. ETH Zurich researchers examined the practice of context tailoring coding agents with repository context files like AGENTS.md, across multiple agents and models. In general, context files reduced the chances of task success relative to no context at all, while increasing inference cost by 20% or more. Generated files were of lower quality than human written ones, but even developer committed files pushed agents into more expansive and costly exploration. The authors concluded that context files should limit the number of specifications to a few.

The moral of the story is to not remove the file. It's like, with permanent instructions, you are carrying a risk with every request, so they need to be weighted: does this instruction change the behavior of the model for the most part? Constructive commands, tight rules, one exception that trips everyone, advanced document reference. That's pretty much the file. Essays on architecture should be elsewhere, they should be stored for on demand access. Permanent instruction files are a cost and an attention tax. That is exactly what they are.

History is where good sessions go to die

Long sessions with agents degrade in a consistent manner. Windows fill with dead branches: the three write attempts that failed before the fourth succeeded, the wide search that found two hundred irrelevant hits before the correct file surfaced, the complete read of a file that was subsequently edited twice. All this information does not contribute to the next decision. Everything is processed again, paid for, and, even worse, attended to.

Two habits mitigate most of this. Phase separation is the first: do the exploration, then extract and condense the relevant portions into a small decision record, and start implementation from the record instead of bringing the exploration along. Spec-driven workflows like Spec Kit and OpenSpec make this formal, but a disciplined handwritten handoff is equally sufficient. The second tactic is the deliberately fresh start. While it may seem like you are losing progress when you reset a session with a compact summary of decisions, touched files, and open questions, it actually helps you regain momentum. Knowing what should survive the reset is critical: current intent, acceptance criteria, unresolved decisions, and any supporting evidence for a justification of a change. Prune those and you have summarized your way into a wall.

Shape the output, because output becomes input

Two additional levers, and then the element that actually interests me.

Tool output deserves a contract. Do not let raw terminal spew, complete logs, or unfiltered API responses sit in the window. For large outputs, write them to a file, return a file path and a brief summary. Request server-side filtering prior to return. For large homogeneous record sets, compact tabular formats like TOON reduce tokens by 30 to 60 percent on repetitive JSON where the same keys are repeated for each row, though the published generation benchmarks say to keep JSON for model output and treat TOON strictly as an input optimization.

Model output also deserves a contract, since every paragraph the model generates becomes input for the next turn. Instead of a narrated journey, asking for modified files, test results, blockers, and next steps, compounds across a long session in a way that is easy to overlook.

The part nobody connects: this is also your attack surface

Everything above reads like a cost exercise. Certainly, there will be security concerns, although almost no one will say it publicly.

Transformers, for instance, do not separate streams by type. There is no stream for instructions vs. data. System prompts, user inputs, fetched documents, tool outputs. All of it is one stream, and the model weighs all of it. This is why OWASP has had it at the top of its list of LLM risks for three years in a row, and why prompt injection is so effective. I'm part of Workstream 4 in the Coalition for Secure AI. We published an MCP Security white paper that defines nearly 40 risks of this type concerning agents. The hostile instruction does not arrive through the front door. It arrives inside a web page the agent retrieved, a document it fetched, or a tool result it accepted without question. Over the past year, there has been substantial evidence, including a malicious MCP server (postmark-mcp) that had 15 clean releases prior to the addition of an exfiltration line, a CVSS score of 9.6 in the core infrastructure of the MCP, and a backdoored release of a widely used LLM gateway that got tens of thousands of downloads within a three-hour window.

The supply chain risk concerning the tool layer is substantial, and I categorized it as a structured entry, tool registry poisoning, in the coalition's secure-AI tooling project, which was accepted and divided between provenance and runtime-tampering risk.

Considering the layers, the adversary may only penetrate the most opaque layer, that is tool output and external schemas. What is most precious to you is hence most hazardous, and only one discipline tackles this.

Security is that discipline. We don't give service root because it might need it someday; we practice least privilege. The agents require the sibling rule, which I have dubbed the least-context principle: admit to the context window only the tokens with a defensible reason to be there, from named sources, in the smallest faithful form, for the shortest useful time. Restrict tool surfaces because schemas are spend and unvetted capability. Compress and filter tool output because verbosity is spend and an injection vector. Prune dead history because stale context is spend, and that is where poisoned instructions persist.

This week, do the following: from your busiest agent, dump one real production query and annotate each span by layer and source. Most teams I have witnessed do this tend to have as much as a third or more of the window being material that nobody selected, and that was unproductive for the task. That third is your overspend, exposure, and it's the same third.

The cheapest token is the one you never admit. It happens to be the safest one too.