Context engineering treats the LLM context window as a token budget, not a bucket. Learn to allocate instructions, retrieval, examples, and history for better output.
A client once forwarded me a "prompt" he was proud of. It was 1,900 words: it opened by begging the model to "act as a world-class senior engineer," said the same instruction three times in slightly different words, pasted a whole 400-line file for "context," and signed off with a stern "DO NOT hallucinate." The output was mediocre and inconsistent, and he wanted me to help him find the magic phrasing that would fix it.
There was no magic phrasing. The problem was never the words; it was the allocation. He was treating the context window like a bucket you fill until it's full, when it's really a budget you spend until you're broke. Every token that goes in competes with every other token for the model's attention, and most of what he'd crammed in was actively working against him. Once I reframe the work this way for people, they stop hunting for incantations and start doing the thing that actually moves the needle: deciding, on purpose, what earns a place in the window.
I've built enough LLM systems in production — coding agents, retrieval pipelines, high-volume classifiers — to say this plainly: the context window is the single most under-managed resource in most AI applications. Teams tune model choice, temperature, and top-p to death while pouring unmanaged garbage into the one input that decides whether the model can even see the answer. This post is about fixing that, and the reframe that makes the fix obvious.
The phrase "prompt engineering" makes it sound like the craft lives in the sentence. Pick the right verb, add "step by step," maybe assign a role. That framing is fine for a one-off chat where you want a limerick. It falls apart the moment you're building something real: a coding agent, a RAG pipeline, a classifier that runs a million times a day.
In those systems the "prompt" is not a sentence you write. It's a payload you assemble at runtime from many sources — system instructions, retrieved documents, prior turns, tool outputs, examples — and hand to the model in one shot. The skill isn't wording. It's assembly. What do you include, in what order, at what fidelity, given a fixed number of tokens? That's an engineering problem, and it looks a lot more like designing a cache or a query plan than like writing copy.
So I've mostly stopped saying prompt engineering. I call it context engineering, because the unit of work is the context window, not the prompt string. And the first rule of context engineering is that the window is scarce, even when the spec sheet says it isn't. A model advertised with a 200K-token window does not give you 200K tokens of usable, equally-weighted attention. It gives you a ceiling, and the useful capacity underneath that ceiling is far smaller and unevenly distributed. Confusing the two is where most teams go wrong.
At assembly time, four categories compete for space in the window. Every real prompt is some mix of these, whether you designed the mix or not:
These four are in direct competition. A token spent on a fifth redundant example is a token not spent on the one retrieved fact that would have made the answer correct. On a recent agent build, our failures were almost never "the model is dumb." They were "the model never saw the thing it needed, because the window was full of history it didn't need."
Naming the four categories pays for itself, because it turns a vague feeling of "the prompt is too long" into an actual budget line you can cut against. When an answer comes back wrong, I don't reword it. I ask which of the four starved the other three. That single question resolves more production issues than any amount of prompt-tweaking, because it points at the mechanism instead of the surface.
I run every candidate chunk through the same three questions before it's allowed into the window. Think of these as the admission test at the door of the budget.
Does this change the output? If I can delete it and the answer doesn't get worse, it was decoration. Most "context" people paste is decoration — whole files where three functions mattered, entire chat logs where one decision mattered. I test this literally: pull the chunk, re-run, compare. If nothing moves, it stays out. This is the ablation test, and it's the closest thing context engineering has to a unit test.
Is this the cheapest form of the information? The same fact can cost 2,000 tokens or 40. A raw API response is expensive; the two fields you actually need are cheap. A full document is expensive; a tight summary is cheap. A JSON blob with every field the endpoint returns is expensive; the three keys the task depends on are cheap. Most of context engineering is finding the compressed form of something you were about to paste raw.
Is it fresh enough to trust? History rots. A tool result from eight turns ago may describe a file state that no longer exists. Stale context is worse than missing context, because the model treats it as true and reasons confidently from a lie. I'd rather drop it than let it mislead. In long-running agents this is the failure mode that quietly poisons everything downstream.
Whatever survives all three goes in. Everything else gets summarized, linked, or dropped. For our coding agent, that meant we stopped shoving whole files into context and started passing a file tree plus the specific ranges the model asked for. Reads went from "the entire repo, badly" to "the four functions that matter." Same window, ten times the signal. The model didn't get smarter — it just stopped drowning.
Tokens aren't fungible. Where a token sits in the window changes how much attention it gets. Models attend hardest to the very beginning and the very end of the context and sag in the middle — the "lost in the middle" effect is real, well-documented across long-context models, and I've watched it eat instructions in production. Retrieval quality can look terrible not because the retriever failed, but because the right passage landed in the dead zone at the center of a long prompt.
So position is a lever I spend deliberately:
A concrete fix from the field: we had a formatting rule the model obeyed maybe 70% of the time. It was sitting at the top of a 6,000-token system block, buried under everything that piled in after it. We moved a one-line restatement of the rule to the very end, right before the user's request. Compliance jumped to near-perfect. Same rule, same words. Different slot. That's the whole lesson in one experiment — the words were never the variable. When an instruction gets ignored, my first move is never to make it louder; it's to move it closer to generation.
Examples are the most misunderstood line in the budget. People add them like seasoning — a couple more can't hurt, right? They can. Every example is tokens you're spending, and their job is to buy more format certainty than a plain instruction can.
Here's the reframe: a good few-shot example is compression. "Return valid JSON matching this schema, no prose, no markdown fences" is an instruction the model may or may not honor. One example showing exactly the input and exactly the desired JSON output communicates the same rule more reliably in fewer tokens, because the model pattern-matches against the shape instead of parsing a description of it.
That changes how I pick examples:
# Two examples that teach the format AND the hard cases,# in fewer tokens than a paragraph describing them.EXAMPLES = [ # normal case: establishes the shape {"input": "Ship the invoice by Friday", "output": {"action": "create_task", "due": "friday", "title": "Ship the invoice"}}, # edge case: no date -> teaches the null, not just the happy path {"input": "Look into the flaky test sometime", "output": {"action": "create_task", "due": None, "title": "Look into the flaky test"}},]One subtlety worth naming: examples also anchor tone and verbosity, not just structure. If your demonstrations are terse, the model tends to answer tersely; if they ramble, so will it. That's leverage when you use it on purpose and a leak when you don't.
If the window is a budget, I want a return-on-spend number, not a vibe. The one I actually track is crude and useful: tokens in versus outcome out. For a given task, how many input tokens did it take to get an acceptable result, and does spending more still move quality?
The shape of that curve tells me almost everything:
On one extraction pipeline running at scale, we were sending roughly 5,000 tokens of "helpful context" per call. I bisected it: cut the input in half, measured accuracy, and it came back statistically flat. Half the context was pure cost. At volume that was a real slice of the monthly bill for exactly zero quality. We'd been paying rent on tokens that did nothing, on every single call, for months. Nobody had noticed because the output was fine — it just cost twice what it needed to.
Two numbers I keep next to that ratio:
You don't need a fancy eval harness to start. A spreadsheet with fifteen representative inputs, run twice — once with the full context, once trimmed — will show you the knee inside an afternoon. Build the harness later, once you know which knob actually pays.
If you want a repeatable process rather than intuition, this is the loop I use to right-size any prompt that matters:
You converge on the real budget in a handful of iterations, and you end up with a number you can defend instead of a prompt you're afraid to touch.
The industry keeps shipping bigger context windows, and the instinct is to fill them. Resist it. A larger window raises your budget ceiling; it doesn't change the fact that attention is finite and unevenly spread. "It fits" and "it helps" are different claims, and long-context models blur them dangerously.
I've watched bigger windows make systems worse in specific, repeatable ways:
The move that consistently helps is aggressive summarization mid-conversation. When history gets long, I compress the old turns into a short running summary — decisions made, current state, open questions — and drop the raw transcript. The agent keeps its memory and loses its baggage. This is context compaction, and it's the single highest-leverage technique for keeping a long-running agent coherent. A big window is a tool for holding more relevant context, not a license to stop deciding what's relevant. The deciding is still the job.
This is roughly the pass I make on any prompt that runs more than a handful of times:
Run that pass once and you'll usually find you can cut a third of the prompt with no loss — and often a gain, because you stopped diluting the parts that mattered.