What I learnt about context from Matt Pocock's AI coding crash course: what it is, why it fills up, and six things I do to keep it small.
A note before you read. This field changes quickly. What follows is based on what was available when I finished Matt Pocock's AI Coding Crash Course. Models, limits and prices will change, so treat the numbers as a guide and check the current details.
What context is
Context is everything the agent can see when it answers: your messages, its replies, the instructions for using Git, the tool calls and their results, the files it has read. That text is turned into tokens, and the model predicts the next piece of text from all of it.
Two things follow:
- There is a maximum. Every model can only take so much.
- You pay for it. Everything you send is billed, and that includes the whole conversation so far.
Context also never starts at zero. The harness wraps the model with a system prompt: which tools are available, what the model should do and what role it should play. Part of the budget is spent before you type anything, and tip 5 below is about shrinking that part.
Smart zone and dumb zone
You might think the model can reason about everything in its context equally well. It can't.
Try it with your own head first. Meet one person at a party and you remember their name. Meet two and you remember both names, plus how they relate: they work together, or they arrived as a pair. Meet ten and there are 45 pairs of people to keep straight. Meet a hundred and there are 4,950. The names aren't the hard part. The connections are.
A model's version goes like this: one word, and it only has to hold that word. Add a second, and it has to hold both and the relationship between them. Every token you add brings more relationships to track. That isn't exactly how models work inside, but it's a good enough picture. The more it has to track, the worse it does. This is called attention degradation.
The course splits this into two zones:
- Smart zone, early in the context. The agent does well at planning, building complex software and making strategic decisions.
- Dumb zone, as the context fills. It struggles even with simple things like writing a file, closing an issue or writing a basic spec.
In the course the dumb zone began at around 150k tokens. You still get results there, but not the best ones, and everything you send carries a huge amount of text, so it costs more too. Work done in the dumb zone tends to be lower quality, and you spend more tokens cleaning it up.
In practice, I try to stay in the smart zone for most tasks. I only stop worrying about it for the ones that don't need peak performance. To see where I am, I run the /statusline command in Claude Code, which shows the context size in my terminal window, so I can watch it fill up before it gets to the dumb zone.
How the agent talks to the model
An AI coding tool has two parts: the model, and the harness around it. The harness is the program you use. It sends a request to the model provider and gets a response back. I'll call this a model provider request.
Almost everything the agent does is a model provider request. Even the faded placeholder text in Claude Code's input box, which suggests what you might type next, is produced by sending a model provider request.
Each time you send a message, the harness sends the whole conversation so far along with it. The request gets longer every time. According to the course, a single turn can last hours and hold hundreds of requests, because the agent keeps calling tools and reading the results.
Sessions and turns
A turn is one message from you, plus everything the agent does until it finishes answering. A session is the context built up over several turns.
In one example from the course, the first turn took five model provider requests, the second took three and the third took four. The last request of the last turn carries the whole history so far, which is the session.
You can clear a session, compact it, or move to another one and come back later.
The model remembers nothing
The model is stateless. The harness is the part that remembers. It keeps the messages from the session and sends them again every time. Clear the session and it's all gone.
So if you want the agent to remember something beyond one session, it has to be saved in the environment, for example in a file in the repo. That's how you build a memory system.
What it costs
Each request sends input tokens (the state of the repo and what the agent knows) and gets output tokens back (what the agent writes). In the course, output tokens cost much more than input tokens.
Some input is cached. Repeated input is billed at a much lower rate, roughly ten times cheaper in the course. Much of the work of arranging the context so it caches well is the harness's job. What you control is how much goes in and whether it's relevant. A small, relevant context is better for the agent and for your wallet.
Six things I do to save context
-
Start a new session for a new task, and again when the agent starts slipping. A fresh session wipes the old conversation. Carrying yesterday's work into today's task fills the window with things that no longer matter. If the agent starts doing simple things badly, you're in the dumb zone: clear the session or compact it. Carrying on costs more and gives worse code that you then have to clean up.
-
Check what you add before you add it. Everything you add is billed again on every later request, and it uses up room in the smart zone. Before you paste a file, a log or a doc, ask whether this task needs it.
-
Match the work to the zone. Do planning and complex building early, while the agent is at its sharpest.
-
Save what matters outside the conversation. The model remembers nothing, and clearing a session loses everything. Put decisions and progress in files in the repo so the next session can read them.
-
Trim your starting context. Look at what your setup sends before you've typed a word. I did this to make more room in the smart zone, where the agent actually thinks through the problem. You may be surprised by what you find, especially if you've never looked. I switch off what I don't use:
- MCP connectors I'm not using in this session.
- Workflows, the dynamic setup that orchestrates several sub-agents. If I won't use them, they go.
- The built-in skills that come with Claude Code.
- Artifacts.
- Extra tools. The course showed that if you deny a tool in
permissions.deny, its definition is removed from the system prompt entirely. You save the tokens on every single request. - The ask-the-user-a-question tool (
AskUserQuestion).
In the course, switching all of that off took my starting context from 44k tokens down to 10.9k.
-
Make your codebase easy to explore. The model is stateless, so every fresh session has to explore your codebase to find what it needs. That is the hardest limit to work around, and the quality of the exploration decides the quality of what comes next. If the agent can't find the information it needs, it probably won't do what you want. Making the code easy to explore is one of the most important things you can do. It's why I built my project foundation skill, which is on GitHub.
Why bother
When an agent does something dumb, it's tempting to blame the model. Before I do, I check the context.
Anthropic describes context as a finite attention budget, and says the aim is the smallest set of high-signal tokens that gets you the result you want (Effective context engineering for AI agents). Chroma tested 18 models and found that every one became less reliable as the input grew, even on very simple tasks (Context Rot). Neither says the model is never the problem. Both point at the part I can control: what I put in the window.