Models, tokens and context windows
Three pieces of vocabulary that explain most of what people find mysterious about cost, length limits and why long conversations go strange.
What you will be able to do
Use the words token, context window and model correctly, and diagnose a length problem when you meet one.
Before you start
- No technical background needed.
Tokens
Models do not read characters or words. They read tokens — chunks that are often a word, sometimes part of one, sometimes punctuation. A rough English rule of thumb is that a token averages around three-quarters of a word, but it varies with the language and the text.
Tokens are the unit of both cost and length. Every limit you meet is expressed in them.
The context window
The context window is everything the model can attend to in one go: your instructions, the conversation so far, any documents you attached, and the response being generated. When you exceed it, something has to be dropped or summarised — and what gets dropped is usually the oldest material, which is often your original brief.
That is the mechanism behind the familiar experience of a long conversation losing the plot. Nothing has degraded. The instructions have simply left the room.
Models are not interchangeable
Providers ship several models at different capability and cost points, and the line-up changes. Rather than memorising names, learn to check the current documentation and pick on the axes that matter: capability on your task, context size, speed and cost.
Use a smaller model for high-volume simple work and a larger one where the reasoning is the point. Paying top rates to reformat a list is a habit worth breaking.
What good looks like
- You put the important instruction near the end of a long prompt, not buried in the middle.
- You start a fresh conversation instead of fighting a stale one.
- You can estimate roughly whether a document will fit.
What it cannot do
- Published context sizes are maximums, not guarantees of quality across the whole window.
- Token counts differ between providers for the same text.
When it goes wrong
- The model ignores an instruction you gave earlier.
- It has likely fallen out of context. Restate it in the current message, or start again with a complete brief.
- Costs higher than expected on a repeated task.
- Check whether you are resending a large context every call, and whether a smaller model would do.
Privacy and data handling
Attachments count as context and are sent to the provider. Redact before uploading rather than after.
Sources2 referenced
- Claude developer documentation
Anthropic · official product documentation · link checked 2026-09-06
- OpenAI platform documentation
OpenAI · official product documentation · link checked 2026-09-06
Links are checked at build time. If one has moved, tell us.
Read next
What generative AI is — and is not
In ten minutes: what these systems are actually doing when they answer you, and what that makes them good and bad at.
Checked 2026-09-06
What context engineering actually means
Prompting is what you ask. Context engineering is what you put in the room before you ask it. The second is where most of the quality lives.
Checked 2026-09-06