Context Windows, Tokens, and Why Your Chat "Forgets"

You've had this happen. A long, productive chat with an AI, and then somewhere near the end it starts contradicting itself, forgetting what you told it, or repeating things you settled twenty messages ago. It feels like the model got dumber.

It didn't. It ran out of desk space. Here's the clean version.

Tokens, in one sentence

A token is a small chunk of text — roughly a word or a piece of a word — and it's the unit the model actually reads and writes in. When people talk about how much a model can "handle," they're counting tokens. That's the whole definition. You don't need to think in tokens day to day; you just need to know they're the currency, and there's a limit to how many the model can hold at once.

The context window is the size of the desk

Here's the metaphor that makes everything click. A context window is the amount of information the model can keep in view while it works. Picture it as the size of its desk.

Everything in your conversation — your messages, its replies, any documents you've pasted — sits on that desk. While there's room, the model can see all of it and work with all of it. But the desk is a fixed size. As the conversation grows, papers pile up. Eventually, to make room for new pages, the oldest ones slide off the edge.

That's what "forgetting" is. The model didn't lose intelligence. The information you're referring to has simply slid off the desk, out of view. It can only work with what's still on the surface.

Why long chats drift

Now the earlier symptoms make sense. In a very long conversation, the instructions you gave at the start may no longer be on the desk. The model is working only from the recent pages, so it drifts from your original intent — not out of carelessness, but because the original intent is no longer in front of it.

This is also why pasting a huge document and then chatting for an hour eventually goes sideways. The document filled the desk. Add enough conversation on top and parts of the document quietly fall off.

Three habits that fix ninety percent of this

You don't need to understand the machinery any deeper than the desk. You just need three habits.

Restate what matters when it matters. If a key instruction was many messages ago and the model seems to have lost it, don't sigh — just say it again. You're putting the important page back on top of the pile.

Start fresh for a new task. When you switch to something genuinely different, open a new conversation rather than continuing a long one. A clear desk works better than a cluttered one. This single habit prevents most "it forgot" frustration.

Summarize, then continue. On a long, valuable chat, ask the model to summarize the key decisions so far, then carry that summary into a fresh conversation. You've kept the conclusions and thrown away the clutter that was crowding them out.

"But newer models have huge desks now"

You may have heard that the latest models have enormous context windows — that the desk is now the size of a dining table, big enough to hold entire books. That's true, and it helps. But it doesn't make the habits obsolete, for two reasons worth knowing.

First, a bigger desk still fills up. If you paste a book and have a two-hour conversation about it, you can still push the early pages off the edge. The limit moved; it didn't disappear.

Second, and less obvious: even when everything fits on the desk, models pay more attention to what's near the top and bottom of the pile than to what's buried in the middle. So a critical instruction sitting in the center of a very long conversation can be technically present but functionally ignored — on the desk, but under a stack of other papers. This is why restating what matters still works even on the newest, roomiest models. You're not adding information it lost. You're moving the important page back to where it's actually looked at.

So treat the bigger window as breathing room, not a reason to abandon the habits. A larger desk you keep tidy will always beat a huge one you let bury the thing you need.

How to tell when you've hit the limit

The useful skill is noticing the drift early, before it wastes your afternoon. There are three reliable tells, and once you know them you'll catch the problem the moment it starts.

The first is contradiction: the model states something now that conflicts with what it agreed to earlier. That's the clearest sign an early page has slid off the desk. The second is repetition: it starts re-suggesting ideas you already discussed and dismissed, because from its point of view, that discussion is no longer there. The third is a subtle loss of your instructions — the tone quietly shifts, the format you asked for erodes, the constraints loosen. Nothing dramatic, just a slow drift back toward generic.

When you spot any of these, don't fight it by re-explaining everything in frustration. Do one of the three habits: restate the single most important instruction, or summarize and start fresh. Treat the tells as a dashboard light, not a breakdown. They're simply telling you the desk is full and it's time to tidy it.

The reader who notices drift early loses a sentence. The one who ignores it loses an hour and blames the tool.

The takeaway

Understanding the limit is how you stop fighting it. The model isn't unreliable and it isn't getting tired. It has a desk of a fixed size, and your job is simply to keep the pages that matter on top.

Once you see it that way, the "forgetting" stops feeling like a flaw and starts feeling like what it is: a very ordinary constraint, easily managed, now that you know it's there.