LLMs, Tokens, and Context Windows

What the heck is an LLM?

LLM stands for Large Language Model. You can think of it as the "brain" powering chatbots like ChatGPT. It's "Large" because it's trained on billions of pages of text, "Language" because it understands human speech, and a "Model" because it's a massive mathematical representation of how language works.

Breaking Down Tokens

If you've ever played Scrabble, you know that words are made of smaller pieces. To an LLM, text isn't read word-by-word; it's read in chunks called Tokens. Think of a token like a syllable or a piece of a word. A general rule of thumb is that 1 token is about 3/4 of a typical English word. For example, "hamburger" might be broken down into three tokens: "ham", "bur", and "ger".

The Context Window: The AI's Short-Term Memory

Imagine a bartender on a busy Friday night. You walk up and order a Margarita, two Beers, and a Coke. They remember that easily. But what if you rattle off 50 complicated drink orders in 10 seconds? The bartender will probably forget the first few drinks you mentioned.

This is exactly what a Context Window is. It's the AI's short-term memory limit for a single conversation. If an AI has a context window of 8,000 tokens, it can "remember" roughly a 10-page document at one time. If you paste in a 100-page book and ask it a question about page 1, it might have already "forgotten" it because that text got pushed out of its context window, just like the bartender forgetting your first drink!