How Large Language Models work

The one-sentence version

A large language model (LLM) is a system that has learned, from a huge amount of text, to predict what piece of text is likely to come next. Everything an AI assistant does, from answering questions to writing poems, is built on that one ability, repeated very quickly.

That sounds too simple to explain all the impressive things these tools do. Let's unpack it.

Step 1: Text becomes tokens

Models don't read letters or whole words the way you do. They break text into tokens: chunks that are often a whole common word, part of a longer word, a punctuation mark, or a space plus a word.

For example, a sentence like "Prompting is unbelievably useful" might be split into pieces roughly like this (the exact split varies from model to model):

Prompt | ing | is | un | believ | ably | useful

You don't need to count tokens to write good prompts. But knowing they exist explains some odd behavior. Because the model sees chunks rather than individual letters, tasks like "How many r's are in strawberry?" or "Write a sentence of exactly 17 words" can trip it up. It isn't looking at the text the way you are.

Step 2: Predicting the next token

Here's the core trick. Given all the tokens so far, the model calculates how likely each possible next token is, picks one, adds it to the text, and then repeats the process, one token at a time, until the response is finished.

A helpful analogy is the autocomplete on your phone's keyboard, but scaled up enormously. Your phone suggests the next word based on a small amount of your typing history. An LLM makes its prediction based on patterns learned from a vast library of books, articles, websites, code, and conversations, and it considers everything in your prompt and the conversation so far.

Another analogy: imagine an extremely well-read improv performer. Give them the opening line of a scene, and they'll continue it in a way that fits, matching tone, topic, and style, because they've absorbed so many similar scenes. They aren't looking anything up. They're producing what fits.

Step 3: Training vs. using the model

There are two very different phases in a model's life:

Two important consequences follow:

1. The model has a knowledge cutoff. It learned from text up to a certain point in time. Unless the tool is connected to web search, it doesn't know about events after that.
2. Your conversation doesn't retrain it on the spot. Correcting the AI helps within that chat, because the correction is now part of the prompt, but the underlying model doesn't instantly "learn" it for everyone.

Why phrasing matters so much

Here's the payoff for prompting. If the model produces whatever text best fits what came before, then your prompt sets the pattern it's continuing.

• Write casually, and you'll tend to get a casual reply.
• Ask for "a few ideas," and you'll get an average-sized list of average ideas.
• Describe an expert audience and a precise format, and the model draws on patterns from expert-level writing in that format.

The first prompt fits countless kinds of text, from textbook definitions to finance blogs, so you'll get something generic. The second narrows the pattern sharply: a specific reader, a specific style, a specific shape. The model doesn't have to guess.

A real-world example

Say you're writing a product description for an online shop. If you paste in three short descriptions you've already written and say "Write one for this new item in the same style," the model will pick up the length, tone, and structure from your examples and continue the pattern. That's next-token prediction working in your favor. You'll learn more about this later as few-shot prompting.

What a language model is not

• It's not a search engine retrieving stored pages (unless a search tool is attached).
• It's not a database of verified facts.
• It's not a person with beliefs or memories of you.

It is a remarkably capable pattern-continuer, and the clearer the pattern you set up, the better it performs.

Quick recap

Text is split into tokens. The model predicts the next token over and over to build a response. It learned its patterns during training and doesn't learn permanently from your chat. Since it continues the pattern your prompt sets, clear and specific phrasing directly shapes the quality of what you get.