Why does an LLM write one token at a time?

You see the sentence "How are you?" A text model first handles it as small pieces called tokens. A token might be a short word, part of a longer word or punctuation. The exact split depends on the model.

You do not need to count tokens while chatting. Just remember that tokens are the pieces the model reads and writes.

Suppose the input ends with "The capital of France is". The model scores possible next tokens. "Paris" should score strongly. "Rome" and "banana" should score lower.

The system selects a token, adds it to the text and repeats. The answer grows piece by piece until the system stops generation.

The model is not pulling a finished answer from a filing cabinet. It builds the answer as it goes. Once a piece has been produced, the next prediction has to continue from it.

This also helps explain made-up answers. Ask for a recipe for "strawberry-garlic pizza" and the model knows what recipes usually look like. It can build a convincing recipe even if it has no evidence that the dish exists.

Next-token prediction is the central loop behind many text-generating LLMs. Real products may add reasoning methods, search, tools, safety checks and other software around that loop.

The loop is powerful, but plausibility is not proof. Important answers still need reliable sources or checks.