How does feedback shape an LLM?

A student cannot improve by reading worked examples forever. At some point, they attempt a problem and receive feedback. Models can go through a similar stage.

The model produces possible answers. A test or a person compares them. The training process then encourages behaviour that led to the better result.

For a maths problem, we can often mark an answer right or wrong. Code can be run against tests. When the result can be checked, the training signal is fairly clear.

If five answers are 7, 9, 12, 14 and 15, and 14 is correct, there is little argument about which attempt should win.

Now ask, "Tell me a funny joke." Five people may rank the same jokes in five different ways. For open-ended work, people compare answers and choose which feels more helpful, clear or safe.

This is why a model can solve an equation and still tell a terrible joke. One task has a clear scoreboard. The other depends on taste, culture and disagreement.

Examples teach a model how to respond. Feedback helps shape which responses it should favour. Different model makers use different post-training methods, so this simple picture is not a description of every training recipe.

Once training is finished, using the model starts a different process. That process is called inference.