How does generative AI work? From tokens and the context window to “confident mistakes”

Generative AI for producing text, images, or sound usually follows a shared idea: it converts data into computable units, and then, based on a “context,” probabilistically generates the next step. In text, these units are often “tokens”; that is, small pieces of a word or even part of a word that the model can string together to produce fluent output.

During training, the model learns from a large amount of examples which tokens usually appear alongside each other in language or visual patterns. But this training is different from “answering.” Training is more like learning patterns through lots of practice, whereas “inference” is when you make a request and the model, based on the same learned patterns and in light of your input, generates the continuation step by step.

One of the keys to understanding the model’s behavior is the “context window.” The model only pays attention to the portion of information that fits within this window; that is, your prompt may be long, but the model may not consider every detail at the moment. Therefore, if something is outside the window or is inconsistent with what the model has seen in the window, the output may become incorrect or unrelated—even if it seems highly convincing linguistically.

“Confidence” in these systems usually does not mean the human psychological sense. When the model says something is “true,” it is really saying that a particular token is more likely to come next. If the input context is ambiguous, there are multiple plausible continuations, or the necessary information was not covered sufficiently in the training data, the model may generate an incorrect result while still having a relatively good probability. Here, the logic of “probability” can lead to “confident mistakes.”

Hallucination often comes from a combination of factors: lack of context, an ambiguous prompt, no real source in the response, or the model producing the “most likely continuation” rather than “checking reality.” A practical solution is to read the output like a plausible draft, not a final document; ask the model to be source-based or specify assumptions, and verify sensitive details again using reliable sources.