← AI for Professionals: Using LLMs at Work
Lesson
Lesson 1: It predicts the next token, it does not look things up
Explain that an LLM generates text by predicting the most likely next token from patterns in its training data, and that by default it does not browse the internet or remember past sessions.
How a language model actually generates text
Prediction, not retrieval
When you type a question into an LLM, it does not open a browser, query a database, or look anything up. Instead, it was trained on an enormous collection of text — books, articles, websites, code — and learned the statistical patterns of what words and phrases tend to follow other words and phrases. Every response is built one small chunk at a time: the model picks the most probable next chunk (called a token), appends it, then picks the next one, and so on until the answer is complete.
Think of it like a very well-read colleague who writes from memory. If you ask 'What is the capital of France?', the model does not fetch the answer from Wikipedia — it generates 'Paris' because that word statistically follows that question in the patterns it absorbed during training. This makes its output a plausible continuation of your prompt, not a verified fact retrieved from a live source.
Two important defaults follow from this. First, the model has no live internet access unless a separate search tool is explicitly connected to it — you will usually see a label like 'Web search: on/off' when that feature is active. Second, it has no memory of previous conversations unless the product you are using specifically stores and re-injects that history. Each new chat starts with a blank slate.
Lesson notes
Prediction, not retrieval
When you type a question into an LLM, it does not open a browser, query a database, or look anything up. Instead, it was trained on an enormous collection of text — books, articles, websites, code — and learned the statistical patterns of what words and phrases tend to follow other words and phrases. Every response is built one small chunk at a time: the model picks the most probable next chunk (called a token), appends it, then picks the next one, and so on until the answer is complete.
Think of it like a very well-read colleague who writes from memory. If you ask 'What is the capital of France?', the model does not fetch the answer from Wikipedia — it generates 'Paris' because that word statistically follows that question in the patterns it absorbed during training. This makes its output a plausible continuation of your prompt, not a verified fact retrieved from a live source.
Two important defaults follow from this. First, the model has no live internet access unless a separate search tool is explicitly connected to it — you will usually see a label like 'Web search: on/off' when that feature is active. Second, it has no memory of previous conversations unless the product you are using specifically stores and re-injects that history. Each new chat starts with a blank slate.