Type "The capital of France is" into a chatbot and it answers "Paris". It feels like the machine looked something up. It didn't. It made a prediction about which piece of text was most likely to come next, and that prediction happened to be right. Almost everything about how large language models work – including why they are brilliant at some tasks and embarrassingly wrong at others – follows from that one fact.
This is a plain-English tour of what an LLM actually is, how it gets built, and what that means for the people using one at work. No equations, but no hand-waving either.
What a large language model actually is
A large language model is a very large mathematical function. You give it a sequence of text; it gives you back a list of probabilities for what the next small chunk of text should be. That's the whole interface.
"Large" refers to the number of adjustable values inside the function, called parameters or weights. Modern models have billions of them. Those numbers are not facts or sentences stored in a database. They are more like the settings of an enormous mixing desk, tuned so that the output sounds like plausible human text.
When you chat with a model, it runs this prediction in a loop: predict the next chunk, append it to the text, predict again, and so on until it produces a stop signal. A paragraph of output is a few hundred individual predictions in a row.
Tokens: how the model reads text
LLMs don't see letters or words. They see tokens – fragments of text produced by a tokenizer. Common words are often a single token ("the", "house"), while rarer words get split ("unbelievably" might become "un", "believ", "ably"). Numbers, code and non-English languages often break into more tokens per word.
This explains some odd behaviour. Ask a model how many times the letter "r" appears in a word and it may stumble, because it never saw individual letters in the first place. It also explains pricing and limits: APIs charge per token, and the context window – how much text the model can consider at once – is measured in tokens, not pages.
How an LLM is trained
Training happens in stages, and each stage shapes behaviour in a different way.
1. Pre-training: reading a huge slice of text
The model starts with random weights and is shown enormous amounts of text – web pages, books, code, articles. For each position it guesses the next token, compares its guess with the real one, and nudges its weights slightly to make the right answer more likely next time. Repeat this trillions of times and something remarkable happens: to get better at predicting text, the model has to absorb grammar, facts, styles of argument, programming patterns and a rough model of how the world is described.
The result is a base model. It is good at continuing text but not at following instructions. Ask it a question and it might respond with three more questions, because that's what a list of exam questions looks like.
2. Fine-tuning: learning to be an assistant
Next, the model is trained on curated examples of instructions paired with good responses. This teaches it the format of a helpful conversation.
3. Learning from preferences
Finally, developers collect judgements about which of two answers is better – more accurate, more helpful, safer – and train the model towards the preferred kind. Techniques here include reinforcement learning from human feedback (RLHF) and related methods. This stage is a big reason why assistants decline some requests, hedge on uncertain points and adopt a consistent tone.
Attention: the idea behind the transformer
The architecture behind nearly all current LLMs is the transformer, introduced by Google researchers in the 2017 paper "Attention Is All You Need". Its key mechanism is attention.
Consider the sentence: "The trophy didn't fit in the suitcase because it was too small." What does "it" refer to? You know it's the suitcase. Attention is the mechanism that lets the model, while processing "it", weigh every other token in the context and decide which ones matter most. The model learns these weightings during training, across many layers, so early layers might track simple grammar while later ones capture meaning and long-range references.
Attention is also why context matters so much. Everything you put into the prompt – instructions, documents, previous messages – is available to be attended to. Anything outside the context window simply doesn't exist for the model in that moment.
Why LLMs hallucinate
Because a language model is trained to produce plausible text, not verified text, it will sometimes generate a fluent, confident answer that is false. This is called hallucination, and it is not a bug that will be patched next month. It is a direct consequence of how the system works.
Hallucinations are most likely when:
- the question is about something obscure, recent or very specific (a citation, a product SKU, a court case number);
- the prompt implies an answer exists ("Summarise the 2019 study that proved X");
- the model is asked for exact figures it has no reliable source for.
The practical fix is to give the model the facts rather than asking it to remember them. Supplying documents in the prompt, or connecting the model to a search index or database – an approach called retrieval-augmented generation (RAG) – dramatically reduces made-up answers because the model can quote rather than recall.
What LLMs are good and bad at
| Strong at | Weak at |
|---|---|
| Drafting, rewriting and summarising text | Precise arithmetic without a tool |
| Explaining concepts at different levels | Facts after its training cut-off |
| Translating and changing tone | Knowing when it doesn't know |
| Writing and explaining code | Long chains of exact logic without checking |
| Extracting structure from messy text | Remembering past chats unless given them |
Many of the weaknesses are addressed by giving the model tools: a calculator, a code interpreter, web search, or access to your company's systems. The model decides when to call the tool, and the tool supplies the reliable part.
Practical takeaways for using LLMs at work
- Treat output as a first draft. Review anything that leaves the building.
- Put the facts in the prompt. Paste the policy, the contract or the data you want it to reason about.
- Ask for sources you can check, and check them.
- Use structure. Clear instructions, examples and a defined output format beat clever wording.
- Don't paste sensitive data into consumer tools unless your company's agreement covers it.
- Match the task to the strengths. Language work: yes. Unverified numbers: no.
The short version
Large language models work by predicting the next token, over and over, using patterns learned from vast amounts of text and shaped by fine-tuning to behave like an assistant. That makes them extraordinary at language and unreliable as an oracle. Once you understand how LLMs work, the right way to use one becomes obvious: give it context, give it tools, and keep a human in the loop for anything that matters.
