AI glossary
Every term below is taught properly inside a lesson: plain English first, the technical name second. This page is a reference, not a shortcut: the lessons show why each distinction matters.
- Prediction machine (series' core mental model, Lesson 1)
- Our name for what a generative AI model actually is: a system that produces output by repeatedly predicting what plausibly comes next, based on patterns learned from examples. Not a database, not a search engine, not a mind.
- Generative AI (Lesson 1)
- AI systems that produce new content, such as text, images or audio, rather than only sorting or scoring existing things. This series focuses mainly on generative AI.
- Training data (Lesson 2)
- The enormous collection of examples (web text, books, code and more) from which a model's patterns were learned. What's thick in the data, the model does well; what's thin or missing becomes a blind spot. The model does not keep the documents themselves. It keeps the tendencies they induced.
- Token (Lesson 3)
- The machine's unit of text: a frequent chunk of characters from a fixed catalogue, often smaller than a word. Models read, write, and are priced and limited in tokens, which also explains why they can struggle to count letters inside a word.
- Probability distribution (Lesson 4)
- At every step the model scores every token in its catalogue as a candidate for what comes next: a ranked list from very likely to absurd. The answer you see is drawn from this list, one token at a time.
- Temperature (sampling) (Lesson 4)
- A setting that reshapes how the next token is drawn from the candidate list. Low: consistent, sometimes flat. High: varied, sometimes inspired, sometimes unreliable. That's why the same question can get different answers, by design, not malfunction.
- Training (and weights) (Lesson 5)
- How the model's skill was built: guess the next token, compare with reality, nudge billions of internal settings (the weights) a tiny amount, repeat at vast scale. Once training stops, the weights freeze. Using a model is not training it.
- Neural network (Lesson 6)
- Layers of simple arithmetic (multiply by learned weights, add, pass on) stacked dozens deep. The name honours a loose 1940s biological inspiration, not a brain in a box. No single weight holds any rule; behaviour is spread across billions of them.
- Embedding (Lesson 7)
- A token's (or sentence's) coordinates in a learned meaning-space, where distance tracks similarity of use. The trick behind semantic search: "laptop won't turn on" sits near "computer fails to boot" with zero shared words. Nearness means "used alike", not "correct".
- Attention (and transformer) (Lesson 8)
- The mechanism where every token scores its relevance to every other token, letting context reshape meaning ("bank" drifts riverward or moneyward). A transformer stacks many parallel attention scans dozens of layers deep. It's the 2017 architecture behind today's mainstream AI.
- Inference (Lesson 9)
- Using the trained, frozen model to produce an answer: the whole pipeline runs once per token, and the growing answer feeds back in. Performing, never practising, and metered per token, which is why long answers cost more.
- Hallucination (Lesson 17)
- When a model produces plausible-sounding content that isn't true. Not a rare glitch: it's the normal prediction process operating where its evidence is thin. Specific checkable details and thin-coverage topics are the highest-risk territory. Fluent confidence is a learned style, not a truth signal.
- Bias (AI bias) (Lesson 18)
- Patterns of unequal portrayal or performance inherited from training data, feedback tuning and provider design choices, not malice, and not eliminated by "being mathematical". Consistency is not the same as objectivity. Often shows up as unequal error rates across groups, invisible in any single output and only visible across many.
- Agent (Lesson 16)
- A model given a goal and permission to loop: plan, act via a tool, observe the result, revise, repeat, until it judges the goal met. The same prediction machine and the same tool-use pattern (Lesson 14), composed into a cycle. Small per-step error rates compound across many autonomous steps, which is why gates, caps and reviewing the trace matter more than the model's "intelligence".
- Prompt injection (Lesson 19)
- Hostile instructions hidden inside content an AI system reads, such as a webpage, document or email, which the model's desk treats as just more context, with no privileged channel telling trusted instructions apart from ingested text. With tools attached, an injected instruction can trigger real actions; with memory, it can persist.
- Context window (Lesson 10)
- The fixed budget of tokens a model can use for the current response: the "desk" holding your conversation, pasted documents and the provider's hidden instructions. Most "it forgot" moments are material dropping off this desk, not memory decay.
- Prompt (and prompting) (Lesson 11)
- What you type to steer a model's continuation. Not a command to an obedient mind. It's context that reshapes which candidates on the probability list look most plausible. A well-briefed prompt states the objective, supplies context, sets constraints, shows an example, and says how to handle uncertainty.
- Retrieval-augmented generation (RAG) (Lesson 13)
- Retrieval-augmented generation adds a search step before the model writes its answer: find passages relevant to the question, place them on the desk, then generate a reply from them, with citations. This reduces fabrication but does not eliminate it. A citation is not proof that a passage was genuinely fetched, since a model can still generate a citation that looks real but was invented. Even a genuine citation only shows a passage was retrieved, not that it's current or faithfully represented.
- API and tool use (Lesson 14)
- An API is a set form another program accepts. Tool use is the model writing a structured request in that form (a "work order") for ordinary software (a calculator, a calendar, a live database) to execute exactly, with the result returned to the desk. The model gained a dialling pattern, not the tool's abilities.
- Memory (assistant "memory" features) (Lesson 15)
- An engineered notebook outside the frozen model: the surrounding system extracts facts worth remembering from your chats, stores them, and loads relevant ones back onto the desk in later conversations. Readable, editable and deletable. Not a change to the model's weights, and not the model "learning" you.
Common AI acronyms
Some of these letters already mean something else in your field. Below is what they mean in AI, in plain English, with no assumed background.
- Artificial general intelligence (AGI)
- A hypothetical future AI system that could match or beat human ability across almost any intellectual task, rather than being good at a narrow set of tasks the way today's systems are. No such system exists yet. Researchers genuinely disagree about what would even count as reaching it, so treat confident claims about "achieving AGI" with caution.
- Generative pre-trained transformer (GPT)
- A type of AI model design, not one specific product. "Generative" means it produces new text rather than only retrieving existing text. "Pre-trained" means it learned its patterns from a huge body of text before anyone put it to work on a specific task. "Transformer" names the particular design, covered in this course, that lets a model weigh which words in a passage matter most to each other. Several branded AI products also use "GPT" as part of their name, so the same letters can mean the general design or one specific product, depending on who is speaking.
- Large language model (LLM)
- An AI system trained on a very large amount of text so it can predict and generate language that reads naturally. It is the general category behind most modern AI chatbots and writing tools, including the models this course explains. When someone says "our LLM," they mean the specific model doing the work. Different LLMs vary in size, training data and setup, so it is worth asking which one is actually in use.
- Model Context Protocol (MCP)
- A shared, open standard that lets an AI system connect to outside tools and data sources in a consistent way, instead of a custom connection being built for every single tool. Think of it as a common plug shape: any tool built to that shape can be connected to an AI system without rebuilding the wiring from scratch.
- Natural language processing (NLP)
- The broad field of getting computers to work with human language: understanding it, pulling meaning out of it, and generating it. It is the umbrella term that covers most of what modern AI language systems do, including the prediction based approach at the centre of this course.
- Retrieval-augmented generation (RAG)
- Covered in full further up this page, in the Retrieval-augmented generation entry (Lesson 13).
- Reinforcement learning from human feedback (RLHF)
- A training step where people rate or rank a model's draft responses, and that feedback is used to further shape how the model behaves. It nudges the model toward answers people find more helpful, honest or appropriate. This usually happens after the model's initial training on large amounts of text, as a later refining stage rather than the first one.
- Small language model (SLM)
- A language model built to be much smaller and lighter than the large models most people picture when they hear "AI." Because it is smaller, it can run faster, more cheaply, and sometimes directly on a personal device rather than needing large scale computing power. That efficiency usually comes with a trade off of less general capability than a large model.
Other AI terms
A few more terms that come up in AI discussions but are not acronyms and are not taught inside a specific lesson.
- Recursive self improvement
- The idea of an AI system helping to design, train or improve future versions of itself, with each improved version then better placed to improve the next one again. The concept matters in AI discussions because a cycle like this could, in theory, speed itself up over time, rather than needing a fresh burst of human effort at every step. As things stand today, this is a hypothetical scenario, not a description of how current AI systems are built. Humans still design, train, test and approve every new model by hand, and no AI system builds a meaningfully better version of itself without people directing and checking the work throughout. You will often see recursive self improvement discussed alongside artificial general intelligence, since both describe possible future turning points rather than present day capabilities, and both are genuinely debated among researchers rather than settled questions.