The Map Where “King” Lives Near “Queen”: Embeddings
By the end you'll be able to
- Explain that tokens are represented as positions in a learned space where distance ≈ similarity of use.
- Explain that the positions were learned from context in the diet — not assigned by definition.
- Predict the consequence: similar-meaning texts can be found with zero shared keywords.
- Predict
- Explore
- Decide
- Field note
⚡ 30-second warm-up — Reaching back to Act 1 — Lesson 2: what makes a topic “thin ice” for an AI assistant?
Little was written about it before the training data was collected — niche funders, internal policies, local detail. Fluency stays constant; evidence doesn't.
Meaning as a place on a map — see how machines find related ideas without sharing a single keyword.
Start with a quiet superpower
This machine has never seen a dictionary. Yet it treats "puppy" as close to "kitten" and far from "engine" — and it can tell that "I can't afford the fees right now" belongs with "financial hardship support". Where could that live?
Commit to a prediction
How does it judge that two phrasings mean similar things?
This lesson changes one thing: you'll stop thinking "AI matches keywords" and start thinking "it places meaning on a learned map — and distance on that map is how related things are found with zero shared words."
Discover the mechanism
Inside the machine, each token is a long list of numbers — coordinates in a space with hundreds of dimensions. Training (Lesson 5) nudged those coordinates so that tokens appearing in similar contexts drifted close together. The result is a geography nobody designed: fruits cluster, cities cluster, and the direction from “Canberra” to “Australia” resembles the direction from “Paris” to “France”. These coordinate-lists are called embeddings.
Whole sentences get coordinates too — which is how a computer can measure that “My laptop won't turn on” sits near “Computer fails to boot” despite sharing almost no words. That single trick powers semantic search, document matching, and the retrieval systems coming in Act 3.
Remember what the distances measure: patterns of usage in the training diet — with all its skews (Lesson 2). Nearness means "used similarly", which is not identical to "means the same": "hot" and "cold" are close neighbours. The map inherits the world's habits, including its prejudices — a seed that grows in Act 4 (see the map below).
the map below is flattened to two dimensions from hundreds — a necessary lie. In real high-dimensional space a word can be near many unrelated neighbourhoods at once, which no paper map can show. And modern embeddings shift with context: “bank” moves toward rivers or money depending on its sentence — the mechanism that moves it is Lesson 8.A city where businesses cluster by trade — a legal district, a restaurant strip, a medical precinct — with no zoning authority. The clustering emerged from countless individual decisions. Directions matter too: the walk from “king” to “queen” points the same way as from “prince” to “princess”.
- Businesses = words and concepts. Locations = embedding coordinates.
- Districts = semantic clusters. No zoning authority = learned from data, not assigned.
- Where it breaks: real cities have two dimensions; this space has hundreds — and "same district" means "used alike", never "endorsed" or "true".
Explore the map yourself
Conceptual illustration — a hand-flattened 2-D sketch of a real idea; actual embedding spaces have hundreds of dimensions. Choose a word to see its nearest neighbours, then try the two sentence pairs — one is a trap.
Sentence similarity (the payoff — and the trap)
Where this bites at work
A prospective student writes: “I can't afford the fees right now.” Zane's enquiry system surfaces the financial-hardship support pathway — zero shared keywords, same place on the map. That's embedding search behind an everyday workplace tool, demystified.
His new habit when searching with AI tools: describe what you mean rather than guessing magic keywords. The map rewards description.
Now make the call
A colleague reviews Zane's enquiry system and objects: “It returned a result with no matching keywords — the search is broken. Switch it back to exact keyword matching.” What does Zane do?
Try 'trust it everywhere' — nearness-is-not-correctness is this lesson's sting.
Apply it to your work
Next time you search a knowledge base, CRM or shared drive with an AI-powered tool, describe what you mean in a full sentence instead of guessing keywords — then note whether the matches improve. You're using the meaning map deliberately.
Prove it to yourself
Two quick questions — untimed, retryable, and they count toward ★ Mastered (completed + decision + self-check all correct). This is what defeats the feeling of "that made sense" evaporating by next week.
Explain it in your own words
Two or three sentences for a colleague. Then compare against the rubric — it checks ideas, not wording, and only you see your note.
✓ Lesson 7 stamped: Atlas Explorer
You navigated the meaning map, caught its trap pair, and can now explain semantic search to anyone — including why nearness is not correctness. Lesson 8 makes the map context-aware.
Evidence & review — how we know what this lesson claims
| Claim | Type | Basis | Review risk |
|---|---|---|---|
| Embeddings place tokens in a learned vector space where proximity tracks distributional similarity | How it works | Primary research literature | Low |
| Sentence-level embeddings enable matching without keyword overlap | How it works | Primary research and production practice | Low |
| The map inherits skews from the training diet | How it works (empirical) | Bias research literature | Medium |
| This page's 2-D map and similarity scores | Teaching device | Hand-flattened illustration, labelled | Medium |
| The unzoned-city analogy | Useful mental model | Limits stated in the lesson | Low |
Concept review due: January 2027. Product-behaviour claims: quarterly.
Where you are in the machine
Deep in the engine — the meaning-map layer that Act 3's search tools will reuse. Select any layer for details.
The machine, bottom to top: training data (2) → token pieces (3) → prediction engine (1, 4–9) → the desk (10–11) → senses (12) → live knowledge (13) → tools (14) → memory (15) → agent loops (16) → human judgement (17–21).
Next: the map learns to move
One puzzle remains: “bank” must sit near rivers in one sentence and money in another. Something decides which context matters — and it has the most famous name in modern AI. Lesson 8: attention.