Courses / Agentic AI from Scratch (Python)

Session 13 of 40

Embeddings: text as numbers, similarity as distance

Notes

What Are Embeddings and How Does Similarity Search Find Text by Meaning?

Agentic AI Foundations · Session 13 — Embeddings: text as numbers, similarity as distance

Before you start: what do you need to create and compare embeddings in Python?

You need this unit’s own code/ folder open in your terminal. It holds closest_chunk.py, load_policies.py (session 12’s loader and splitter, carried over), data/policies/ with the policy files and prompts/1-write-closest-chunk.md. Install the libraries with pip install langchain-openai langchain-community langchain-text-splitters. You set the two settings from session 1 (LLM_BASE_URL and LLM_API_KEY) and one new one, EMBED_MODEL, because an embedding model is a different model from the chat model. This session calls an embedding model, so it needs a key. Have paper or a notes app ready for the Flashcards and the Model answer.

Summary: what is an embedding and why does it let search find text by meaning?

An embedding is a list of numbers that places a piece of text so that texts with similar meaning sit close together. An embedding model makes the list. The program makes one list for each policy chunk and one for the employee’s question, then scores every chunk against the question with cosine similarity, a measure of how closely two lists point the same way. That is how “Can I keep leftover holiday days for next year?” finds the clause about unused annual-leave days although the wording differs. The question and the chunks must be embedded with the same model, and a high score is used here to rank chunks, not a guarantee that the chunk answers the question.

Key concepts: embeddings, vectors, cosine similarity and embedding models defined

Term Definition
Embedding A list of numbers that places a piece of text so that texts with similar meaning sit close together.
Vector The mathematical name for that list of numbers; an embedding is a vector.
Embedding model The model that turns text into an embedding. It is a different model from the chat model; different models give different numbers for the same text.
EMBED_MODEL The setting that names the embedding model: by default gemini-embedding-001 in this code.
Cosine similarity A score for how closely two lists point in the same direction; near 1 is very similar, and near 0 means they barely point the same way.
similarity(a, b) The function in closest_chunk.py that computes cosine similarity with only the math module.
embed_documents The LangChain method that embeds a list of texts: all the chunks, in one call.
embed_query The LangChain method that embeds one question.
TOP The setting for how many closest chunks to print; 3 here (code/closest_chunk.py:21).

Explanations: how do embeddings turn text into numbers you can compare?

Why can’t a word match find “leftover holiday days” in a policy about carrying over leave?

Picture a librarian who matches only the words on the spine. You ask for “keeping spare vacation” and she finds nothing, because the book says “carry over unused annual leave”. The same failure here: an employee asks, “Can I keep leftover holiday days for next year?” The leave policy says “Up to 5 unused annual-leave days carry over into the next leave year.” A search for the question’s words finds no match, because the policy never says “leftover” or “holiday”. The cause is that the two texts mean the same thing in different words.

LangChain’s embeddings overview describes the answer: an embedding model transforms raw text into a fixed-length vector of numbers that captures its semantic meaning, so a program can compare and search text by meaning rather than exact words. Texts with similar ideas are placed close together.

What is an embedding, and how does the code make one?

Think of coordinates on a map. Two cafes on one street have nearby coordinates, and “carry over leave” and “keep unused days” get nearby numbers in the same way. Or think of books shelved by topic rather than by title: “borrow a car” and “rent a vehicle” end up on one shelf. OpenAI’s embeddings guide defines an embedding as a list of floating point numbers whose distance measures how related two texts are.

A real embedding has hundreds or thousands of numbers. Our run printed this first line (code/README.md:54):

34 chunks, each is now 3072 numbers long.

The length depends on the model. The picture below uses just two numbers, to show the idea only:

 more about leave
   ^
   |   * "carry over leave"
   |   * "keep unused days"     <-- close together
   |
   |                  * "lost laptop"
   |                  * "reset password"
   +--------------------------------------> more about IT

In closest_chunk.py, step 5 loads the chunks from session 12 and makes the embeddings:

chunks = load_chunks()
chunk_vectors = embeddings.embed_documents(
    [chunk.page_content for chunk in chunks])
question_vector = embeddings.embed_query(question)

embed_documents embeds a list of texts, here all the chunks, in one call. embed_query embeds one question. The embeddings object is an OpenAIEmbeddings, the embedding counterpart of ChatOpenAI, built from the same base URL and key; the comment at code/closest_chunk.py:23 says check_embedding_ctx_length=False stops LangChain counting tokens the OpenAI way, which does not suit other providers.

How does similarity work, and what is cosine similarity?

Two compass needles that point the same way agree, whatever their length. Cosine similarity scores that: near 1 means the two lists point the same way, and near 0 means they are at right angles. LangChain’s overview lists it beside Euclidean distance and dot product as common ways to compare vectors, and OpenAI recommends cosine similarity. In the code it is a few lines of plain Python:

def similarity(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.sqrt(sum(x * x for x in a))
                  * math.sqrt(sum(y * y for y in b)))

Multiply the two lists number by number and add them up (the dot product), then divide by the lengths of the two lists so size does not matter. Search is then simple, and OpenAI’s guide describes the same recipe for retrieval: embed the query, compare it with each document, and return the highest scored. Step 6 scores every chunk, sorts the best first, and prints the top TOP:

scored = []
for chunk, vector in zip(chunks, chunk_vectors):
    scored.append((similarity(question_vector, vector), chunk))
scored.sort(key=lambda pair: pair[0], reverse=True)

Run it with python closest_chunk.py. Ours printed (code/README.md:54; scores and sizes depend on the embedding model):

34 chunks, each is now 3072 numbers long.

Question: Can I keep leftover holiday days for next year?

score 0.708  leave-policy.md / 4. Annual leave
4.1 Request annual leave in the HR portal before you book travel.   4.2 Up to 5 unused annual-leave days carry over into the next leave year. Carried-over days must be used by 31 March; days not used

score 0.683  leave-policy.md / 4. Annual leave
4.6 Leave you have already taken cannot be swapped for pay.   4.7 Two people in the same team may not take the same week of leave unless the manager agrees.   4.8 Employees who leave the company are p

score 0.667  leave-policy.md / 2. Leave year
2.1 The leave year runs from 1 April to 31 March.   2.2 Each employee's balance is shown in the HR portal and is updated at the start of every month.

The best chunk holds clause 4.2, although the question’s words differ from it. Read the scores with care. A score is not a probability and this code sets no pass mark: it uses the scores to put the chunks in order. The closest chunk is not always the right one; here the second place, clause 4.6, is about leave but does not answer the question. A later session passes several chunks to the model and lets it choose. Ask your own question with python closest_chunk.py "Can I work from another country?".

What is the EMBED_MODEL setting, and why must the question and the chunks use the same model?

Failure first. Someone changes the embedding model in one place and not another, and the search breaks: an error about list sizes, or chunks ranked in an order that makes no sense. The cause is that each embedding model has its own map: a point “at 5, 3” on one cartographer’s grid is somewhere else on another’s. Numbers from two models cannot be compared. OpenAI’s guide describes embedding the query with the same model as the documents before computing the similarity.

That is why EMBED_MODEL is one setting used for both calls. The code reads it from the terminal first, then from the script; the options below are the ones listed in the comments at code/closest_chunk.py:15–19:

Provider EMBED_MODEL
Gemini (the default in the script) gemini-embedding-001
OpenAI text-embedding-3-small
Ollama (local) nomic-embed-text, after ollama pull nomic-embed-text

If your chat provider does not offer an embedding model, use a provider from the table for EMBED_MODEL, with that provider’s LLM_BASE_URL and key for this session. LangChain lists what to weigh when choosing an embedding model: quality, cost, latency, vector length, how much text it accepts, languages and licence. It advises a small test on your own data, because leaderboard scores do not always carry over. For a first run, the default in the script is enough.

What are three situations where search by meaning surprises you, and what does each mean for the design?

  1. A size error, or nonsense rankings. The question was embedded with a different model from the chunks.
  2. An error with 404 or “model not found”. EMBED_MODEL is not an embedding model your provider offers.
  3. The top chunk is about leave but does not answer. The closest chunk is about the same topic, and closest is not the same as correct.
What you learned The design decision Where it shows
Words differ, meaning matches Search by embeddings, not by key words embed_documents, embed_query
Numbers belong to the model that made them Use one EMBED_MODEL for both sides The settings at the top
A score is used to rank Read the chunk text, not only the score The printed output
The closest chunk may not answer Hand several chunks to the model A later session

Basic assignment: ask “What do I do if my laptop is stolen?” Does the top chunk come from it-security-policy.md? Then ask “What is the capital of France?” and compare the scores with those of a real policy question.

Project step: find the closest chunks. Run python closest_chunk.py and check that the chunk holding clause 4.2 comes first for the default question. Change TOP to 5 and see where in the list the first off-target chunk appears. The build prompt is prompts/1-write-closest-chunk.md. Stop when you can explain what each of the six numbered steps in the script does.

Practice: explain and predict. First, answer the Model answer question below aloud in three beats before reading it. Second, for each situation, say which of the three beats it breaks:

  1. A search finds the right chunk although the words differ.
  2. A size error or nonsense rankings after changing EMBED_MODEL in one script only.
  3. The top chunk is on topic but does not answer.

Flashcards: can you answer these embedding questions aloud?

Say the answer aloud before revealing it. Speaking is a different skill from reading, so practise both.

1. What is an embedding? A list of numbers that places a piece of text so that texts with similar meaning sit close together. An embedding model produces it.
2. Why can a meaning search find a chunk that shares no key word with the question? Both texts become lists of numbers that capture meaning, so a paraphrase lands close to the chunk even when the words differ. A word match would find nothing.
3. What is cosine similarity? A score for how closely two lists of numbers point in the same direction: near 1 is very similar, near 0 is at right angles. The code computes it with the `math` module only.
4. What do `embed_documents` and `embed_query` do? `embed_documents` embeds a list of texts, here all the chunks in one call; `embed_query` embeds one question.
5. Why does the code need an `EMBED_MODEL` setting? An embedding model is a different model from the chat model, so it needs its own name. The default is `gemini-embedding-001`; other providers use other names.
6. What if your chat provider has no embedding model? Use another provider's embedding model for `EMBED_MODEL`, with that provider's base URL and key for this session.
7. Why embed the question and the chunks with the same model? Each model has its own map, so numbers from two models cannot be compared. Mixing them gives a size error or meaningless scores.
8. Does a score of 0.708 mean the chunk is a certain match? No. A score is not a probability and this code sets no pass mark; it uses scores to rank the chunks.
9. Is the closest chunk always the right one? No. In our run the second place, clause 4.6, was about leave but did not answer the question. Read the chunk text, and later sessions let the model choose among several chunks.
10. What does the program print for each of the closest chunks? The score to three decimals, the chunk's `source` and `heading` from session 12, and the first 200 characters of its text (`code/closest_chunk.py:60`).

Model answer: how do you explain how an AI assistant finds text by meaning?

The core question this session answers: “An examiner asks: how can the helper find the right policy chunk when the employee’s words are different from the policy’s?”

Three beats, in this order: text becomes numbers · closeness is relevance · same model for both sides. Skip any of these three and the answer is incomplete.

  1. Text becomes numbers: an embedding model turns each chunk and the question into a list of numbers that places the text by meaning.
  2. Closeness is relevance: similar meanings get close numbers, so scoring every chunk against the question and sorting finds the paraphrase.
  3. Same model for both sides: the question and the chunks must be embedded with the same model, or the numbers are not comparable.

The full answer, said aloud:

(Text becomes numbers) “An embedding model turns each policy chunk, and the employee’s question, into a list of numbers. The list places the text by meaning, like coordinates on a map.

(Closeness is relevance) Texts that mean similar things get close numbers. I score the question against every chunk with cosine similarity and print the top few, so ‘leftover holiday days’ finds the carry-over clause even though the words differ. The score is used to rank the chunks, so I read the chunk too.

(Same model for both sides) The question and the chunks come from the same embedding model, because each model has its own map. If the model changes, the chunks must be embedded again.“

Missing a beat is a failure state. The follow-up question lands where you went quiet: skip the last beat and the next question is why search broke after a model change. Practise until the three come out without prompting.

Why this matters: where do embeddings matter in real systems?

In a vendor or budget discussion: teams are asked which embedding model to use and what it will cost. OpenAI bills embedding requests by the number of input tokens, so embedding the same chunks again and again has a price, which saving the numbers avoids.

When search results look strange: one thing to check first is whether the question and the stored numbers came from the same model.

In the rest of this course: later sessions save these numbers in a Chroma folder so they are computed once, hand the closest chunks to the model to answer, and tune how many chunks to retrieve.

After this unit: what is the assignment and what comes next?

Assignment. Run your own question through closest_chunk.py, and read which chunk came first and why. The brief for the project work is the project file. Keep closest_chunk.py and load_policies.py: the next session starts from them.

What’s next. The next unit covers vector databases: saving the chunks in a Chroma folder so they are embedded once, then searching it.

Want help building this with someone?

Tell us what you are working on and we will write back about a mentor-led cohort, live sessions or 1:1 training. It is only an enquiry: you choose whether to go further.

Ask about mentor-led training

TechEazy Consulting training material — not affiliated with or endorsed by OpenAI, Anthropic, LangChain or any other company named in this course.