Courses / Agentic AI from Scratch (Python)
Session 12 of 40
Document loaders, chunking strategies and metadata
Notes
How to Load Documents and Split Them Into Chunks With Metadata in LangChain
Agentic AI Foundations · Document loaders, chunking strategies and metadata
Before you start: what do you need to load and split documents in LangChain?
You need this unit’s own code/ folder open in your terminal. It holds load_policies.py, data/policies/ with 5 policy files (leave, expenses, IT security, remote work and learning) and prompts/1-write-load-policies.md. Install two libraries with pip install langchain-community langchain-text-splitters. This session calls no model, so it needs no key and no settings. Have paper or a notes app ready for the Flashcards and the Model answer.
Summary: what are document loaders, chunks and metadata, and why do they matter for RAG?
Before the helper can answer from the company’s own documents, those documents have to get into the program and be cut into pieces. A document loader reads each file and returns a Document: the text, plus metadata saying where it came from. A text splitter then cuts the text into chunks small enough to be found and read on their own, first at the policy’s own headings and then, for any section still too long, by size. Each chunk carries labels for its file, its policy and, where it has one, its section heading, so a later step can name where an answer came from, and the next sessions turn the chunks into numbers, store and search them. How many chunks the 5 policies give is yours to find by running the code.
Key concepts: loader, chunk, splitter, overlap and metadata defined
| Term | Definition |
|---|---|
Document |
LangChain’s standard object for a piece of loaded content, with page_content (the text) and metadata (a dictionary of labels). |
| Document loader | A component that reads a source such as a file and returns a list of Document objects through a standard interface. |
| Chunk | A piece of a document cut small enough to be found and read on its own, and to fit within the model’s context window. |
| Text splitter | A component that cuts documents into chunks. This session uses two: MarkdownHeaderTextSplitter and RecursiveCharacterTextSplitter. |
chunk_size |
The setting for the maximum size of the chunks a splitter returns. In the code it is MAX_CHARS, 600 characters. |
chunk_overlap |
The setting for how many characters neighbouring chunks share. The code sets it to 0, so it asks for no shared text. |
| Chunking strategy | The rule for where to cut: fixed size, recursive, by document structure or by meaning. |
| Metadata | Labels carried beside a chunk’s text: here source (the file name), policy (the title) and heading (the section). |
load_chunks() |
The one function in load_policies.py that returns all the chunks, so later sessions can import it. |
Explanations: how do you load documents and cut them into chunks?
Why can’t you just paste a whole policy into the prompt?
Think of an employee who needs one rule and is handed the whole binder of policies. They waste time, and they may miss the line. A model has the same problem. LangChain’s RAG tutorial notes that models can struggle to find information in very long inputs, and that filling the context window with large amounts of content is not token efficient. In session 9 the whole of every policy went into every prompt; with 5 policies, and more later, that stops being practical.
The fix is the index card. Cut the binder into cards with one idea each, so a search can pull the right card. LangChain’s splitter guide describes the same aim: chunks that can be retrieved individually and that fit within the model’s context window limit. In the helper a card is a chunk.
What is a document loader, and how does the code load the 5 policies?
A post-room scanner turns paper into text a program can use. A librarian’s catalogue card tells you which shelf a book came from. A loader does both jobs: it returns the text and a note of where it came from. LangChain describes loaders as giving a standard interface for reading data from different sources into the Document format, so a PDF and a CSV come out in the same shape as a text file.
In load_policies.py, step 2a is three lines:
loader = DirectoryLoader(str(POLICY_FOLDER), glob="*.md",
loader_cls=TextLoader)
documents = loader.load()
DirectoryLoader with TextLoader reads every .md file in data/policies/. Each file becomes one Document: its text is page_content, and its labels are metadata. The loader already sets source to the file path. The folder holds 5 policies: the three from session 11 and two new ones, remote-work-policy.md and learning-policy.md. Like the first three, the new ones are invented; Fernhill Systems is a made-up company. Five files give enough chunks that the right one has to be found (code/README.md:22).
What goes wrong when you cut a policy badly, and what is a chunk?
First the failure. An employee asks how many unused leave days carry over, and the helper’s answer stops short. The cut landed in the middle of the carry-over clause, so the deadline sits in a different piece from the rule. Neither piece answers “what is the deadline?”. The cause is that the cut ignored where one rule ends and the next begins. A chunk is the opposite: a piece cut so that it can be found and read on its own.
A text splitter makes the cuts. Our code makes them in two stages. The first stage follows the document’s own structure:
by_heading = MarkdownHeaderTextSplitter(
headers_to_split_on=[("#", "policy"), ("##", "heading")])
for section in by_heading.split_text(document.page_content):
...
MarkdownHeaderTextSplitter cuts at the headings you list: # is the policy title and ## is a section heading, such as 2. Learning budget. Each piece keeps its headings as metadata under the names policy and heading. The text before the first ## has no heading, which is why chunk 1 prints heading: None. The policies are organised by numbered clauses, so cutting at headings keeps each rule whole, where a cut by character count would run through a clause.
The second stage is a size limit:
MAX_CHARS = 600
...
by_size = RecursiveCharacterTextSplitter(
chunk_size=MAX_CHARS, chunk_overlap=0)
chunks = by_size.split_documents(sections)
chunk_size is the largest a chunk may be, so a section longer than MAX_CHARS is cut again, and the metadata stays on each part. RecursiveCharacterTextSplitter tries to cut at a paragraph break first, then a line break, then a space. chunk_overlap is the number of characters shared between neighbouring chunks; here it is 0, so the code asks for no shared text. The 600 is this code’s own setting; the project step changes it so you can see what it does.
Which chunking strategy should you use for which document?
Choosing a strategy is like choosing how to cut a loaf: by the slice, by the crust or by the weight of each piece. A cut by the slice follows the natural layout; a cut by weight ignores it. For documents with structure such as Markdown, HTML or JSON, LangChain’s guide says it is beneficial to split by that structure, and for most use cases it suggests starting with RecursiveCharacterTextSplitter. A Pinecone guide describes fixed-size chunking as deciding a number of tokens per chunk, and semantic chunking as using embeddings to find where the topic shifts.
| Strategy | How it cuts | Good for | Watch out for |
|---|---|---|---|
| Fixed size | A set number of characters or tokens | Plain text with no structure | Can cut mid-sentence |
| Recursive | Paragraph break, then line break, then space, then single characters | A sensible default | Ignores headings |
| By document structure | At headings or sections | Policies and manuals | Needs a consistent layout |
| Semantic | Where the topic shifts, found with embeddings | Long, flowing text | Costs embedding calls |
Our code uses two: by structure first, then recursive for any section that is still too long. The same Pinecone guide gives a test for any chunk: if it makes sense to a person without the surrounding context, it will make sense to the model as well. It also says chunks that are too small or too large may lead to imprecise search results, so read your chunks rather than trust a setting. The README states the project’s own reasoning: a policy section is a good chunk because the policies are organised by numbered clauses.
What is metadata, and why does every chunk keep its source?
A luggage tag tells you where a bag came from even when it looks like the others. A footnote keeps a claim and its source together. Metadata does that for chunks.
Failure first. A reviewer asks which policy an answer came from, and nobody can say, because the chunks were kept as bare text. The cause is that the source was dropped when the text was cut. In our code the first splitter returns new pieces that do not carry the file name, so step 2c puts it back:
for section in by_heading.split_text(document.page_content):
section.metadata["source"] = Path(
document.metadata["source"]).name
sections.append(section)
Now each chunk carries source (a file name such as learning-policy.md), policy and heading. Run python load_policies.py and read the output. Ours printed, for the first chunks (code/README.md:47; your own count is what you write down):
34 chunks
--- chunk 1 (83 characters)
source : learning-policy.md
policy : Fernhill Systems - Learning Policy
heading: None
Fernhill Systems is an invented company. This policy is sample data for the course. ...
--- chunk 2 (89 characters)
source : learning-policy.md
policy : Fernhill Systems - Learning Policy
heading: 1. Purpose
1.1 This policy explains how the company supports employees who want to learn new skills. ...
--- chunk 3 (306 characters)
source : learning-policy.md
policy : Fernhill Systems - Learning Policy
heading: 2. Learning budget
2.1 Every employee has a learning budget of 600 for each leave year. The leave year is the one set in the Leave Policy. ...
The order of the files may differ on your computer. These labels are what later sessions use to name the file an answer came from. Print the chunks and read them before anything searches them: a loader that ran without errors does not show that the pieces are useful.
What are three situations where loading goes wrong, and what does each mean for the design?
- The helper quotes half of the carry-over rule. A cut by character count split the clause, and the second half sits in another chunk.
- A reviewer asks which policy an answer came from and nobody can say. The chunks were saved as bare text, so the source was lost.
- Many chunks print
heading: None. The policy file does not use##headings, and the first splitter cuts only at the headings you list.
| What you learned | The design decision | Where it shows |
|---|---|---|
| A cut can split a rule | Cut at the policy’s own headings first | MarkdownHeaderTextSplitter |
| A long section is still too long | Cut it again at a size limit | MAX_CHARS |
| The source is lost on a split | Copy source onto every piece |
Step 2c in load_policies.py |
| Chunk quality decides answer quality | Print and read chunks before searching | The program’s printout |
Basic assignment: set MAX_CHARS = 150 and run again. How many chunks now? Read one cut section: does it still make sense on its own, and does it still show its source and heading?
Project step: load the policies. Run python load_policies.py and write down the chunk count. Set MAX_CHARS = 2000 and see which chunks get bigger and which stay the same. Add a sixth policy file of your own to data/policies/ using the same # and ## layout, and check that its heading appears in the output. The build prompt is prompts/1-write-load-policies.md. Stop when you can explain every line of the printout.
Practice: explain and diagnose. First, answer the Model answer question below aloud in three beats before reading it. Second, for each situation decide whether the fix is the loader, the cut or the metadata, and write your reason in one sentence:
- An answer cannot name its file.
- A rule is split across two chunks.
- A new policy file does not show up in the output.
Flashcards: can you answer these document loading questions aloud?
Say the answer aloud before revealing it. Speaking is a different skill from reading, so practise both.
1. What does a document loader return?
A list of `Document` objects, each with `page_content` (the text) and `metadata` (where it came from). LangChain loaders share a standard interface, so different file types come out in the same shape.2. Which loader does `load_policies.py` use, and what does it read?
`DirectoryLoader` with `TextLoader`. It reads every `.md` file in `data/policies/`, one `Document` per file, with `source` already set to the file path.3. Why cut documents into chunks?
Models can struggle to find information in very long inputs, and chunks are small enough to be retrieved individually and to fit within the context window. A later search can then hand over a few chunks instead of everything.4. What does `MarkdownHeaderTextSplitter` do in the code?
It cuts each file at its `#` and `##` lines and keeps those headings as metadata named `policy` and `heading`.5. What do `chunk_size` and `chunk_overlap` set?
`chunk_size` is the maximum size of a chunk; the code uses `MAX_CHARS`, 600 characters. `chunk_overlap` is the characters shared between neighbouring chunks; the code sets it to 0.6. Why does a chunk print `heading: None`?
It is text before the first `##` in a file, so there is no section heading to copy. Many `None` headings mean the file does not use `##` headings.7. Why does the code copy `source` onto each piece?
The heading splitter returns new pieces that do not carry the file name. Without the copy, a later answer could not say which file it used.8. Name the four chunking strategies and one use for each.
Fixed size for plain text, recursive as a sensible default, by document structure for policies with headings and numbered clauses, semantic for long flowing text where the topic shifts.9. Which two strategies does the code combine, and why?
By structure first, so each rule stays whole, then recursive for any section longer than `MAX_CHARS`. A cut by character count alone would run through a clause.10. How do you check that your chunks are good?
Print them and read some: find a known rule, such as the learning budget in clause 2.1, and check it is whole and labelled with its `source` and `heading`. A loader that ran without errors does not show the pieces are useful.Model answer: how do you explain getting company documents into an AI assistant?
The core question this session answers: “A colleague asks: how do the company’s policy files get into the helper’s program, and why are they cut up first?”
Three beats, in this order: load · cut · keep the labels. Skip any of these three and the answer is incomplete.
- Load: a loader reads each file into a
Documentwith text and metadata. - Cut: a splitter makes pieces small enough to be found and read alone: by heading first, then by size.
- Keep the labels: each chunk keeps
source,policyandheading, so a later answer can name where it came from.
The full answer, said aloud:
(Load) “A loader reads each policy file into a
Document: the text, plus metadata saying where it came from. Ours reads every.mdfile in the policies folder.(Cut) Then a splitter cuts the text into chunks small enough to find and read alone, because models can struggle to find a fact in very long input. Our policies have numbered sections, so I cut at the headings first and then cut any section that is still too long.
(Keep the labels) Each chunk keeps its file, policy and heading, so the search can return a piece and the answer can say where it came from.“
Missing a beat is a failure state. The follow-up question lands where you went quiet: skip the labels and the next question is how anyone checks a source. Practise until the three come out without prompting.
Why this matters: where do document loading and chunking matter at work?
In a scoping meeting: an engineer is asked how documents get into a knowledge assistant. The honest answer starts with what format the documents are in and how they can be cut.
When an assistant gives a half-answer: a wrong or partial answer often traces back to the pieces, not the model. A reviewer who prints some chunks and reads them can see whether a rule was split.
In the rest of this course: later sessions turn these chunks into numbers, store them in a folder, and answer from them, citing the source of the chunk used.
After this unit: what is the assignment and what comes next?
Assignment. Change MAX_CHARS and compare the chunk counts, then add a sixth policy file. The brief for the project work is the project file. Keep load_policies.py: later sessions import it.
What’s next. Embeddings come next: turning each chunk and a question into numbers, and printing the chunks closest to the question.
Want help building this with someone?
Tell us what you are working on and we will write back about a mentor-led cohort, live sessions or 1:1 training. It is only an enquiry: you choose whether to go further.
Ask about mentor-led trainingTechEazy Consulting training material — not affiliated with or endorsed by OpenAI, Anthropic, LangChain or any other company named in this course.