Courses / Agentic AI from Scratch (Python)
Session 9 of 40
LangChain v1: chat models, prompt templates, LCEL and typed replies
Notes
How to Build a LangChain Chain With a Chat Model, a Prompt Template and a Typed Reply
Agentic AI Foundations · Session 9 — LangChain v1: chat models, prompt templates, LCEL and typed replies
Before you start: what do you need to build a LangChain chain?
You need the code folder for this session, the code/ folder, open in your terminal. It holds helpdesk_chain.py, run_test_set.py, the prompt in model_prompts/system.md, data/policies/ with the policy files (expenses-policy.md, it-security-policy.md, leave-policy.md), data/test_set.json with the test cases, and two build prompts in prompts/. Switch your environment on and set your three settings (LLM_BASE_URL, LLM_API_KEY, LLM_MODEL) as in session 1. One new thing to install: pip install langchain-core langchain-openai pydantic. Have paper or a notes app ready for the Flashcards and the Model answer.
Summary: how do you move the helper onto LangChain?
The helper from session 8 worked, but it built its messages by hand and parsed JSON text from the reply. This session rebuilds it on LangChain, a Python library of standard parts for programs that use a language model. The model becomes a chat model built from the same three settings, the prompt becomes a template with blanks that lives in a file, and the reply becomes a typed class with three named fields. The three parts are joined with the pipe symbol into a chain, and the function ask_helper(question) stays, so the session 8 test runner works with little change. You then run the test set to check that nothing broke.
Key concepts: LangChain, chat model, prompt template, chain, LCEL and typed reply defined
| Term | Definition |
|---|---|
| LangChain | A Python library of ready-made parts for programs that use a language model, such as a chat model, a prompt template and a way to join them. |
| Chat model | LangChain’s object for the language model, built once from your settings; here ChatOpenAI. |
| Prompt template | Prompt text with blanks in curly brackets, such as {policies} and {question}, filled in before each call. |
| Chain | Parts joined with the pipe symbol |, so each part’s output becomes the next part’s input. |
| LCEL | The way of joining LangChain parts with operators such as the pipe, so a chain is built from parts in a declared order. |
| Typed reply | A reply fitted into a class you wrote, with named fields and types, instead of free text you parse yourself. |
HelpDeskAnswer |
The class with the three fields answer, section and needs_human. |
with_structured_output |
A method on the chat model that asks for a reply in the shape of a class, such as HelpDeskAnswer. |
ask_helper |
The function other files call: question in, a dictionary of three fields out, or None if the chain failed. |
Explanations: how do the parts of a LangChain chain fit together?
Where are we? What does this session add to the HelpDesk Copilot?
A reply follows five steps: pick the policy section, build the request, read it as tokens, write the reply token by token, and give the answer. This session rebuilds step 2, building the request, as named parts: a template for the text, a model for the settings and a typed class for the reply. The code is in code/. For now all three policies go into the prompt, so step 1 is simple. Sessions 12 to 15 replace that with search.
Failure first. Say a teammate wants to try a different model this afternoon, but the settings and the prompt text are spread through the code and the reply is free text the program parses by hand. Each change is a hunt through files, and an odd character in a reply can break the parser. Splitting the helper into named parts removes that hunt.
What is LangChain, and what does a chat model do?
Think of kitchen appliances with standard plugs: you swap the kettle without rewiring the kitchen. Or building bricks with the same studs, where each brick does one job and clicks to the next. LangChain is a library of such parts for programs that use a language model.
The first part is the chat model. In session 1 you called a service with OpenAI(...); here ChatOpenAI is LangChain’s wrapper for the same kind of service, built from the same three settings:
api_key = os.environ.get("LLM_API_KEY", LLM_API_KEY)
if not api_key:
print("No API key. Paste it into LLM_API_KEY above, or set it in your terminal (session 1 README, Step 3).")
sys.exit(1)
model = ChatOpenAI(
base_url=os.environ.get("LLM_BASE_URL", LLM_BASE_URL),
api_key=api_key,
model=os.environ.get("LLM_MODEL", LLM_MODEL),
temperature=0,
)
The four lines above the model are a check, like a cashier who looks for your card before ringing anything up. With no key in the file and none in your terminal, the program prints that one line and stops, instead of failing later with a long error about missing credentials.
temperature=0 asks for the least random replies, though identical wording on every run is not guaranteed. LangChain’s own documentation says ChatOpenAI accepts a base_url setting, which is how it points at the service in your settings. It is like a phone with the number saved: one contact, called from anywhere in the program. To change the model, or to move to another provider that offers an OpenAI-compatible endpoint, you change the settings block. A provider without such an endpoint needs a different integration class in place of ChatOpenAI.
What is a prompt template, and why do all three policies go into the prompt?
A form letter has gaps: the wording stays and the name and date change. A prompt template is a prompt with gaps, written in curly brackets. The helper’s prompt is the file model_prompts/system.md. It tells the helper to answer only from the policy text, not to invent a figure or a section number, how to fill the three reply fields, when to set needs_human to true, and then ends with the blank:
The policies:
{policies}
The code reads the file and builds the template. {policies} is filled in once, when the program starts; {question} is filled in on every call:
policies = "\n\n".join(path.read_text() for path in sorted((DATA / "policies").glob("*.md")))
system_text = (Path(__file__).parent / "model_prompts" / "system.md").read_text()
prompt = ChatPromptTemplate.from_messages([("system", system_text), ("human", "{question}")])
prompt = prompt.partial(policies=policies)
Two choices differ from session 8. First, the policies are short, so the code reads them once and puts them all in the prompt: the simplest thing that works, with no loop to read. Second, the ticket tool is gone, so the helper answers from policies only, and the test set has a new case, claiming an expense, in place of the two ticket cases. Because the prompt is a file, a teammate can read and change it without touching Python.
What is a chain, and how does the pipe join the parts?
A coffee machine has pipes inside: water goes in one end, passes the heater, then the brewer, and coffee comes out. You press one button. A form that goes through two desks is the same: the first desk fills in the blanks, the second desk answers. A chain is parts joined so that each part’s output feeds the next. In LangChain you join them with the pipe symbol, a style called LCEL. LangChain’s reference describes a sequence built with the | operator, one output serving as the next input.
chain = prompt | model.with_structured_output(HelpDeskAnswer)
Read it left to right: fill the template, send it to the model, return a HelpDeskAnswer.
{"question": ...} --> prompt --> model --> HelpDeskAnswer
fill the blanks send it reply as three fields
One call to chain.invoke({"question": question}) runs every part in order, so you do not carry the result from one part to the next by hand.
The file adds one safety net to the chain. A provider can refuse a typed reply with a 400 error. A spare tyre is the picture: you hope not to use it, but the trip goes on when you do. with_fallbacks gives the chain a second chain to try when the first raises that error; the second asks for the same HelpDeskAnswer, this time returned as a tool call (method="function_calling"):
chain = chain.with_fallbacks(
[prompt | model.with_structured_output(HelpDeskAnswer, method="function_calling")],
exceptions_to_handle=(BadRequestError,),
)
BadRequestError is the 400 error from the openai package, which ChatOpenAI uses underneath. Any other error still reaches ask_helper, below.
What is a typed reply, and what does ask_helper do with it?
In session 5 the prompt asked for JSON and your code parsed the text. A typed reply replaces that. You write a class that lists the fields and their types, and with_structured_output asks the model for a reply in that shape. LangChain’s documentation describes asking a chat model to shape its reply to a schema you give it. Think of a form with three boxes (a short text, a section name and a tick box) instead of a free letter:
class HelpDeskAnswer(BaseModel):
answer: str = Field(description="A short answer, with any figure written as digits.")
section: str = Field(description='The policy section used, such as "Leave Policy 4.2", or "none".')
needs_human: bool = Field(description="True when a person must take over.")
A typed reply fixes the shape, not the truth. OpenAI’s guide on structured outputs says they can still contain mistakes, so a reply that fits the form can still say something wrong. That is why the test set stays.
ask_helper keeps its name and its job:
def ask_helper(question):
try:
return chain.invoke({"question": question}).model_dump()
except Exception as error:
print("The chain failed:", error)
return None
It returns a plain dictionary, or None if anything in the call failed. An empty key never gets this far: the check above the model stops the program first. The broad except Exception also catches a wrong key, a connection problem or a provider error, as well as a reply that could not become a HelpDeskAnswer. Because the name and the shape are the same, run_test_set.py is almost unchanged: it imports from helpdesk_chain and reports “no typed reply came back” when None returns. The last lines of helpdesk_chain.py start with if __name__ == "__main__":; they run only when you start that file yourself, not when the test runner imports it.
How do I run the chain and score it with the test set?
Run the helper once:
python helpdesk_chain.py
Your own run will differ in wording. A real run printed this:
Question: I have 7 unused leave days. How many days can I keep?
answer: You can carry over up to 5 unused annual-leave days into the next leave year.
section: Leave Policy 4.2
needs_human: False
Ask your own question by adding it after the file name, for example python helpdesk_chain.py "Who approves leave?". Then run the test set:
python run_test_set.py
PASS - carry-over days -
PASS - who approves leave -
PASS - lost laptop -
PASS - claiming an expense -
PASS - section that does not exist -
PASS - policy that does not exist -
PASS - sick leave allowance -
PASS - off-topic question -
Passed 8 of 8
Run it more than once; a case can pass on one run and fail on the next. When the helper was first run, it passed 7 of 8 (code/README.md:96): it answered the travel question from a line in the expenses policy instead of handing over to a person. One rule added to model_prompts/system.md (a named policy that does not exist means needs_human) took the count to 8 of 8 (code/README.md:92). That is what a pass count is for.
If something fails, the table helps:
| You see | Do this |
|---|---|
ModuleNotFoundError: langchain_openai |
Run the pip install line above, with your environment switched on. |
An error with 401 |
The service did not accept the key (a typing slip, or a key that is no longer valid); check your settings as in session 1. With no key at all, the program stops earlier with No API key. |
The chain failed: and a message |
Read the message: it may be a key, connection or provider error, or a reply that could not become a HelpDeskAnswer. Fix the cause it names, or run it again, or try another model in LLM_MODEL. |
ModuleNotFoundError: helpdesk_chain |
Run run_test_set.py from the folder that holds both scripts. |
What are three situations where plain calls hurt, and what does each mean for the design?
- A teammate wants to try another model this afternoon, but the settings are scattered through the code. The model was not one named part built from one settings block.
- A reply comes back as free text and the program breaks on an odd character. The program parsed text by hand instead of asking for a typed reply.
- The rebuilt helper looks fine on one question, and nobody knows whether it still handles a missing policy. The session 8 test set was not run on the new version.
| What you learned | The design decision | Where it shows |
|---|---|---|
| One model, one settings block | Build the chat model once | ChatOpenAI(...) in helpdesk_chain.py |
| A prompt is text with blanks | Keep it in a file and fill the blanks | model_prompts/system.md |
| Free text breaks parsers | Ask for a typed reply | HelpDeskAnswer |
| A rebuild can break old behaviour | Keep ask_helper and re-run the test set |
run_test_set.py |
Basic assignment: rebuild the helper as a chain and run three test-set questions through it.
Project step: rebuild the helper and score it. Write helpdesk_chain.py with your coding assistant using prompts/1-write-helpdesk-chain.md, then run_test_set.py using prompts/2-write-run-test-set.md. Run both. Then change temperature=0 to temperature=1, run the test set twice and note whether the count moves, and add a ninth case to data/test_set.json that you think will break the helper. Stop when you have a pass count for both temperatures.
Practice: explain the move. Answer the Model answer question below aloud in four beats before reading it. Then say which of the four parts you would change to try another model, and which to change to reword the prompt.
Flashcards: LangChain chain, prompt template and typed reply questions with answers
Say the answer aloud before revealing it. Speaking is a different skill from reading, so practise both.
1. What is LangChain, and why move the helper onto it?
It is a Python library of ready-made parts for programs that use a language model. Moving the helper onto it gives the model, the prompt and the reply type their own names, so each can be changed and tested on its own.2. What does `ChatOpenAI` do in `helpdesk_chain.py`?
It builds the chat model from the same three settings as session 1, with `temperature=0` for the least random replies. Changing the model, or moving to another OpenAI-compatible provider, means changing the settings block; a provider without that kind of endpoint needs a different integration class.3. What is a prompt template?
It is prompt text with blanks in curly brackets, filled in before each call. In the helper, `{policies}` is filled once at start-up and `{question}` on every call.4. Why does the prompt live in `model_prompts/system.md` and not in the code?
A teammate can read and change a file without touching Python, and a change shows up as a change to one file. The code only loads the text and builds the template.5. Why do all three policies go into the prompt in this session?
They are short, three files in `code/data/policies/`, so reading them once and putting them all in is the simplest thing that works. Sessions 12 to 15 replace it with search, so only the right part goes in.6. What does `prompt | model.with_structured_output(HelpDeskAnswer)` mean?
Read it left to right: fill the template, send it to the model, return a `HelpDeskAnswer`. The pipe joins the parts so each output becomes the next input.7. What is a typed reply, and what does it replace?
It is a reply fitted into a class with named fields and types. It replaces asking for JSON in the prompt and parsing the text yourself.8. Does a typed reply mean the answer is correct?
No. It fixes the shape, not the truth, and OpenAI's guide says structured outputs can still contain mistakes. That is why the test set is run on the new helper.9. Why was `ask_helper` kept with the same name?
Other files call it, so keeping the name and the shape (question in, a dictionary of three fields or `None` out) lets you rebuild what is inside without rewriting `run_test_set.py`.10. What does `The chain failed:` tell you?
It means something in the chain call raised an error and `ask_helper` returned `None`. The cause can be a key, connection or provider error, or a reply that could not become a `HelpDeskAnswer`; read the message after it, then fix that cause or try another model in `LLM_MODEL`.11. What happened when the rebuilt helper was first scored, and what fixed it?
It passed 7 of 8 (`code/README.md:96`), because it answered the travel question from a line in the expenses policy. One rule in `system.md`, that a named policy that does not exist means `needs_human`, took it to 8 of 8 (`code/README.md:92`).Model answer: how did you move the helper onto LangChain, and how do you know it still works?
The core question this session answers: “You moved the helper onto LangChain. What are the parts, how are they joined, and how do you know it still works?”
Four beats, in this order: chat model · prompt template · chain with a typed reply · test set. Skip any of these four and the answer is incomplete.
- Chat model:
ChatOpenAI, built from the same three settings as session 1, withtemperature=0. - Prompt template: text in
model_prompts/system.mdwith blanks; the policies are filled in once and the question on every call. - Chain with a typed reply:
prompt | model.with_structured_output(HelpDeskAnswer)fills the template, calls the model and returns three named fields. - Test set: run
run_test_set.pyon the new helper and read the pass count, more than once.
The full answer, said aloud:
(Chat model) “I built the model as one named part, a
ChatOpenAImade from the same three settings as before, so changing the model, or moving to another OpenAI-compatible provider, means changing the settings.(Prompt template) The prompt is a file with two blanks. The policies are filled in once, and the question on every call, so anyone can read or change the wording without touching the code.
(Chain with a typed reply) I joined the template and the model with the pipe, and asked the model for a reply in the shape of a class with three fields. The function
ask_helperkept its name, so the rest of the code did not change.(Test set) A typed reply fixes the shape, not the truth, so I ran the test set on the new helper, read the pass count, and ran it more than once.“
Missing a beat is a failure state. The follow-up question lands where you went quiet: skip the test set and the next question is how you know the rebuild did not break anything. Practise until the four come out without prompting.
Why this matters: where do you use chains, templates and typed replies at work?
In a design review: someone asks why the feature uses a library and not plain calls. The answer is parts with names: a model you can swap, a prompt a teammate can edit, a reply type that other code can rely on.
In a prompt change request: a support lead wants new wording. When the prompt is a file with blanks, the change is one file, and the test set tells you whether it helped.
When you pick up someone else’s code: you can read prompt | model.with_structured_output(...) left to right and know what runs.
In the rest of this course: session 10 adds a second chain in front of this one, session 11 gives the chain a memory, and sessions 12 to 15 replace “all the policies in the prompt” with search.
After this unit: what is the assignment and what comes next?
Assignment. Rebuild the helper as a chain and send three test-set questions through it. The brief for the project work, scoring the whole test set and trying a higher temperature, is the project file. Keep helpdesk_chain.py and your test set: the next session builds on them.
What’s next. Session 10 adds prompt chaining: a first chain sorts the question, then a second chain answers from the right policy.
Want help building this with someone?
Tell us what you are working on and we will write back about a mentor-led cohort, live sessions or 1:1 training. It is only an enquiry: you choose whether to go further.
Ask about mentor-led trainingTechEazy Consulting training material — not affiliated with or endorsed by OpenAI, Anthropic, LangChain or any other company named in this course.