Courses / Agentic AI from Scratch (Python)
Session 40 of 40
Final demos, the deployment plan, and next steps
Notes
How to Finish an LLM Capstone: Final Demo, Deployment Plan, Usage Counts and Next Steps
Agentic AI Foundations · Session 40
Before you start: what do I need before I plan the deployment and give the final demo?
You need the code folder for this session open in your terminal, and every command below runs from inside it, as the folder’s README.md does. It holds everything from sessions 37 to 39, plus two new scripts, code/usage_summary.py and code/run_demo.py, and a template for your plan, code/data/deployment_plan.md, where every blank says FILL IN. code/check_deliverables.py changed: its last row now passes only when the plan is filled in. Choose the pack you used in session 39 with export DATA_PACK=banking, using your own pack name, and set LLM_BASE_URL, LLM_API_KEY, LLM_MODEL and EMBED_MODEL. Build the pack with python build_index.py and python make_tickets_db.py, and make the evaluation report with python score_capstone.py unless you still have the one from session 39 for the same pack. No cloud account is needed: the plan is a decision you could carry out, not a deployment you do. Have paper or a notes app ready for the flashcards, the model answer and your plan.
Summary: how do I hand over an AI assistant I built?
A handover needs three things beyond working code. A plan says where the service would run, who looks after it, where its secrets live and what watching it costs. A count says what it has actually been doing, taken from the records it already writes. A demo shows it to someone who has not seen it, in five beats, with one failure shown on purpose. code/usage_summary.py makes the count, code/run_demo.py runs the demo, and the template in code/data/deployment_plan.md becomes your notes/deployment-plan.md. When all five deliverables have their evidence, you choose one next step and write its first task as one sentence.
Key concepts: what do the terms for handing over an AI app mean?
| Term | Definition |
|---|---|
| Deployment | Making an application available to its users, normally by putting it on a remote machine with a server program. |
| Deployment plan | A short document: where it runs, who looks after it, where its secrets live, what watching it costs and what is not covered. |
| Managed web service | A hosting choice where the provider runs the application for you, such as AWS App Runner or Azure App Service. |
| Function service | A hosting choice that runs a small piece of code in response to an event, such as AWS Lambda. |
| Alarm | A rule that watches one number over a period and acts when it crosses a threshold. |
| Audit trail | audit_trail.jsonl: one record for each answer, with no reply text and no token counts. |
| Usage summary | Counts of answers, agents, tools, refusals and approvals made from the records the Copilot writes. |
| Final demo | A scripted showing of the Copilot in five beats, ending with evidence and one failure. |
Explanations: how do the plan, the count and the demo work?
What does a deployment plan say?
Think of a moving-day checklist. The boxes and the van are not the plan; who unlocks the door and who pays are. FastAPI describes deploying an API as making it available to users, normally on a remote machine with a server program, and it says you can run the server yourself or use a cloud service that does part of the work. Your plan, copied from code/data/deployment_plan.md to notes/deployment-plan.md, has five parts, and each is a question a reader will ask:
- Where it runs, and why. Your laptop, a server you run, a managed web service or a function service. Say why this one and not the others, in terms of how much you want to operate yourself.
- Who looks after it. A role, not a name, for restarts, secrets, quality, alarms and data, and the first action each takes.
- Settings and secrets. The names, never the values, where each lives and who may read it.
- What watching it costs. The lines for model calls, hosting, tracing and the account total, what drives each, how you watch it and the threshold at which an alarm tells a person.
- What is not covered yet. At least two gaps, such as who signs users in, or what happens when the model service is down. A plan that names none has not been read hard.
Keep the plan in plain words and do not put a key, a password or a price copied from memory in it.
Where could it run, and who restarts it?
| Choice | Who looks after what |
|---|---|
| Your laptop | You. Good for learning and demos. |
| A server you run | You handle restarts after a crash, HTTPS, and how many copies run. |
| A managed web service | The provider runs it. AWS describes App Runner as a fully managed service for web applications and API services, and Azure App Service lets you run web apps and REST APIs without managing the underlying infrastructure. |
| A function service | A small piece of code runs per event, as with AWS Lambda. A poor fit for a program that loads an index. |
On a container service such as Amazon ECS, tasks can run on AWS Fargate, where ECS manages the infrastructure. Whichever you choose, HTTPS is normally provided by a component outside the application server, a TLS termination proxy. FastAPI advises that whatever restarts a crashed application is an external component, because a crashed app cannot restart itself. And to serve more requests, fastapi run with --workers starts several worker processes, each of which would load its own copy of the index.
How should the plan treat settings and secrets?
The plan lists the names the code reads: LLM_API_KEY, API_KEYS, LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY and DATA_PACK. It says where each lives, in the host’s secret store and not in a file, and who may read it. Streamlit’s documentation says to add a secrets file to .gitignore so you do not commit your secrets, and the same habit applies to any file that holds a key. python settings.py shows SET or MISSING for each secret, never a value.
AWS says a secret left unchanged for a long period is more likely to be compromised, which is the case for rotating it. It also recommends two roles for a secret: one that manages secrets and one that can use the secret at run time. If a key was ever typed into a file, replace it at the provider, because anyone who saw the file has seen the key.
How do I count what the Copilot did?
code/usage_summary.py counts three records it already writes.
| Record | What it holds |
|---|---|
audit_trail.jsonl |
One record for each answer: the pack, role, agents, tools and decision |
audit.log |
The ticket server’s line for each tool call, allowed or refused |
requests.log |
One line for each API call, with how long it took, if you ran the API |
It counts answers by pack, role, decision, agent and tool, and the ticket server’s calls by outcome. A refused answer or a refused call is a record too: it shows what people try. It does not know tokens or money. The audit trail leaves out the reply text and holds no token counts, on purpose. Langfuse tracks the usage and cost of each LLM call and breaks them down by user, session and model, so tokens and cost come from there. Run it after python run_demo.py or after a few questions.
How do I estimate what watching it costs?
Multiply three numbers you can measure and add the price from your provider. The questions a day you expect, times the steps for each question, which python usage_summary.py gives as different tools per answer, times the tokens for each step, which Langfuse shows. Then use the price from your provider’s own price page on the day you write the plan, never a price from memory. When cost is not sent with a call, Langfuse infers it by matching the model to a model definition that stores a price for each usage type.
Cost has other lines. Tracing volume can be controlled with a sampling rate between 0 and 1, which Langfuse applies on the client. For the account total, AWS sends estimated charges to CloudWatch several times daily, and a billing alarm triggers when current billing exceeds the threshold you set. That alarm does not use projections from usage so far, so give it headroom. A cost budget can carry an alert threshold, too. For each line your plan says what drives it, how you watch it and the level at which an alarm tells a person.
What is a final demo, and what are its five beats?
Think of showing a house to a buyer: you walk in an order that answers their questions, and you point out the flaw yourself. code/run_demo.py runs the demo on your pack in five beats.
| Beat | What the viewer should see |
|---|---|
| 1 What it is for | The company, its topics, and the number of documents and records |
| 2 A cited answer | The reply names its source file and section |
| 3 An action with approval | The ticket count does not change after the no, and goes up by exactly one after the yes |
| 4 Protection | The override phrase is turned away; the restricted question gets “I don’t know” for the default role, and the other role gets an answer with a source |
| 5 Evidence | The pass mark, the result, and one failure shown on purpose |
By default the script gives the yes in beat 3 and prints that it is doing so, so the demo runs by itself: the script stands in for a person. --ask-me makes it ask you at the keyboard, as in real use, and --pause waits for Enter between beats. Beat 5 reads reports/evaluation.md, so make it first with python score_capstone.py. Beat 3 leaves one ticket in build/<pack>/tickets.db; python make_tickets_db.py starts it clean again. Each demo question writes an audit record, so usage_summary.py can count the demo afterwards.
Show the failure yourself, with its score. A reviewer who finds a failure you hid trusts the rest less. If the last run had none, say so, and say that one run is not proof.
What next step should I choose?
Choose one and write its first task as one sentence in your plan. Good first tasks are small and checkable:
- Add five more questions to
test_set.json. - Fix the first failure of
reports/evaluation.md. - Add a second data pack of your own.
- Put a sign-in in front of the API, so the role comes from the company’s own login and not from a shared key.
What goes wrong at a handover?
- Nobody knows who restarts it. The plan named a tool, not a role. Fix: a role for each job, and something outside the app to restart it.
- The cost estimate is far too low. The price came from memory and the tokens for each step were never measured. Fix: read the steps from
usage_summary.py, the tokens from Langfuse and the price from the provider’s page. - The ticket count does not go up after the yes. The ticket agent may not have been asked. Fix: run the beat again, or use
--ask-meand answer yes yourself.
| What you learned | The design decision | Where it shows |
|---|---|---|
| A crashed app cannot restart itself | Name a role and an outside restarter | notes/deployment-plan.md |
| The audit trail has no tokens | Take tokens and cost from Langfuse | code/usage_summary.py |
| A hidden failure costs trust | Show one on purpose | code/run_demo.py, beat 5 |
| A check cannot judge quality | A person reads all five deliverables | code/check_deliverables.py |
Flashcards: test yourself on the handover
Say the answer aloud before revealing it. Speaking is a different skill from reading, so practise both.
1. What are the five parts of the deployment plan?
Where it runs and why, who looks after it, settings and secrets, what watching it costs, and what is not covered yet.2. Why does the plan name roles and not people?
People change, and a role such as the owner of restarts stays. A job with no role has no owner.3. Why must something outside the app restart it?
A crashed app cannot restart itself, so the restart belongs to a component outside it.4. What goes in the plan for a secret, and what stays out?
The setting's name, where it lives and who may read it. Never the value, and never a price copied from memory.5. What does `usage_summary.py` count?
Answers by pack, role, decision, agent and tool from the audit trail, the ticket server's allowed and refused calls from `audit.log`, and the API calls from `requests.log` if you ran the API.6. Why is a refused answer worth counting?
It is a record of what people try, so it shows where the protection is being used.7. Where do tokens and cost come from, and why not the audit trail?
From Langfuse, which records usage and cost for each call. The audit trail leaves out reply text and token counts on purpose.8. How do you estimate a day's model cost?
Questions a day, times steps for each question, times tokens for each step, then the price from your provider's own page on the day you write the plan.9. What are the five beats of the final demo?
What it is for, a cited answer, an action with approval, protection, and evidence with one failure.10. Why show a failure on purpose?
A reviewer who finds a failure you hid trusts the rest less.Model answer: how do I explain a handover in an interview?
The core question this session answers: “A manager says: show me the Copilot, and tell me what it would take to run it for real.”
Four beats a manager listens for, in this order: what it does and who for · protection and evidence · who runs it · what it costs and what is open. Skip any of these four and the answer is incomplete.
- What it does and who for: answers from this organisation’s documents, with a source.
- Protection and evidence: the demo’s protection beat, the pass mark and one failure.
- Who runs it: a role for each job, and an outside restarter.
- What it costs and what is open: the cost lines with alarms, and the gaps.
The full answer, said aloud:
(What it does and who for) “It answers questions from this organisation’s documents and records, names its source, and asks a person before it creates a ticket. I will show a cited answer and an approval.”
(Protection and evidence) “A bad instruction is turned away, and the default role cannot reach the restricted file. Here is the pass mark, the result and one failure, with its score.”
(Who runs it) “My plan names a role for restarts, secrets, quality, alarms and data. Something outside the app restarts it, and the keys live in the host’s secret store.”
(What it costs and what is open) “Cost is questions a day, times steps, times tokens, with the price from the provider’s page. Each line has an alarm. Still open: who signs users in, and what happens if the model service is down.”
Missing a beat is a failure state. The follow-up lands where you went quiet: skip the last beat and you are asked what could go wrong. Practise until the four come out without prompting, then stop talking and let the questions come.
Why this matters: why does a deployment plan and a demo matter at work?
In a conversation with a teammate or manager: after “it works” comes “who looks after it, and what does it cost?” A plan with roles and a cost estimate from measured numbers is what turns a demo into a decision.
On the job: the people who run servers ask where it runs, how it restarts and where its secrets live. Having written answers makes the handover short.
In the rest of this course: this is the last session. The deliverables you checked here are what you show: the working app, the diagram, the evaluation report, the security assessment and the plan.
After this unit: what do I do after this lesson, and what comes next?
Assignment. Build your pack and run python score_capstone.py. Run python run_demo.py and check each beat against the table above, then run it with --ask-me and answer yes yourself. Run python usage_summary.py and find the usage of one demo question in Langfuse if you set its keys. Copy data/deployment_plan.md to notes/deployment-plan.md, replace every FILL IN, and run python check_deliverables.py until all five rows pass. Rehearse the five beats aloud, with the failure and its score, then give the demo to a classmate and ask them one question you did not prepare for. Project step. Write code/usage_summary.py and code/run_demo.py (the build prompts are code/prompts/1-write-usage-summary.md and code/prompts/2-write-run-demo.md). The brief for the project work is the project file. After the course: choose your next step and write its first task as one sentence.
Want help building this with someone?
Tell us what you are working on and we will write back about a mentor-led cohort, live sessions or 1:1 training. It is only an enquiry: you choose whether to go further.
Ask about mentor-led trainingTechEazy Consulting training material — not affiliated with or endorsed by OpenAI, Anthropic, LangChain or any other company named in this course.