Student Notes — Chapter 9: Run it: logs, cost per ticket, alarm, quality check, teardown

AWS in Action: AI edition · Chapter 9

Before you start

Nothing is needed to read this chapter or to run its offline checks beyond Python and the development tools listed in the course repository’s requirements-dev.txt. The offline checks use fakes and read local files, so they need no AWS account, no GitHub account and cost nothing. The live steps can be done from the AWS console plus CloudShell, and they need: an AWS account; a user that is not the root user (as the root user, a Bedrock step can stop with an AccessDeniedException that says the root user is not authorized); a Region that offers Amazon Bedrock and S3 Vectors; Amazon Nova Lite and Amazon Titan Text Embeddings V2 shown in Amazon Bedrock → Model catalog; a budget alert; your own GitHub repository set up as in chapter 8, with the escalation email confirmed; the chapter 8 bootstrap stack redeployed, because it gained permissions; and a few messages sent through a stack before you read its logs. The steps are written in SETUP.md at the top of the course repository, and the checklist for this chapter is in chapters/09-run-it/README.md under Before you try. The live steps create real resources, call the model and cost money, so you delete everything at the end. This chapter builds on chapters 6 and 8. Recap: an SNS topic is a named channel that emails everyone subscribed, and an email subscription delivers nothing until its owner confirms it; the workflow deploys staging, then prod after a person approves, using a bootstrap stack that holds its roles; a stack built from a template is created and deleted as one unit. Have paper or a notes app open for the Flashcards and the Model answer.

Three ways to follow this chapter, the same as before, and you can mix them:

• Watch: read the chapter and follow the build on screen. Nothing needed.
• Copy and run: copy the folder chapters/09-run-it/ and run its checks.
• Build with AI: open chapters/09-run-it/prompts/, run each prompt in an AI coding tool, then run that prompt’s check. The checks for the whole chapter are in chapters/09-run-it/prompts/CHECK.md.

Summary

This chapter turns the finished helpdesk into something you can run. Each function writes one structured summary line per request; metric filters turn those lines into counts, a dashboard draws them, and two alarms email the team through the topic from chapter 6. You work out cost per ticket with a formula that states no price, because you read the current price yourself, and you add a quality check that asks a fixed list of questions after each deploy and stops the run before prod if an answer has the wrong shape. A short voice demo shows where the helpdesk could go, with no build steps. Last, you delete the prod stack, the staging stack and the bootstrap stack, and run three checks that must print nothing.

Key concepts

Term	Definition
Structured log line	A log entry that writes the facts of one request as labelled fields in JSON, so a tool can filter and add them up.
Logs Insights	CloudWatch’s search page for logs. A query picks fields, filters lines and adds them up.
Token count	How many tokens went into the model and how many it wrote back, as the model reports after each call.
Metric	A number CloudWatch keeps over time, such as the count of errors.
Metric filter	A rule that watches a log group for lines matching a pattern and adds one to a metric each time a line matches.
Dashboard	One page of charts in CloudWatch.
Alarm	A watcher on one metric that is OK while the metric is within a limit you set and in ALARM when it is outside, and can run an action such as an email.
Cost per ticket	What the helpdesk cost to run over a period, divided by the tickets it created in that period.
Quality check	A fixed list of helpdesk questions asked after each deploy that fails when an answer has the wrong shape.
Teardown	Deleting every stack in the right order and then proving the account holds none of them.

Explanations

Orientation: where this chapter sits

Chapters 2 to 8 built the helpdesk. This chapter lets you see it while it runs, hear about it when it breaks, check it after every deploy and remove it. In the finished helpdesk the monitoring is the part around the API:

Today you build the monitoring part of the picture.

Try this first: the helpdesk has run for a week and your manager asks whether it is healthy and what one ticket costs. Wait until an employee complains; read the helpdesk’s own counts of errors, tokens and tickets and work out the cost; or divide the whole AWS bill by the number of employees. Pick one and say why. The chapter builds the second.

By the end you can:

• explain how a log line becomes a chart and an email;
• work out a cost per ticket from your own totals and the current prices;
• run the quality check, test the alarm, and delete every stack and prove it.

Part 1. The goal: a dashboard, an alarm and a clean delete

Picture a shop with a notebook by the till, a wall of charts, a smoke detector and a closing-up checklist. The helpdesk gets the same four things. Every function writes one summary line per request. Those lines become counts of errors, tokens and tickets. A dashboard draws the counts, and an alarm emails the team when errors appear. A quality check asks fixed questions after each deploy. At the end you work out cost per ticket and delete every stack.

The code is in the course repository under chapters/09-run-it/. Its pieces: the four handlers in chapters/09-run-it/app/, the template chapters/09-run-it/template.yaml, the bootstrap template chapters/09-run-it/deploy-role.yaml, the quality check in chapters/09-run-it/app/quality/check.py with its questions in chapters/09-run-it/app/quality/questions.json, the formula in chapters/09-run-it/app/ops/cost.py, and the workflow chapters/09-run-it/.github/workflows/deploy.yml. A prompts/ folder holds one prompt per build step with a check for each.

One change reaches back through the chapters, so that the helpdesk also works outside us-east-1. The functions call the model through the Region’s inference profile, chosen by the template’s ModelRoute parameter (auto by default) and shown in the ModelInUse output, exactly as in chapters 2 to 7. The bootstrap template’s permissions boundary therefore allows the model in every Region and the profile in this one, and its create-time tag conditions accept the project and environment tags that every resource now carries next to chapter. Every resource name starts with the Project (only helpdesk here) and the Environment, so the stack helpdesk-staging holds helpdesk-staging-ask, helpdesk-staging-errors and so on. If you built the bootstrap stack from an earlier version of this chapter, redeploy it.

In the system picture, CloudWatch is the new box: each function sends it one log line per request, it holds the logs, the dashboard and the alarms, and an alarm goes to the SNS topic from chapter 6, which emails the IT staff. GitHub Actions from chapter 8 now also runs a quality check against the functions.

Costs money. The live steps create real resources and call the model. The CloudWatch pricing page lists custom dashboards, alarms, custom metrics and log data among what CloudWatch charges for, each with a free tier you should read there. Deploy, try and delete in one sitting.

Part 2. Read the helpdesk’s logs in CloudWatch

A hotel check-in form has labelled boxes for name, room and nights. A diary page is one long sentence. If you must find everyone who stayed three nights, the form is easy and the diary is a slog. A till receipt has a column for items and a column for prices, so adding a total is easy. A structured log line is the form and the receipt: it writes the facts of one request as labelled fields, in a format called JSON, so a tool can filter them and add them up. The AWS documentation says JSON structured logs make logs easier to search and analyse.

The AWS documentation says the default log format for Python functions in Lambda is plain text, and with plain text, reading them for “how many requests failed today” meant reading every line. Now each function writes one line per request, like this (the values are made up):

{"event": "request", "route": "tickets",
 "outcome": "ok", "status": 200, "role": "employee",
 "latency_ms": 412, "model_calls": 2,
 "input_tokens": 1830, "output_tokens": 164, ...}

The line holds numbers and words about the request, not what anyone wrote: no message, no answer, no ticket title, no email and no user id. The outcome is one of ok, blocked, refused, bad request or error, and the role is employee, staff or none. Logs are read by more people than chat is, and they last, which is why the private text stays out. For the tickets and triage routes the line also carries the model calls, the input and output tokens and, for tickets, the number of tickets created. The template sets the function’s log format to JSON and the log level in the function’s logging settings; the AWS documentation recommends setting the level there, not in code, when the format is JSON.

To read the lines, open Logs Insights, CloudWatch’s search page for logs, choose the log groups of one stack (for example /aws/lambda/helpdesk-staging-tickets and /aws/lambda/helpdesk-staging-triage) and run:

fields @timestamp, route, outcome, role, latency_ms
| filter event = "request"
| sort @timestamp desc
| limit 20

Because the lines are JSON, each field is a column you can filter and add up. To total a day, filter the lines where the model was called and add up the tokens and tickets:

filter event = "request" and model_calls > 0
| stats sum(input_tokens) as input_tokens, sum(output_tokens) as output_tokens, sum(tickets_created) as tickets by bin(1d)

If route, outcome and role show as empty columns, the log format is not JSON, and the metric filters in Part 5 will match nothing. That is the first thing to check. The commands are also in chapters/09-run-it/README.md.

Part 3. Count tokens per request

A taxi meter counts distance, and a phone bill counts minutes. A model’s meter counts tokens, the pieces of text from chapter 1. A token count is how many tokens went into the model and how many it wrote back; the Converse response reports both for each call, as input tokens and output tokens.

There are two places to read them, and they answer different questions.

Where you read it	What it counts	What it covers
The log lines	The tokens the model reported for each request	The tickets and triage routes, per request
Bedrock’s own metrics	Calls made to the model, and their input and output tokens	The whole account, by model

The Bedrock metrics are published under the AWS/Bedrock namespace: InputTokenCount, OutputTokenCount and Invocations, with a model dimension (the id the function calls, so the inference profile id when a profile is used). The dashboard divides the logged tokens by the logged number of model requests to show tokens per request for tickets and triage, and shows Bedrock’s tokens per model call for the model you chose.

Here is the gap to know about. The knowledge-base answers behind /ask report no token count to our code, so /ask writes none to the log line and is not in the logged totals. The AWS page lists the operations the Bedrock metrics count (Converse, ConverseStream, InvokeModel and InvokeModelWithResponseStream), and this chapter does not claim that the metrics include or leave out the knowledge-base calls. When you share a number, say what it leaves out.

Part 4. Work out cost per ticket with a formula

A sandwich shop sets the price tag by dividing what it spent making all the sandwiches by how many it made. A phone plan is the month’s cost divided by the calls you made. Cost per ticket is the same: what the helpdesk cost to run over a period, divided by the number of tickets it created in that period.

Three everyday cases show where it goes wrong. A taxi firm adds fuel, repairs and wages and divides by trips; leave out the wages and each trip looks cheaper. A school canteen divides food and gas bills by meals served; mix the bills of one week with the meals of another and the answer is wrong. A courier with no parcels this month has nothing to divide by. The helpdesk is the same: tokens and other costs, over one period, divided by tickets; with no tickets there is no cost per ticket.

The formula, as in chapters/09-run-it/app/ops/cost.py:

cost per ticket = total cost / tickets
total cost = input tokens / per x input price
           + output tokens / per x output price
           + other costs

per is how many tokens one price covers. The Bedrock pricing page quotes some models’ prices per 1,000 tokens and others per 1M tokens, so read the unit on the page. Nothing in this course states a price. You read the current prices for your model on the pricing page, on the day you work it out, and note the date. Where each number comes from:

Number	Where you read it
Input tokens and output tokens	A Logs Insights total for the period (the query in Part 2)
Tickets	A Logs Insights total of tickets created
Input price and output price	The Bedrock pricing page, today
Per	The unit that page quotes its price in
Other costs	You add up what else to count, such as the guardrail, the knowledge base and the dashboard

Run it with the command, replacing each placeholder with your own number:

python app/ops/cost.py --input-tokens <n> --output-tokens <n> --tickets <n> --input-price <p> --output-price <p> --per <n> --other-costs <c>

The command refuses to run without every price and without at least one ticket, and it prints the result in the currency of the prices you entered. The offline check uses obviously made-up round numbers that are not prices, and a run with zero tickets that prints exit 2.

Situation to avoid: a cost per ticket that looks lower than your bill suggests. Cause: the /ask route logs no tokens, so its model use is missing from the token totals, and other costs may be missing too. Say what the number leaves out.

Part 5. A dashboard and an alarm that email the team

Tally marks on a whiteboard, one for each time a customer complains, turn complaints into a count. A turnstile at a gate counts each person who passes. A metric filter does that for log lines: it watches a log group for lines that match a pattern and adds one to a metric, a number CloudWatch keeps over time, each time a line matches. The AWS documentation describes metric filters as a way to look for terms and patterns in log data as it is sent to CloudWatch.

The template has eleven metric filters, one metric each, under the name Helpdesk/<environment>. There are five metric names: Errors (a request that ended in an error, from all four functions), ModelRequests, InputTokens and OutputTokens (from the tickets and triage functions), and TicketsCreated (from the tickets function). The filters are no new code: they count the lines you already write.

Read it from the first box: a function writes a line, the log group keeps it, the metric filter counts matching lines, the count is a metric, an alarm checks the metric, and when it trips it writes to the topic from chapter 6, which emails the team. Nobody has to watch a screen.

A dashboard is one page of charts. HelpdeskDashboard draws six widgets:

Widget	What it shows
A text note	The cost-per-ticket formula, with no price
API requests and server errors	The API’s request count and server errors
Requests that ended in an error	The Errors metric, with the errors alarm drawn on it
Tokens per request	Logged tokens divided by model requests
Tokens per model call	Bedrock’s own token counts for this model
Tickets created	The TicketsCreated metric

A smoke detector is silent until there is smoke; a fuel warning light means you do not watch the gauge. An alarm works like that: it watches one metric, is OK while the metric is within the limit you set, is in ALARM when it is outside, and can run an action, here an email through the SNS topic. You have two:

Alarm	Watches	Name it gets
ErrorsAlarm	The Errors metric from our log lines	<project>-<environment>-errors, for example helpdesk-staging-errors
ApiServerErrorsAlarm	The API’s own server-error count	<project>-<environment>-api-5xx

Both add up errors over one period and trip at one error; the period and the limit are the course’s choices set in the template, and the AWS documentation says an alarm’s period is the length of time used to evaluate the metric for each data point, in seconds. Both email through the escalation topic, so the address you confirmed in chapter 6 receives them. An email subscription delivers nothing until it is confirmed, so an unconfirmed address means a silent alarm.

Test the path without breaking anything. AWS documents set-alarm-state as a way to set an alarm’s state temporarily for testing, and changing it to ALARM sends the notification:

aws cloudwatch set-alarm-state --alarm-name helpdesk-staging-errors --state-value ALARM --state-reason "test"

An email arrives, and the alarm returns to its real state by itself, often within seconds. If no email arrives, check that the chapter 6 subscription is confirmed. The stack outputs DashboardName and ErrorsAlarmName name the dashboard and the errors alarm.

Situation to avoid: an alarm that exists but never emails. Cause: the subscription was never confirmed, or the log format is not JSON so the filter counts nothing. The test above separates the two: if the hand-set alarm emails, the path works and the metric is the problem.

Part 6. A small quality check after each deploy

A pilot works through the same checklist before every flight, and a baker tastes the first loaf before the shop opens. A quality check is that for the helpdesk: a fixed list of real questions asked after each deploy, failing when an answer has the wrong shape.

The questions are in chapters/09-run-it/app/quality/questions.json:

Question	What must happen
“How do I reset my password?”	The answer cites the password-reset page
“How do I get VPN access?”	The answer cites the VPN page
“How do I ask for a new laptop?”	The answer cites the new-laptop page
“Which stock should I buy this year?”	The question is blocked, with no sources

The check also fails on a status other than 200, no answer text, an answer far too long, a source that is not a policy page with an excerpt, a normal question that was blocked, and any answer or excerpt that looks like a secret. It checks the shape, not whether the wording is perfect. Add a question by adding an entry that expects a cited policy page or a block.

Read it from the first box: the workflow deploys the stack and then waits for the knowledge base ingestion job to finish, because the documentation says that each time files change the data source must be synced before they are indexed. It then asks each question of the ask function. If every answer has the right shape, staging passes and prod waits for a person to approve. Otherwise the run stops before prod, because the prod job waits for the staging job to succeed. An exit code of 0 means all passed, 1 means an answer had the wrong shape, and 2 means the check could not run.

To run it by hand with your own credentials:

python app/quality/check.py wait --knowledge-base-id <id> --data-source-id <id> --job-id <id>
python app/quality/check.py run --function-name helpdesk-staging-ask

What it proves and what it does not. It invokes the ask function directly with the claims of a made-up employee, so it needs no password and stores no secret. That proves the function, the knowledge base, the model and the guardrail work together after a deploy. It does not test sign-in or the API’s token check; the unit tests and chapter 7’s live steps cover those.

Three situations show what goes wrong:

Situation 1: the cost that looks too low. Cost per ticket is lower than the bill suggests. Cause: the /ask route logs no tokens, so its model use is missing from the totals.

Situation 2: the alarm email that does not arrive. You test the alarm and nothing is in the inbox. Cause: the topic’s email subscription from chapter 6 was never confirmed.

Situation 3: the failed check. The check fails after a staging deploy. Cause: an answer had the wrong shape, so prod waits; read the line that names the question and the problem.

Part 7. Voice-call demo: a short teaser

Your helpdesk takes typed questions. A phone call is the same question spoken. This part is a short demo with no build steps: someone speaks a helpdesk question and hears an answer. The demo uses Amazon Nova Sonic, a model that provides real-time, conversational interactions through bidirectional audio streaming, according to the AWS documentation. Nothing in this chapter’s code or template builds it, and building it belongs to separate advanced sessions. The chat helpdesk you built is the base a voice front end could sit on.

Part 8. Teardown: delete every stack, confirm it is clean

A hotel guest checks out and the room is cleared; a room left occupied keeps being charged. A stack you do not delete keeps costing money. There are three stacks and nothing else in AWS:

Stack	What it holds	Delete
helpdesk-prod	The live helpdesk, its logs, alarms and dashboard	First
helpdesk-staging	The same template, for testing	Second
github-bootstrap	The roles the workflow uses to deploy	Last

The bootstrap stack goes last because the workflow’s roles live in it. Empty each site bucket first: the documentation says you can delete buckets only if they do not have any objects in them.

for ENV in prod staging; do
  BUCKET=$(aws cloudformation describe-stacks --stack-name "helpdesk-$ENV" --query "Stacks[0].Outputs[?OutputKey=='SiteBucketName'].OutputValue" --output text)
  aws s3 rm "s3://$BUCKET" --recursive
  aws cloudformation delete-stack --stack-name "helpdesk-$ENV"
  aws cloudformation wait stack-delete-complete --stack-name "helpdesk-$ENV"
done
aws cloudformation delete-stack --stack-name github-bootstrap
aws cloudformation wait stack-delete-complete --stack-name github-bootstrap

Each wait polls until the stack status is DELETE_COMPLETE, so when it returns the stack is gone. “Deleted” is a claim, so prove it with three checks. Each should print nothing:

aws cloudformation list-stacks --stack-status-filter CREATE_COMPLETE UPDATE_COMPLETE UPDATE_ROLLBACK_COMPLETE ROLLBACK_COMPLETE DELETE_FAILED --query "StackSummaries[?starts_with(StackName, 'helpdesk-') || StackName == 'github-bootstrap'].StackName" --output text
aws cloudwatch describe-alarms --alarm-name-prefix helpdesk- --query "MetricAlarms[].AlarmName" --output text
aws logs describe-log-groups --log-group-name-prefix /aws/lambda/helpdesk- --query "logGroups[].logGroupName" --output text

The first lists stacks in a live or failed state whose names start with helpdesk- or are the bootstrap stack; the second lists alarms with the prefix helpdesk-; the third lists log groups with the prefix /aws/lambda/helpdesk-. A stack that shows DELETE_FAILED names the resource that blocked it; the documentation says a deletion can fail because one resource fails to delete. Fix that one and delete again.

What stays outside AWS: the GitHub environments, the repository variables and your code. Remove those in your repository settings if you no longer want them. The bootstrap stack removes the GitHub sign-in provider if that stack created it, so delete it only when no other project in the account uses the provider. The commands are in chapters/09-run-it/README.md.

Looking back. Each chapter added one piece you can name and delete:

Chapter	What the helpdesk gained
2	A hosted model behind /ask
3	A private chat page over HTTPS
4	Answers from the IT knowledge base, with a cited page
5	Tickets through tool use
6	A triage loop that emails IT staff
7	Sign-in, two roles and a guardrail
8	Staging, prod and a workflow that deploys
9	Logs, a dashboard, an alarm, a check and a clean delete

What to do with each signal. This table turns the chapter into a routine:

Signal	What it usually means	What you do
The errors alarm email	A function logged a request that ended in an error	Query for outcome error and read the route and status
The API server-error alarm	The API answered with a server error	Read the function’s log group
Tokens per request rising	Requests send or write more	Compare input and output tokens by route
Cost per ticket rising	Tokens, other costs or tickets moved	Recompute for one period, part by part
The check fails	An answer had the wrong shape	Read the failing question and fix before prod

What changed in the template

Templates in this course are cumulative: chapter 9 holds the 45 resources chapter 8 held, plus 14 new ones, and each function also gets a JSON log format and an INFO log level in its logging settings.

Group	Resources in the template
Error filters	AskErrorsFilter, TicketsErrorsFilter, TriageErrorsFilter, SigninErrorsFilter
Tickets filters	TicketsModelRequestsFilter, TicketsInputTokensFilter, TicketsOutputTokensFilter, TicketsCreatedFilter
Triage filters	TriageModelRequestsFilter, TriageInputTokensFilter, TriageOutputTokensFilter
Alarms	ErrorsAlarm, ApiServerErrorsAlarm
Dashboard	HelpdeskDashboard

The bootstrap template deploy-role.yaml gains the permissions the new pieces need, so redeploy it before the first run.

Practice: what do you do?

Decide each one before reading the call: what you do.

Problem	Our call and reason
A Logs Insights query shows route, outcome and role as empty columns	The log format is probably not JSON, so the filters match nothing: check the function’s log setting
Cost per ticket looks lower than the bill suggests	/ask logs no tokens and other costs may be missing: say what the number leaves out
You test the alarm and no email arrives	Confirm the email subscription from chapter 6, then test again
The quality check fails in staging	Read which question failed and why; prod waits until it passes
A colleague asks you to quote today’s price per token	Read the pricing page today; do not quote from memory
You finish the chapter	Delete prod, staging, then github-bootstrap, and run the three checks

Flashcards

Say the answer aloud before revealing it. Speaking is a different skill from reading, so practise both.

1. What is a structured log line, and why does the helpdesk write one?

It writes the facts of one request as labelled fields in JSON, so a tool can filter them and add them up. Like a till receipt with an item column and a price column. The helpdesk writes one per request, with numbers and words about the request and no private text.

2. What does the summary line leave out, and why?

It leaves out what anyone wrote: the message, the answer, a ticket title, an email address and a user id. It keeps only the route, outcome, role and numbers. Logs are read by more people than chat and they last, so private text stays out.

3. What does it mean if a Logs Insights query shows empty columns for route, outcome and role?

The log format is probably not JSON, so the fields are not parsed. The metric filters would then match nothing, so the first check is the function's log format setting in the template.

4. What are the two places to read tokens, and which route is in neither per-request total?

The log lines hold the tokens the model reported per request for tickets and triage; Bedrock's own metrics count model calls and their tokens for the account, by model. The /ask route is not in the logged totals, because the knowledge-base answers report no tokens to our code.

5. What is cost per ticket?

What the helpdesk cost to run over a period, divided by the tickets created in that period. Like a sandwich shop dividing what it spent making all the sandwiches by how many it made. With no tickets there is no answer.

6. Write the formula and say where each number comes from.

Cost per ticket is total cost over tickets, where total cost is input tokens over per times the input price, plus output tokens over per times the output price, plus other costs. Tokens and tickets come from Logs Insights; prices and the per unit from the pricing page, read today; other costs you add up.

7. Why does this course state no price?

Prices change and differ by model, and the pricing page quotes some per 1,000 tokens and others per 1M tokens. You read the current price on the day and note the date, and the command refuses to run without every price.

8. What is a metric filter, and which five metrics do ours write?

A rule that watches a log group for lines matching a pattern and adds one to a metric each time one does, like tally marks on a whiteboard. Ours write Errors, ModelRequests, InputTokens, OutputTokens and TicketsCreated under the name Helpdesk/environment.

9. Trace a log line to an email.

A function writes a JSON line, the log group keeps it, a metric filter counts matching lines into a metric, an alarm checks the metric, and when it trips it writes to the SNS topic from chapter 6, which emails the team. A confirmed subscription is needed for the email to arrive.

10. How do you test the alarm without breaking anything, and what if no email arrives?

Run set-alarm-state to set the errors alarm to ALARM by hand; the alarm returns to its real state by itself, often within seconds. If no email arrives, check that the email subscription from chapter 6 is confirmed, then the junk folder.

11. What does the quality check ask, and what does it prove and not prove?

Three questions that must cite their policy page and one off-topic question that must be blocked. It proves the function, knowledge base, model and guardrail work together after a deploy, because it calls the ask function directly as a made-up employee. It does not test sign-in.

12. In what order do you delete the stacks, and how do you prove the account is clean?

Prod, staging, then github-bootstrap last, emptying each site bucket first and waiting for each delete. Then run three checks that must print nothing: a stack list filtered by status, the alarms with the helpdesk- prefix, and the log groups with the /aws/lambda/helpdesk- prefix.

Model answer

The core question this chapter answers: “How do you know the helpdesk is healthy, what does a ticket cost, and how do you remove it all?”

Five things a manager listens for, in this order: log line · dashboard and alarm · quality check · cost per ticket · teardown. Skip any of these five and the answer is incomplete.

1. Log line — one JSON line per request, with no private text; filters turn lines into counts
2. Dashboard and alarm — the counts are drawn, and an alarm emails the team
3. Quality check — fixed questions after each deploy; a bad shape stops the run before prod
4. Cost per ticket — a formula with prices read live, and counts from the logs
5. Teardown — prod, staging, then the bootstrap stack; three checks print nothing

The full answer, said aloud:

(Log line) “Each function writes one JSON line per request: the route, how it ended, the role, the time and the tokens, and nothing anyone wrote. Metric filters count those lines into errors, tokens and tickets.

(Dashboard and alarm) A dashboard draws those counts on one page. Two alarms, one on our errors and one on the API’s server errors, email the team through the topic from chapter 6, once the subscription is confirmed. I test that by setting an alarm by hand.

(Quality check) After each deploy a fixed list of questions runs against the ask function as a made-up employee. A wrong shape stops the run before prod. It does not test sign-in.

(Cost per ticket) Cost per ticket is the tokens at their prices plus other costs, over the tickets for one period. I read the prices on the pricing page that day, and say that /ask logs no tokens.

(Teardown) I delete prod, then staging, then the bootstrap stack, emptying the buckets first, and run three checks that print nothing.“

Missing a point is a failure state. The follow-up question lands where you went quiet: finish without the alarm and the next question is who hears when it breaks; finish without the cost and the next question is what a ticket costs; finish without the teardown and the next question is how you know nothing is still running. Practise until the five come out without prompting, then stop talking and let the questions come.

Why this matters

In a conversation with a manager: a helpdesk that answers questions is asked whether it is healthy and what it costs. The five points answer both, and the helpdesk is a specific example.

On the job: an engineer on call, a finance owner watching the model bill, an IT lead who needs to be told, and a reviewer approving a release ask versions of the same questions. The alarm, the formula and the quality check answer them.

In the rest of this course: this is the last build chapter. Chapter 10 is the roadmap: how to keep learning, how to explain what you built, and what to do next. A reader who knows these eight parts can say what each piece of a running assistant is for.

After this unit

Assignment. The chapter project asks you to deploy, read your logs, test the alarm, run the quality check, work out a cost per ticket from your own totals and today’s prices, and delete every stack. The brief itself is the project file. Keep your repository: there is nothing more to build.

What’s next. The next chapter is the roadmap: how to keep learning this, how to explain what you built, and your next steps.