Student Notes — Chapter 2: The helpdesk’s first answer: a hosted model behind an API

AWS in Action: AI edition · Chapter 2

Before you start

Nothing is needed to read this chapter or to run its offline checks beyond Python and the development tools listed in the course repository’s requirements-dev.txt. The offline checks use a fake model, so they need no AWS account and cost nothing. The live steps (deploying the stack and asking the real model) need an AWS account of your own; they create real resources and cost money for the requests you send, so you delete the stack at the end. You can follow these steps from the console, plus CloudShell, a terminal in your browser, for the one step that has no console route (sending a question with curl), so there is nothing to install. Sign in as an IAM or IAM Identity Center user, not the root user: as the root user, a step can stop with AccessDeniedException ... :root is not authorized. Check these first: the top bar of the console shows your user name; its Region is one the template lists (RegionRoute); your budget alert is listed under Budgets; Amazon Nova Lite is shown in the Bedrock Model catalog; and chapters/02-ask-endpoint/template.yaml is on your own computer, because the console uploads it from there. The one-time setup is in SETUP.md at the top of the course repository, and this chapter’s own table is in chapters/02-ask-endpoint/README.md under “Before you try”. The same steps as commands are in that README under Deploy, Try it and Delete. Have paper or a notes app open for the Flashcards and the Model answer.

Three ways to follow this chapter, and you can mix them:

• Watch: read the chapter and follow the build on screen. Nothing needed.
• Copy and run: copy the folder chapters/02-ask-endpoint/ and run its checks.
• Build with AI: open chapters/02-ask-endpoint/prompts/, run each prompt in an AI coding tool, then run that prompt’s check. The checks for the whole chapter are in chapters/02-ask-endpoint/prompts/CHECK.md.

Summary

This chapter gives the helpdesk its first real answer: an employee’s question goes to one address, a small Python function checks it and asks a hosted model, and the answer comes back. You pick a model from the AWS documentation, call it from an AWS Lambda function through the Converse API, and put the function behind an HTTP endpoint so that POST /ask works. You then describe the whole thing in one CloudFormation template, so that the same stack of resources can be created and deleted in a single step. You test it with curl, read the function’s logs, and delete the stack so nothing keeps costing money. Because the address is public until sign-in arrives in chapter 7, the build also caps the rate of requests and the length of a question.

Key concepts

Term	Definition
API	A doorway one program uses to ask another program to do something, in an agreed format.
Endpoint	One address on an API. Ours is POST /ask: send a question to it, get an answer back.
JSON	A small text format of labelled values, such as {"question": "..."}. The request and the reply are both JSON.
AWS Lambda / Lambda function	A service that runs your code when something calls it, with AWS managing the servers. A Lambda function is the code; ours is one Python function named handler.
Converse API	Amazon Bedrock’s single way to send a conversation to a model and read the reply, in the same shape across models that support it.
System prompt	The instructions you give a model before the user’s text, such as “you are the IT helpdesk assistant and answer in at most five short steps”.
API Gateway (HTTP API)	The AWS service that gives a function a public address and passes each request to it.
Template	A text file that describes the AWS resources you want. CloudFormation uses it as a blueprint.
Stack	The set of resources CloudFormation builds from a template, created, updated and deleted together as one unit.
IAM role	A set of permissions that a service, here the function, acts under. Ours lets the function write its own logs and call one named model.
Request cap (throttling)	A limit on how many requests an address accepts: ours is a burst of 5 and a steady rate of 1 per second.
CloudWatch Logs	Where AWS keeps the lines your function prints, in a log group named after the function.

Explanations

Orientation: where this chapter sits

Chapter 1 was the plan of the house. This chapter lays the first room. The finished helpdesk has a chat page calling a helpdesk API in front of a hosted model, with a knowledge base, ticket tools, a triage agent and sign-in around it:

Today you build the middle: the helpdesk API and its call to the hosted model. The chat page is chapter 3.

Try this first: an employee’s question has to reach a model hosted on AWS. Do you make the call from the employee’s browser with the access key inside the page, from a small function you run behind one address, or by an engineer pasting each question by hand? Pick one and say why. The chapter builds one of them.

By the end you can:

• explain how a question travels from an employee to a hosted model and back;
• read the handler and the template and say what each piece is for;
• create the stack, ask it a question, read its logs and delete it.

Part 1. The goal: POST /ask answers

You order food at a takeaway window. You say what you want in the agreed way, and the same kind of thing comes back. You do not walk into the kitchen. A form with fixed boxes works the same way: the question goes in the “question” box, so the receiver knows where to look.

An API is a doorway like that: one program uses it to ask another program to do something, in an agreed format. An endpoint is one address on the doorway. The helpdesk’s endpoint is POST /ask. POST means “send something to it” and /ask is the address. You send a question as JSON, a small text format of labelled values, and you get an answer back as JSON:

POST /ask
{"question": "my laptop won't connect to VPN"}

The reply is {"answer": "..."} with the text of the model’s answer. The rest of the chapter builds what sits behind that address.

The code for this chapter is in the course repository under chapters/02-ask-endpoint/: the function is chapters/02-ask-endpoint/app/ask/handler.py, its tests are chapters/02-ask-endpoint/tests/test_ask.py, and the stack description is chapters/02-ask-endpoint/template.yaml. In the repository there is also a prompts/ folder with one prompt per build step and a check for each. The prompt for the function is chapters/02-ask-endpoint/prompts/01-handler.md. A prompt is finished when its check passes.

Part 2. Pick a model

A model lives with the cloud provider, like a specialist you hire by the hour rather than a tool you own. You choose one from the provider’s current list and then call it by its id. Four facts decide the choice for this chapter, taken from the AWS documentation:

Question	What the documentation says
Which model does this chapter use?	Amazon Nova Lite, id amazon.nova-lite-v1:0: a low-cost model that answers quickly, reachable in most Regions.
Do I switch access on first?	Access to many Amazon Bedrock models is on by default. Amazon’s own models are not sold through the AWS Marketplace and need no Marketplace step.
What if I choose a Claude model?	Anthropic models need a one-time first-use form before the first call.
Where do I read a model’s id?	On the model’s page in the Bedrock documentation, which also shows an example call.

A text-only, low-cost model suits short helpdesk answers. Think of choosing a courier: the cheapest fast one is fine for a letter, and you pay for a bigger one when the job needs it. The id is kept in one place, a template parameter called ModelId, so changing the model is a one-line change and not a rewrite. Costs depend on the model and the number of requests; read the current figure from the provider’s pricing page when it matters, and chapter 9 adds cost watching.

Part 3. Call the model from a function

A cook is called in when an order arrives. The kitchen is not theirs to run, and you supply the recipe. A vending machine is another picture: one request goes in and one response comes out. AWS Lambda works like that. It runs your code when something calls it, and AWS manages the servers underneath. A Lambda function is that code. Ours is one Python function named handler, in chapters/02-ask-endpoint/app/ask/handler.py.

First, two failures the handler is built to survive. The requests and replies are what the chapter’s tests check.

Situation 1: the empty question. An employee presses Send on an empty box.

POST /ask   body: {"question": ""}
Reply:      400  {"error": "Send JSON like {\"question\": \"...\"}"}
Model:      not called

If the handler passed this to the model, you would pay for a reply to nothing. Cause: nothing checks the input. The handler checks first and refuses a missing, blank or non-text question, and one that is too long, with status 400:

    if not isinstance(question, str) or not question.strip():
        return reply(400, {"error": 'Send JSON like {"question": "..."}'})
    if len(question) > MAX_QUESTION:
        return reply(400, {"error": f"Keep the question under {MAX_QUESTION} characters"})

The cap on length exists because the address is public until sign-in arrives in chapter 7, and long input costs money.

Situation 2: the model is down. The call to the model fails.

POST /ask   body: {"question": "How do I reset my password?"}
Reply:      502  {"error": "The assistant is unavailable. Try again."}
Log:        model call failed  (the full error stays in the log)

Without care the employee would see an internal error message, which helps nobody on the desk. Cause: the failure was not caught. The handler catches it, writes the detail to the log where an engineer can read it, and returns status 502 with a short safe message.

Now the call itself. The Converse API is Amazon Bedrock’s one way to send a conversation to a model and read the reply, in the same shape across models, like a standard order form that any kitchen accepts:

        response = bedrock.converse(
            modelId=MODEL_ID,
            system=[{"text": SYSTEM_PROMPT}],
            messages=[{"role": "user", "content": [{"text": question}]}],
            inferenceConfig={"maxTokens": 400, "temperature": 0.2},
        )
        answer = response["output"]["message"]["content"][0]["text"]

system holds the instructions (the system prompt); messages holds the employee’s question. inferenceConfig holds settings that are common to all models: maxTokens caps the length of the reply in tokens, and temperature sets how varied the wording is, where a low value keeps answers steady. The reply comes back in a standard envelope, so the text is read from output, then message, then content, and does not depend on a model-specific field. The model id comes from the MODEL_ID environment variable, so changing the model does not mean rewriting the code.

The handler is tested with a fake model. chapters/02-ask-endpoint/tests/test_ask.py replaces the model client with one that records what it was sent and returns a canned answer, or raises an error. So pytest -q checks the good question, the empty and oversized questions (the model is not called) and the model failure, with no AWS account and no cost.

You can also build the handler with an AI coding tool. The prompt in chapters/02-ask-endpoint/prompts/01-handler.md begins:

Create app/ask/handler.py, a Python AWS Lambda handler named `handler` for POST /ask.

- Read the model id from the MODEL_ID environment variable.

and its check is:

python -c "import ast, pathlib; ast.parse(pathlib.Path('app/ask/handler.py').read_text())"
grep -n "MAX_QUESTION" app/ask/handler.py

If a check fails, give the failure to the tool and run the prompt again, and read the file it wrote before you trust it.

Part 4. Put the function behind an address

A reception desk takes each visitor and passes them to the right person; the visitor does not wander the building. API Gateway does that for a function: it is the AWS service that gives the function a public address and passes each request to it. A telephone switchboard is another picture: it connects the call, and the conversation happens elsewhere.

The path of one request:

curl  -->  HTTP API    -->  Lambda function  -->  Bedrock model
           POST /ask        handler.py            Nova Lite 

curl  <--  {"answer": "..."}  <------------------  the reply

With an HTTP API, the request body reaches the function as text in the body field of the event, which is why the handler reads the body and parses it as JSON. The function returns a status code, headers and a body, and API Gateway sends them back to the caller. API Gateway also needs permission to call the function, and the template grants it. So the call to the model is made in the function, behind one address.

Here is the failure first.

Situation 3: the pasted address. Someone pastes the helpdesk address into a staff chat, and by evening a script is calling it in a loop (illustrative):

Staff chat, Friday (illustrative):
  Someone pastes the helpdesk address into the channel.
  By evening a script is calling it in a loop.

Anyone with the address can send questions over the weekend, and the answers cost money. Cause: the address is open until sign-in is added, and nothing limits how often it is called. The template sets a request cap on the stage, which publishes the API:

  DefaultStage:
    Type: AWS::ApiGatewayV2::Stage
    Properties:
      ApiId: !Ref HelpdeskApi
      StageName: $default
      AutoDeploy: true
      # Public until sign-in (ch 7): cap requests so a leaked URL cannot run up the bill
      DefaultRouteSettings:
        ThrottlingBurstLimit: 5
        ThrottlingRateLimit: 1

Think of a turnstile that lets one person through at a time with a short burst allowed, or a queue at a counter. The stage allows a burst of 5 requests and a steady rate of 1 per second. Together with the 1000-character limit in the handler, this limits the damage. It is a cap, not a lock: it does not stop a caller. The lock is sign-in, in chapter 7.

Part 5. Create it all from one template

A blueprint describes a building, and the building is made from it. If the building is knocked down, the blueprint is still there. A recipe works the same way: it makes the same dish whenever you cook it. A template is a text file that describes the AWS resources you want; CloudFormation uses it as a blueprint. A stack is the set of resources CloudFormation builds from it, managed as one unit: created, updated and deleted together. A template can take parameters, such as the model id, whose values are chosen when the stack is created.

Our template, chapters/02-ask-endpoint/template.yaml, contains these pieces:

Piece	What it is for	Name in the template
Log group	Where the function’s printed lines are kept	AskFunctionLogGroup
Role	The permissions the function acts under: write its own logs, call one named model	AskFunctionRole
Function	The handler code, written inline	AskFunction
HTTP API and route	The address and POST /ask	HelpdeskApi, AskRoute
Integration	Connects the route to the function	AskIntegration
Stage	Publishes the API and holds the request cap	DefaultStage
Invoke permission	Lets API Gateway call the function	AskInvokePermission

The role is an IAM role: a short list of actions the function is allowed to take. Think of a staff badge that opens two doors. Ours lets the function write its own logs and call the single model named by ModelId. The model permission is written against that one foundation model, built from the partition and region with no account number in the file. Calling the model through Converse needs permission for the bedrock:InvokeModel action, and the policy does not list a separate Converse action.

The handler code is placed inline in the template, so one file creates the whole stack. For Python, CloudFormation writes inline code to a file named index, so the template’s handler setting is index.handler. The template pins a generally available Python runtime, python3.13, and uses the AWS SDK that the Lambda Python runtime already includes, so no package is uploaded. Resources that can carry tags are tagged chapter: session-02, so you can find what this chapter made. The template has no Retain deletion policy, which matters in Part 7.

You can also build the template with an AI coding tool. The prompts chapters/02-ask-endpoint/prompts/03-template.md and chapters/02-ask-endpoint/prompts/04-endpoint.md build it in two steps, and cfn-lint template.yaml is their check.

Part 6. Test it and read the logs

Costs money. The live steps create real resources and call a real model. Deploy, ask and delete in one sitting.

aws cloudformation deploy --stack-name helpdesk-dev-ch2 --template-file template.yaml --capabilities CAPABILITY_IAM --parameter-overrides Project=helpdesk Environment=dev
API_URL=$(aws cloudformation describe-stacks --stack-name helpdesk-dev-ch2 --query "Stacks[0].Outputs[?OutputKey=='AskUrl'].OutputValue" --output text)
curl -s -X POST "$API_URL" -H "content-type: application/json" -d '{"question": "How do I reset my password?"}'

The first command creates the stack when the name is new, and updates it when the name already exists. It needs --capabilities CAPABILITY_IAM because the template creates a role; without it, the command returns an error. The second reads the address from the stack’s output. The third sends one question and prints the answer. A second request with "question": "" returns status 400 with an error message, which is the check that the handler’s first guard works. These commands are also in chapters/02-ask-endpoint/README.md, and the whole live check is in chapters/02-ask-endpoint/prompts/CHECK.md.

A ship’s logbook records what happened on the voyage; a till roll records each sale. CloudWatch Logs does that for the function: it is where AWS keeps the lines the function prints. By default, Lambda writes to a log group named /aws/lambda/<function-name>. Ours is created by the template with the same name pattern and a short retention. To read the last few minutes:

aws logs tail /aws/lambda/helpdesk-dev-ask --since 5m

You will see a line such as this for each answered question:

answered a question of 27 characters

The log says a question arrived and how long it was, not what it said. If the model call failed, the full error is in the log, even though the employee saw a safe message.

Where did that answer come from?

Read the answer you got again. It reads well, but nothing in this chapter gave the model a single company page. It answered from what it already knows in general, much like a new temp on their first day who has not yet seen your office: helpful about VPNs in general, blind to yours.

The employee asked	What the model can draw on	What it cannot know
“My VPN won’t connect”	Common fixes for VPNs in general	Which VPN product your company runs, its settings, your support page
“How do I reset my password?”	Typical reset steps	Your sign-in portal and your password rules

If your company uses a different VPN product from the one the model has in mind, the answer names menus and settings your company does not have, in the same sure tone. That is chapter 1’s made-up answer, happening in your own stack.

Save this answer: it is your baseline. In chapter 4 you send the same question again. That time the helpdesk first finds the matching page in the company’s own IT pages and hands it to the model with the question, and the model answers from that page and names it. Finding the page first and then writing the answer is called retrieve-and-generate (RAG). Put the two answers side by side and you can see what it changes.

Part 7. Delete it and confirm

A hotel guest checks out and the room is cleared; a room left occupied keeps being charged. A stack you do not delete keeps costing money.

aws cloudformation delete-stack --stack-name helpdesk-dev-ch2
aws cloudformation wait stack-delete-complete --stack-name helpdesk-dev-ch2

When a stack is deleted, CloudFormation deletes all of its resources unless a deletion policy keeps specific ones. A resource with no deletion policy is deleted by default, and a resource set to Retain would outlive the stack and keep incurring charges until you delete it. Our template has no Retain, so the whole stack goes. A delete cannot be stopped once it has begun. The wait command checks every 30 seconds until the stack’s status is DELETE_COMPLETE. Deleted stacks are hidden from the console’s stack list by default, so a list that no longer shows helpdesk-dev-ch2 confirms it. A delete can fail if a resource will not delete; the stack then shows a failed-delete state, and you read the reason and try again.

What the first answer leaves open, and where the course fixes it:

What is still open	What the helpdesk will do	Built in
Anyone with the address can call it	Sign-in and roles	Chapter 7
Employees need a page, not curl	A chat page that shows each reply whole	Chapter 3
It can make things up	Answer from the company’s own pages, cited	Chapter 4
It cannot act	Let tools create and read tickets	Chapter 5
Nobody watches cost or errors	Logs, cost and an alarm	Chapter 9

Practice: which piece handles it?

Decide each one before reading the call: which piece handles it, and what the employee sees.

Problem	Our call and reason
An employee sends an empty question	The handler refuses it with 400 and does not call the model
The hosted model refuses the call	The handler logs the failure and returns 502 with a safe message
A pasted address is called in a loop	The stage’s request cap limits it; sign-in in chapter 7 stops it
You finish the chapter and leave the stack running	It keeps costing money: delete the stack and wait
You want to swap in a different model	Update the stack with another ModelId; the role follows it. Editing the function’s MODEL_ID alone is refused, because the role allows only the stack’s model. An Anthropic model needs its one-time form first

Flashcards

Say the answer aloud before revealing it. Speaking is a different skill from reading, so practise both.

1. What are an API and an endpoint?

An API is a doorway one program uses to ask another program to do something in an agreed format, like a takeaway window. An endpoint is one address on it; ours is POST /ask, which takes a question and returns an answer.

2. Why does the call to the model happen in a function and not in the employee's browser?

A page's contents can be read by anyone who opens it, so a key inside it is no longer secret. A function behind one address keeps the permission on the server side, where you can limit what it may do and how often it is called.

3. What is AWS Lambda, in plain words?

It runs your code when something calls it, and AWS manages the servers underneath. Like a cook called in when an order arrives, you supply the recipe, which here is one Python function named handler.

4. What does the Converse API give you?

One way to send a conversation to a model and read the reply, in the same shape across models that support it. You pass the model id, the system instructions, the user's message and settings such as maxTokens, and read the text from the output.

5. What does the handler do before it calls the model, and why?

It refuses a missing, blank, non-text or over-long question with status 400. A model call costs money and the address is public until sign-in, so a bad question should not reach the model.

6. What happens when the model call fails, and what does the employee see?

The handler catches the failure, logs the full error and returns status 502 with a short safe message. The employee sees that the assistant is unavailable, and the detail stays in the log for an engineer.

7. What does API Gateway do for the function?

It gives the function a public address and passes each request to it, then sends the function's reply back to the caller. It does the work of a reception desk and needs permission to call the function.

8. What is the request cap, and what is it not?

The stage allows a burst of 5 requests and a steady rate of 1 per second, which limits how much a pasted address can cost. It is a cap, not a lock: it does not stop a caller, and sign-in in chapter 7 does that.

9. What are a template and a stack?

A template is a text file that describes the AWS resources you want, like a blueprint. A stack is the set of resources built from it, created and deleted together as one unit, like the building made from the blueprint.

10. What may the function's role do?

It may write its own logs and call the one model named by the ModelId parameter. Like a staff badge that opens two doors, it limits what the function could do if its code were ever misused.

11. Why read the logs, and what does ours record?

CloudWatch Logs keeps the lines the function prints. Ours records that a question arrived and its length, not its text, and holds the full error when a model call fails.

12. How do you delete the chapter's work, and how do you know it is gone?

Run delete-stack and then wait stack-delete-complete, which polls until the stack is gone. The template has no Retain, so its resources go with the stack, and the console list no longer shows it.

Model answer

The core question this chapter answers: “An employee types a question into the helpdesk. How does it reach a hosted model and come back, and how do you build it and remove it cleanly?”

Four things a colleague listens for, in this order: front door · function · template · clean-up. Skip any of these four and the answer is incomplete.

1. Front door — an HTTP endpoint passes the request to the function
2. Function — what it checks, how it calls the model, what it returns
3. Template — how one file builds the whole stack, with its limits
4. Clean-up — how you delete it and know it is gone

The full answer, said aloud:

(Front door) “The employee’s question goes to an HTTP endpoint, POST /ask, which API Gateway puts in front of a function and which passes each request on.

(Function) The Lambda function checks the question first: a blank or oversized one is refused with a 400 and does not reach the model. Then it calls the model through the Converse API with a system prompt and the question, and returns the answer. If the model call fails, it logs the error and returns a safe 502.

(Template) One CloudFormation template creates the function, a log group, a role that may write its logs and call one named model, and the endpoint with a request cap, together as one stack. The cap limits a pasted address; sign-in comes later.

(Clean-up) When I am done I delete the stack and wait until it is gone, so nothing keeps costing money.“

Missing a point is a failure state. The follow-up question lands where you went quiet: finish without the clean-up and the next question is what happens to the cost, and finish without the template and the next question is how you would build it again. Practise until the four come out without prompting, then stop talking and let the questions come.

Why this matters

In a conversation with a colleague: a team wiring a model into an internal tool wants someone who can say plainly how a question reaches it, what stops it running up a bill, and how it is removed. The four points leave no obvious gap, and the helpdesk is a specific example.

On the job: a developer connecting a model to an application, a security review of a new endpoint, and an engineer tearing down a test environment ask versions of the same questions. The handler, the role, the request cap and the delete answer them.

In the rest of this course: the build chapters that follow extend this template and end with a delete. A reader who knows these seven parts can predict why each later piece exists.

After this unit

Assignment. The chapter project asks you to deploy the stack, ask the helpdesk your own questions, read the logs and delete it. Keep the template: the next chapter extends it. The brief itself is the project file.

What’s next. The next chapter gives employees a chat page with replies shown whole, served from a private S3 bucket through CloudFront, extending this chapter’s template.