Chapter 7 — Fine-Tuning, Retrieval, or Prompting

Course: Getting Started with AI

Prerequisites: You can read and run a short Python script. From earlier chapters and from Build and Evaluate RAG Applications you need three ideas, each restated in a sentence in the notes before it is used: a prompt is the text a model reads before it answers (and its tokens are paid for on every request), a retrieval pipeline looks facts up in your documents and puts them in the prompt, and a failing question can be traced to the stage that lost its answer. Fine-tuning, context problem, behaviour problem, few-shot examples and break-even are defined in the notes before they are used.

What the reader can do after this chapter:

• Sort a failing question into a knowledge problem or a behaviour problem, and name the lever (prompt, retrieval, a tool call, examples in the prompt, or fine-tuning) that matches it.
• Compute, from stated token counts and prices, how many requests it takes before a one-off fine-tuning cost is repaid by shorter prompts.
• Defend a choice between the three approaches in a design review by naming what it buys, what it costs to maintain and what evidence would change the choice.

Core question this chapter answers: When a model’s answers are wrong, how do you decide whether to change the prompt, the information the model is given, or the model itself, and what does each choice cost to keep working?

Connections:

• Builds on: the failure-diagnosis chapter of Build and Evaluate RAG Applications, which diagnosed six failing questions and mapped each to a repair. This chapter opens with the causes.tsv file your project wrote there, and asks which of those repairs a change to the model itself would have made.
• Leads into: Building AI Agents, which lets a model call tools. It opens with the tickets this chapter’s project marks as needing live data, which no document and no training run can supply.

Your answers are scored

This chapter is assessed. The quiz is marked against a key held separately from your materials. Questions 1–8 are multiple choice and are marked by comparing your letter with the key. Questions 9–10 ask for a short answer in an exact format, and each one is marked by a rule printed with the question, so you can check that your answer has the right shape before you submit it.

• Nothing is judged by reading it for style. An answer either matches the rule or it does not.
• All three project tasks are scored from an evidence file of key: value lines. Every line’s accepted value, or the rule that decides it, follows from what the task prints, so you can check your own file before you submit it.

Getting Started with AI — Chapter 7: Fine-Tuning, Retrieval, or Prompting

Before you start

Nothing to install for the reading. For the project you need Python 3 and nothing beyond its standard library. You do not need an account with any model provider or an API key: the project sorts tickets with a printed rule and does arithmetic on printed prices, and never calls a model.

Keep one question in mind while you read: is this answer wrong because the model was not given something, or because it does not behave the way I need? By the end of the chapter you should be able to say which, name the change that fits, and say what that change costs to keep running.

Summary

When a model’s answers are wrong, there are three changes to try: change the prompt, change the information the model is given, or change the model itself by training it further on examples. They are not steps on one ladder, because they fix different things. Missing, outdated or private knowledge is a problem of what the model was given, and the changes that fit are retrieval from documents or a call to a live system. Inconsistent format, tone or reasoning is a problem of how the model behaves, and the changes that fit are examples in the prompt or, when many examples are needed, fine-tuning. Each change has a maintenance cost: tokens paid on every request, a pipeline to keep current, or a training set to prepare and a model to retrain. A defensible choice names the kind of problem, the lever that fits it, the cost of keeping that lever working, and the evidence that would change the choice.

Key concepts

Term	Definition
Prompt engineering	Changing the text sent to a model (its instructions, context or examples) to change what it returns, without changing the model.
Few-shot examples	A small number of worked examples of the task, placed in the prompt, so that the model sees the behaviour wanted instead of only being told.
Retrieval-augmented generation (RAG)	Looking facts up in a store of documents at request time and putting them in the prompt, so that the model answers from them (covered in Build and Evaluate RAG Applications).
Fine-tuning	Continuing the training of a model on a dataset of examples of the task, so that the behaviour is built into the model’s weights instead of the prompt.
Context problem	A failure caused by what the model was given: the needed fact was missing, out of date or private. It is repaired by changing the context.
Behaviour problem	A failure in how the model acts although it had the facts: format, tone or reasoning that is wrong or inconsistent. It is repaired by changing instructions, examples or the model.
Learned memory	Knowledge or behaviour the model picks up from many examples during training.
In-context memory	Information placed in the prompt for one request, which the model uses and then forgets.
Hold-out set	Examples kept out of the training data and used only to measure the trained model.
Break-even	The number of requests at which a change with a one-off cost, repaid by cheaper requests, has cost no more than the alternative.
Tool call	A request, made by the model, for a system outside the model to do something or fetch something, such as a live order status (taught in Building AI Agents).
Evaluation set	A set of questions with their expected answers, used to measure each change (the gold set and the cases of Build and Evaluate RAG Applications).

Explanations

The failure, before the explanation

Here is a proposal from a team meeting about the policy assistant of the retrieval chapters of Build and Evaluate RAG Applications.

Proposal: “The assistant keeps giving out-of-date policy answers and gets the house tone wrong. Let’s fine-tune the model on the handbook and on our old support replies. One training run fixes both.”

Two tickets are folded into one plan. After the training run, the tone improves: the replies now sound like the old support replies. The policy answers do not. The handbook is edited the following week, and the assistant keeps repeating the old rule, because the old rule is now part of the model, and no chunk, no version number and no citation exists to check. The team ran one expensive change against two different problems, and it fixed the one it suits.

An engineer who has fixed bugs will want one root cause for one symptom, and here there were two. To see them apart you need a way to sort failures by kind.

Two kinds of problem: what the model has, and how the model acts

One provider’s guide on improving accuracy starts by rejecting the idea of a single ladder. Many guides, it says, “paint it as a simple linear flow - you start with prompt engineering, then you move on to retrieval-augmented generation, then fine-tuning. However, this is often not the case - these are all levers that solve different things, and to optimize in the right direction you need to pull the right lever” (OpenAI, Optimizing LLM Accuracy). It names two axes. Context optimization is for when “the model lacks contextual knowledge because it wasn’t in its training set, 2) its knowledge is out of date, or 3) it requires knowledge of proprietary information”, and it “maximizes response accuracy”. LLM optimization is for when “the model is producing inconsistent results with incorrect formatting, 2) the tone or style of speech is not correct, or 3) the reasoning is not being followed consistently”, and it “maximizes consistency of behavior”.

The guide gives an exam analogy for the same split. Two ways to get the right answer in an exam: attend class and see many worked examples, which is learned memory, solved in a model “by showing examples of the prompt and the response you expect”; or bring the textbook and look the answer up, which is in-context memory, solved by putting the relevant information in the prompt, “either in a static way using prompt engineering, or in an industrial way using RAG”.

A second way to see it: an apprentice and a reference manual. An apprentice who has watched a hundred repairs has absorbed a way of working, and cannot tell you today’s part price. A manual on the bench has today’s part price, and says nothing about how to hold the tool. When a repair goes wrong, the first question is which one was missing. Tone and format are the apprentice’s skills. A policy rule that changes is the manual’s page.

That sorts the proposal. The out-of-date policy answers are a context problem and the house tone is a behaviour problem. One training run is a behaviour fix, and it was aimed at both.

Prompting: the cheapest change, and the one that defines the target

Take the tone ticket. Before any training run, try telling the model the tone in the prompt and showing it two examples of a good reply. If that works, the ticket is closed with no pipeline and no training. The same guide says “prompt engineering is typically the best place to start”, and gives a reason: “it forces you to define what accuracy means for your use case”, because you start with an input and must judge whether the output matches what you expected, and “if it is not what you want, then the reasons why will show you what to use to drive further optimizations” (OpenAI, Optimizing LLM Accuracy). It recommends leaving this stage with an evaluation set: “if we have a set of 20+ questions and answers, and we have looked into the details of the failures and have a hypothesis of why they’re occurring, then we’ve got the right baseline”.

A second example: writing a failing test before you fix a bug. The test is the definition of “fixed”, and you have it before choosing an approach. A prompt and an evaluation set do the same job for a model.

Prompting has a cost and a limit. The cost is tokens. Instructions and examples travel with every request, and Chapter 2 priced tokens. The limit, in the guide’s words: “the biggest issue with prompt engineering is that it often doesn’t scale - we either need dynamic context to be fed to allow the model to deal with a wider range of problems than we can deal with through adding content to the context, or we need more consistent behavior than we can achieve with few-shot examples.” Those two limits point to the next two levers.

Retrieval and tools: when the model needs facts it was not given

Go back to the out-of-date policy answers. The right repair is the pipeline taught in Build and Evaluate RAG Applications: look the fact up in the current document, put it in the prompt, and cite it. That has two properties that training does not give. A fact placed in the prompt comes from a chunk you can show the reader, and a changed document is a re-ingest, not a training run. The original paper on retrieval-augmented generation names the underlying limitation of a model’s stored knowledge: pre-trained models’ “ability to access and precisely manipulate knowledge is still limited”, and “providing provenance for their decisions and updating their world knowledge remain open research problems” (Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks).

There is direct evidence on putting new facts into a model by training. One study compared unsupervised fine-tuning with retrieval across knowledge-intensive tasks and found that “unsupervised fine-tuning offers some improvement, RAG consistently outperforms it, both for existing knowledge encountered during training and entirely new knowledge”, and that “LLMs struggle to learn new factual information through unsupervised fine-tuning” (Ovadia et al., Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs). It adds that exposing the model to many variations of the same fact during training could alleviate this. That is a result for the models and tasks in that study; read it as a reason to measure before training on facts, not as a law.

Not every missing fact lives in a document. “What is the status of order 4412 right now?” and “how many units are in stock?” live in systems whose answers change from request to request, so any document would be out of date by the time it was indexed. The model needs to ask the system. That is a tool call, and Building AI Agents builds it. In this chapter’s project, a ticket whose needed fact is not in the prompt and lives in such a system is sorted to the tool lever, and one whose fact lives in documents that change rarely is sorted to retrieval.

The cost of the retrieval lever is a pipeline that has to be kept right: chunking, indexing and re-ingesting when documents change, and the failure modes Build and Evaluate RAG Applications lists. It also adds tokens to each request: the retrieved chunks.

Fine-tuning: when many examples are the only way to show the behaviour

Now return to the tone ticket and suppose the prompt, with its instruction and five examples, still does not hold the tone, and the team has two hundred good replies. This is the case for fine-tuning. The guide says it is “typically performed for one of two reasons”: to “improve model accuracy on a specific task: training the model on task-specific data to solve a learned memory problem by showing it many examples of that task being performed correctly”, or to “improve model efficiency: achieve the same accuracy for less tokens or by using a smaller model” (OpenAI, Optimizing LLM Accuracy). It also says what the work is: “the fine-tuning process begins by preparing a dataset of training examples - this is the most critical step, as your fine-tuning examples must exactly represent what the model will see in the real world”, and “we always recommend maintaining a hold-out set to use for evaluation”.

A second way to see it: moving a lookup table’s contents into compiled code. The values become part of the program. That is fast at run time and has nothing to look up, and a change to a value now means a rebuild. Fine-tuning moves a behaviour from the prompt into the model in the same way, and a change to the behaviour means another training run on a new dataset.

That is the cost of this lever. It needs a dataset prepared and checked, a hold-out set to measure with, and a repeat of both whenever the wanted behaviour changes. The guide’s own example of a typical flow ends with a retraining step: “re-train the fine-tuned model on the new training examples which include our enhanced RAG inputs”. Fine-tuning also does not repair what the prompt never contained: the model still cannot see a fact that was neither given to it nor learned.

The levers stack, and the arithmetic of one-off versus per-request cost

The guide says that learned memory and in-context memory, the two levers, “are additive, not exclusive - they stack, and some use cases will require you to use them together to use optimal performance”. A system can retrieve the policy text and also run a model tuned to the house tone. The decision is made per failure, not once per system.

Take the tone ticket again, and price two ways to fix it. These figures are invented for teaching, and not the price of any provider’s model. A request carries a 400-token base prompt. Option A puts 1,500 tokens of examples in every prompt, at 2 cost units per 1,000 input tokens: (400 + 1,500) × 2 ÷ 1,000 = 3.8 units per request. Option C fine-tunes: 6,000 units once, then no examples in the prompt, at 3 units per 1,000 tokens on the tuned model: 400 × 3 ÷ 1,000 = 1.2 units per request. Option A pays 3.8 on every request. Option C pays 1.2 on every request and 6,000 once, so C’s total is 6,000 + 1.2N against A’s 3.8N for N requests. They are equal near N = 2,308 requests, and past that point C is cheaper. At 1,000 requests, A costs 3,800 and C costs 7,200. At 5,000, A costs 19,000 and C costs 12,000.

Two things follow from the arithmetic. The cheaper option depends on how many requests there will be, which is a forecast, so state it and say what would change it. And the arithmetic compares only the costs it contains: it leaves out the effort of preparing and checking the dataset, the retraining when the behaviour changes, and the quality of the result. Compare options that fix the same failure: a retrieval change and a fine-tune fix different problems, and putting a price on each does not make them substitutes.

Three situations you will meet, and what they mean for your design

Before the table, three situations:

1. A product manager asks to train the model on the company’s price list so that it always quotes prices correctly. The price list changes whenever the business decides to change it.
2. A team’s assistant returns the right facts every time, but a downstream system rejects a quarter of the replies because they do not follow the required layout.
3. A support assistant must quote the current delivery status of an order. The team has indexed the shipping policy documents and the assistant still guesses.

What you see	What kind of problem	What to do instead
Facts that change, and a request to train on them (situation 1)	Context: the fact must come from a current source	Retrieve from the current list, and cite it; train only if the problem that remains is behaviour
Right facts, wrong layout (situation 2)	Behaviour: instructions or examples, then fine-tuning if many examples are needed	Add the layout and a few examples to the prompt, measure, and consider fine-tuning only if that does not hold
A live status the documents cannot hold (situation 3)	Context from a live system	A tool call that asks the system, which Building AI Agents builds

Flashcards

Say the answer aloud before revealing it. Speaking it is what exposes the gaps that reading past a written answer hides.

What are the two kinds of problem that decide which lever to use?

A context problem, where the model lacked a fact because it was missing, out of date or private, and a behaviour problem, where the facts were there but the format, tone or reasoning was wrong or inconsistent. Context optimization maximises accuracy; changing the model maximises consistency of behaviour.

Why is it wrong to see prompting, RAG and fine-tuning as one ladder?

They solve different things. One provider’s guide says they are levers that solve different things, and you need to pull the right one. Retrieval supplies facts; fine-tuning changes behaviour; neither is a later version of the other.

What are learned memory and in-context memory?

Learned memory is what a model absorbs from many examples in training, like attending class. In-context memory is information placed in the prompt for one request, like having the textbook open. The first is changed by training, the second by changing the prompt or retrieval.

Why start with prompt engineering?

It is the cheapest change, and it forces you to define what a right answer is: you must judge each output. Its by-product is an evaluation set of questions with expected answers, which later changes are measured against. One guide suggests 20 or more questions.

What does one provider say limits prompt engineering?

That it often does not scale: you either need dynamic context to handle a wider range of problems than can be put in the prompt, or more consistent behaviour than few-shot examples can reach. Those two limits point to retrieval and to fine-tuning.

What did one study find when comparing fine-tuning and retrieval for new facts?

That unsupervised fine-tuning gave some improvement but retrieval consistently outperformed it, for existing and for new knowledge, and that the models struggled to learn new facts through unsupervised fine-tuning. It is a result for that study’s models and tasks.

Why does a changed document favour retrieval over fine-tuning?

With retrieval, a changed document is a re-ingest and the answer cites the current chunk. With fine-tuning, the old fact is inside the model, so a change means another training run, and there is no chunk to check.

Which kind of fact calls for a tool instead of retrieval?

A fact in a system that changes from request to request, such as an order status or a stock count. Any document would be out of date by the time it was indexed, so the model has to ask the system, which is a tool call.

What does the guide call the most critical step of fine-tuning, and what should be kept aside?

Preparing the dataset of training examples, which must exactly represent what the model will see in the real world. A hold-out set should be kept aside, never trained on, to measure the result.

What decides whether examples in the prompt or fine-tuning is cheaper?

The number of requests. Examples in the prompt cost tokens on every request, while fine-tuning costs a one-off amount and then shorter prompts. The break-even is where the one-off cost has been repaid, and the forecast of requests is an assumption to state.

Model answer

The question: “Our assistant gives wrong answers. Should we fine-tune a model, build retrieval, or improve the prompt?” This is the question a lead engineer asks before approving a plan and an interviewer for a role building with models asks to hear a defensible choice. A strong answer hits five beats, in this order:

1. Diagnose — sort each failure into a context problem (the model lacked a fact) or a behaviour problem (it had the facts and acted wrongly).
2. Prompt first — start with instructions, examples and an evaluation set, because it is the cheapest change and defines what right means.
3. Match the lever — retrieval or a tool for missing, changing or private facts; examples in the prompt, then fine-tuning, for behaviour that needs many examples.
4. Cost — name what each change costs to keep working: tokens on every request, a pipeline to keep current, or a dataset and a retraining run.
5. Evidence — state what measurement would change the choice, and measure each change on the same evaluation set.

“I wouldn’t pick a technique before I knew what kind of failure this is. I’d sort the failures into two groups. If the model lacked a fact, because it was missing, out of date or private, that is a context problem. If it had the facts and the format, tone or reasoning was wrong, that is a behaviour problem. I’d start with the prompt, because it is the cheapest change and because it forces me to define a right answer and build a small evaluation set. Then I’d match the lever. For facts that live in documents I’d use retrieval, which also gives me a chunk to cite and turns a document change into a re-ingest. For facts in a live system, like an order status, I’d use a tool. For behaviour, I’d try instructions and examples first, and consider fine-tuning only if many examples are needed and the prompt can’t hold it. I’d put numbers on the cost of each: tokens on every request for examples, a pipeline to keep current for retrieval, and a dataset, a hold-out set and retraining for fine-tuning, and I’d work out how many requests it takes for a one-off cost to pay for itself. The levers can be combined, so I’d decide per failure, measure on the same evaluation set, and say what result would make me change my mind.”

Missing a beat is a failure state. Without diagnose the answer picks a technique before it knows the problem. Without prompt first it skips the cheapest change and the evaluation set. Without match the lever it treats the three approaches as interchangeable. Without cost the choice cannot be defended in a design review. Without evidence it is a preference.

Why this matters

• Design reviews ask for it. A proposal to fine-tune, or to build retrieval, is judged by whether the failure it addresses is the kind that approach fixes, and by who will maintain it.
• Money follows the choice. Tokens on every request, a one-off training cost and a pipeline to run are different cost shapes, and which is cheapest depends on a forecast of requests that has to be stated.
• Wrong levers waste a quarter. Training a model on facts that change, or building retrieval for a tone problem, delivers work that is finished and does not fix the ticket.
• Talking to customers. When a customer asks why the assistant “still says the old rule” after a retrain, an engineer who can explain that the rule is inside the model and a document edit does not reach it gives a better answer than one who proposes another training run.

After this chapter

The project for this chapter is Choose the Lever and Price the Change (project.md). You will sort ten failing requests into the lever that fits each with a printed rule, work out from stated token counts and prices how many requests it takes for a one-off fine-tuning cost to be repaid, and finish by writing decisions.tsv: each ticket with its lever.

Building AI Agents opens with the tickets your file marks tool: requests whose answer lives in a system that changes from request to request. It shows how a model asks for such a fact, and how to limit what it is allowed to ask for.

Hands-on Project — Choose the Lever and Price the Change

Objective

Sort ten failing requests into the lever that matches each one, count how many requests it takes for a one-off fine-tuning cost to be repaid by shorter prompts, and write a decisions file, so that a choice between prompting, retrieval, a tool and fine-tuning rests on a stated rule and stated numbers.

What this project needs: Python 3 and its standard library only. No task needs an account with a model provider, an API key or any payment. The tickets and the prices are printed below, and every check is an exact comparison, so every result can be checked exactly. The prices are invented teaching figures, not the price of any provider’s model. All three tasks are required.

Your evidence file. Keep one plain-text file, chapter7-evidence.txt. Each task tells you which lines to add. Every line has the form key: value, one per line, with the key in lower case exactly as printed. Write what your code printed, not what you expected it to print. Where a value is a list, write its items comma-separated with no spaces, in the order given.

The data and the rules for all tasks

Copy this into your Python file. Each ticket describes one failing request as (ticket id, what happened, needs_fact, fact_in_prompt, fact_source, instruction_given, behaviour_wrong, examples_needed):

• needs_fact — the answer depends on a fact the model could not be expected to know on its own;
• fact_in_prompt — the correct, current fact was in the prompt when the model answered;
• fact_source — where that fact lives: "documents" (text that changes rarely) or "live" (a system whose answer changes from request to request), or "none" when no fact is needed;
• instruction_given — the prompt already told the model what to do about the failing behaviour;
• behaviour_wrong — the facts were fine, but the format, tone or reasoning of the reply was wrong or inconsistent;
• examples_needed — how many different worked examples it would take to show the model the wanted behaviour (0 when none are needed).

TICKETS = [
    ("k1", "The assistant quotes a 14-day return rule that was replaced by a 7-day rule.", True, False, "documents", True, False, 0),
    ("k2", "A customer asks for the current status of order 4412 and the assistant guesses.", True, False, "live", True, False, 0),
    ("k3", "Seven of twenty replies break the required JSON layout, though the layout is in the prompt.", False, True, "none", True, True, 3),
    ("k4", "Replies miss the house tone; it takes about forty varied examples to show what good looks like.", False, True, "none", True, True, 40),
    ("k5", "The assistant says to ask IT about the VPN; the page about the VPN exists and was not retrieved.", True, False, "documents", True, False, 0),
    ("k6", "Replies run to long paragraphs; the prompt says nothing about length.", False, True, "none", False, True, 0),
    ("k7", "A customer asks how many units are in stock now and the assistant guesses.", True, False, "live", True, False, 0),
    ("k8", "A twelve-step checklist is followed inconsistently; about fifteen varied examples would show it.", False, True, "none", True, True, 15),
    ("k9", "The assistant does not know what an internal product codename means.", True, False, "documents", True, False, 0),
    ("k10", "The required sign-off line is dropped on some replies; two examples would show it.", False, True, "none", True, True, 2),
]

CAUSES = ["add-document", "re-ingest", "re-chunk", "raise-k", "prefer-newest", "none"]
CONTEXT_LEVERS = {"add-document", "re-ingest", "re-chunk", "raise-k", "prefer-newest"}

The lever rule. For a ticket, take the first of these that applies:

1. needs_fact is true and fact_in_prompt is false: the lever is tool when fact_source is "live", otherwise retrieval.
2. instruction_given is false: the lever is prompt.
3. behaviour_wrong is true: the lever is prompt-examples when examples_needed is 5 or fewer, otherwise fine-tune.
4. Otherwise the lever is none.

Task 1 — Choose the lever for each ticket

Write choose_lever(ticket), which returns one of tool, retrieval, prompt, prompt-examples, fine-tune or none, following the lever rule. Print the lever for all ten tickets, and the number of tickets for each lever.

Then, for the six repairs in CAUSES (the repairs your project in Build and Evaluate RAG Applications mapped each ticket to), print how many are in CONTEXT_LEVERS: those are repairs to what the model is given, not to the model.

Stop there: ten tickets, one rule. You do not need a model or a command-line interface.

Add to your evidence file:

t1_counts: <the ticket count for each lever, in the form retrieval=N,tool=N,prompt=N,prompt-examples=N,fine-tune=N>
t1_k1: <lever for k1>
t1_k6: <lever for k6>
t1_k8: <lever for k8>
t1_causes_context: <how many of the six repairs in CAUSES are in CONTEXT_LEVERS>

Output: one Python file with a lever function, and five evidence lines.

Task 2 — Price the change: examples in the prompt, or fine-tuning

This task stands on its own: it needs only the figures below.

A team has a behaviour problem that two different changes would fix. The figures are invented.

• Every request carries a base prompt of 400 tokens.
• Option A, examples in the prompt: add 1,500 tokens of examples to every request. Input tokens cost 2 cost units per 1,000 tokens.
• Option C, fine-tuning: a one-off training cost of 6,000 cost units. After it, the examples are no longer in the prompt, so each request carries only the 400-token base prompt, but input tokens on the fine-tuned model cost 3 cost units per 1,000 tokens.

Compute, as exact arithmetic (multiply everything by 1,000 first if you want to avoid decimals):

1. The cost of one request under A and under C.
2. The total cost of 5,000 requests under A and under C. For C, include the one-off training cost.
3. The break-even: the smallest whole number of requests N for which the total cost under C is less than or equal to the total cost under A.
4. Which option has the lower total at 1,000 requests: print a or c.

Add to your evidence file:

t2_per_request_a: <cost units per request under A, one decimal place>
t2_per_request_c: <cost units per request under C, one decimal place>
t2_total_a_5000: <total cost under A at 5,000 requests, whole number>
t2_total_c_5000: <total cost under C at 5,000 requests, whole number>
t2_breakeven: <the break-even request count, whole number>
t2_cheaper_at_1000: <a or c>

Before you submit, write one sentence for yourself on what this arithmetic leaves out about the two options; it is not scored.

Output: the arithmetic, and six evidence lines.

Task 3 — Write the decisions file

This task stands on its own: it needs the data and the lever rule above.

Write decisions.tsv: one line per ticket, in order, with two fields separated by a single tab character: the ticket id and the lever from the lever rule.

Add to your evidence file:

t3_rows: <number of lines in decisions.tsv>
t3_tool_ids: <the ids of the tickets whose lever is tool, in order>

Output: a decisions.tsv of ten lines, and two evidence lines.

What carries forward

Keep decisions.tsv and chapter7-evidence.txt. The next course, Building AI Agents, opens with the tickets your file marks tool: requests whose answer lives in a system that changes from request to request, which neither a document nor a training run can hold, and which has to be looked up live at request time: by your application before the call, or by the model calling a tool.