Chapter 9 — Decision Records and Defending a Design

Course: Taking AI to Production

Prerequisites: You can read and run a short Python script. From Chapter 8 you need three ideas, each restated in the notes before it is used: a trade-off is what a choice gives up for what it gains, the nearest cheaper option is the costliest option that still costs less than the chosen one, and a cap is a hard limit on a total across a design. Architecture decision record, status, superseded, confidence level and reversal trigger are defined in the notes before they are used.

What the reader can do after this chapter:

• Check a draft decision record against a template and list each problem it has.
• Write a record for each decision in a design from its trade-off line, with a rejected option and a trigger.
• Supersede an accepted record when a cap changes, leaving the old record in the log with its status changed.

Core question this chapter answers: How do you record a design decision so that someone who was not there can follow it and later change it, and how do you defend it against a reviewer who would have chosen differently?

Connections:

• Builds on: Chapter 8’s trade-off file. The project opens with the lines of your tradeoffs.tsv, one challengeable decision each.
• Leads into: nothing further in this course. The project ends by naming the files that make up your portfolio.

Your answers are scored

This chapter is assessed. The quiz is marked against a key held separately from your materials. Questions 1–8 are multiple choice and are marked by comparing your letter with the key. Questions 9–10 ask for a short answer in an exact format, and each one is marked by a rule printed with the question.

• Nothing is judged by reading it for style. An answer either matches the rule or it does not.
• All three project tasks are scored from an evidence file of key: value lines and two tab-separated files. Every accepted value follows from what the task prints.

Taking AI to Production — Chapter 9: Decision Records and Defending a Design

Before you start

Nothing to install for the reading. For the project you need Python 3 and nothing beyond its standard library. You do not need an account with any cloud or model provider or an API key: the design, the trade-off lines and the draft records the project uses are printed in the project file.

Keep one question in mind while you read: if I left the team tomorrow, could a stranger tell why each decision was made, and what would have to change for it to be made differently?

Summary

A design is only as durable as the reasons behind it, and reasons that live in someone’s memory are lost when that person moves on. An architecture decision record (ADR) captures one significant decision with its context, the options considered, the outcome with its trade-offs and the status of the decision. Once a record is accepted it is not edited. When circumstances change, a new record supersedes the old one and the two are linked, so the history of the thinking stays readable. A challenge from someone who would have chosen differently is answered from the record: the requirement the decision served, the option the challenger prefers and what it would save, what it would give up or break, how sure you are, and what new fact would change the decision.

Key concepts

Term	Definition
Architecture decision record (ADR)	A short document describing one significant architectural decision: its context, the decision and its consequences, usually with the options considered and a status.
Decision log	The collection of a project’s ADRs, read as the history of how the system came to have its shape.
Context	The facts and requirements that made the decision necessary, stated without arguing for the outcome.
Options considered	The alternatives that were weighed, including the ones that were ruled out.
Decision outcome	The option chosen and the main trade-offs it carries.
Consequences	What follows from the decision, including the costs and the limits it brings.
Status	Where the record stands. The AWS guide names Proposed, Accepted, Rejected and Superseded, and this chapter uses those four.
Superseded	A record whose decision has been replaced by a later record. It stays in the log and points to its replacement.
Confidence level	A statement of how sure the author is of the decision, so that a low-confidence decision is reconsidered sooner when new information arrives.
Trade-off, nearest cheaper option	From Chapter 8: a trade-off is what a choice gives up for what it gains, and the nearest cheaper option is the costliest option that still costs less than the chosen one.
Cap	From Chapter 8: a hard limit on a total across the design, such as cost per 1,000 requests or seconds added to a reply.
Challenge	A reviewer’s objection that proposes a different decision, because of a different priority, a cheaper option or a new fact.
Reversal trigger	A stated fact whose change would lead you to reopen the decision, such as a cap being raised. The term is this chapter’s own and not a field in the cited guides.

Explanations

The failure, before the explanation

A team chose a separate store for each customer’s documents, at a higher cost, because a customer contract required it. Nothing was written down except a line in a planning meeting. A year later a new engineer joins and sees a bill larger than the shared alternative would cost. She proposes the cheaper design, and it looks reasonable on a diagram. The team spends a long meeting rebuilding the reasons from memory, and the contract clause is half-remembered and attributed to the wrong customer. In a second case a team lead edits the reasoning in the design document after the fact to fit what was built, and now nobody can tell what was believed when the decision was made.

Neither team made a wrong decision. They made a decision with no durable reason attached, and a later reader could do nothing but guess or re-argue it.

What a decision record holds

Two examples first. A ship’s log records the position, the weather and the reason for a change of course, so that an inquiry can see what the captain knew at the time. A court judgment sets out the facts, the arguments considered, the ruling and its consequences, so that a later case can rely on it. A record of a design decision does the same job for a design.

The sources agree on a small core. The AWS guide says each ADR describes the architectural decision, its context and its consequences, and that at a minimum each should define the context, the decision itself and the consequences. The Azure guide lists consistent elements: a problem statement with context, options considered, the decision outcome including important trade-offs, and status. It adds that the record should capture the confidence level, because “sometimes an architecturally significant decision is made with relatively low confidence” and documenting that “could prove useful for future reconsideration decisions”. One early write-up of the format gives five parts: title, context, decision, status and consequences, and it describes the context as “value-neutral. It is simply describing facts”, with the whole document “one or two pages long” (Nygard, Documenting Architecture Decisions). The sources differ in detail, and none is the only correct template, so this chapter uses the common core plus the options and the confidence.

An example in the shape of Chapter 8’s trade-off lines. For a library’s lending assistant, a line reading X1 B 5 - wait says: the nearest cheaper option for decision X1 saves 5, loses no requirement, and breaks the wait cap. As a record it reads: context, one branch’s staff must not read another branch’s documents and the total wait is capped at 6 seconds. Options, a shared index with a filter and a separate index for each branch. Outcome, the separate index, because the filter option adds seconds and breaks the cap. Consequence, higher cost. Confidence, high while the cap stands. Status, accepted.

• Applied use. Use one template for every record, and fill all of it: title, status, context, options, decision with trade-offs, consequences and confidence.
• Signal. “Why was this decided?”, “who agreed to this?” and “what else did you consider?” are questions a record answers. A design document that shows only the final diagram does not.
• Practical consequence. The record focuses on the reason and not on how the team built it, and the AWS guide says that makes it easier for others to adopt the decision and helps prevent architects who were not involved from overruling it. A record without justification loses value over time, because readers cannot judge whether the decision still applies.
• Versus the distractor. The tempting answer is a long design guide. The Azure guide says to avoid making records into design guides and to link supplementary material while keeping the decision clear on its own.

What deserves a record

A map-maker does not mark every footpath. A newspaper does not print every staff memo. The Azure guide gives the filter: include “choices that affect the system’s structure, key quality attributes, or are difficult to reverse”. The same guide notes that the record documents key decisions including alternatives that were ruled out. In a design like Chapter 8’s, the choice of an isolation, identity or approval option passes this filter, because each carries a requirement and can move a cap. The choice of a variable name does not.

• Applied use. For each decision in a design, ask whether it changes structure, a quality attribute such as cost or waiting time, or whether it would be hard to undo. Write a record when the answer is yes.
• Signal. A stem that mentions “hard to reverse”, “affects cost or latency” or “a reviewer will ask why” is describing a decision worth recording.
• Practical consequence. Too few records leave decisions to be rediscovered. Too many bury the important ones. The Azure guide says to keep records pithy, assertive, on-topic and factual.
• Versus the distractor. The tempting answer is to record everything, or to record nothing because the code shows what was done. Code shows what, and the record shows why.

Status, immutability and superseding

A bank corrects a mistaken entry by adding a reversing entry, not by erasing the old line, so the books show both. A statute is amended by a new act that refers to the old one, and the old text remains readable. ADRs work the same way. In the AWS guide, once the team accepts an ADR it becomes immutable, and if new insights require a different decision the team proposes a new ADR. When the team accepts the new one, it supersedes the previous one, and the owner changes the old record’s state to Superseded. The Azure guide gives the same rule: the log is an append-only record, so “don’t go back and edit accepted records”, write a new record that supersedes the original and link the two.

The AWS guide also describes the states before acceptance. An owner provides the ADR as Proposed, and the team reviews it. If the review finds actions, the ADR stays Proposed. The team may reject it, and then the owner records a reason for the rejection to prevent future discussions on the same topic. If the team approves, the owner adds a timestamp, a version and a list of stakeholders, and sets the state to Accepted. Nygard’s early format adds “deprecated” as a status, which shows that sources name different sets. The project uses the AWS four.

A small case. A decision log has four records, and one accepted record says to use a shared index with a filter. A new fact arrives: the contract now requires separate storage. The team writes record five, which chooses separate storage and names record three as superseded. The log now holds five records, one of them marked Superseded. It does not hold four edited ones.

• Applied use. Leave the decision and reasoning of an accepted record as written. Add a new record, link it, and change only the status and successor of the old one.
• Signal. “The decision changed”, “we now know” and “the cap was raised” in a stem point toward a new record. “Fix a typo in the context” is the kind of change a note in the record can handle, and a change of decision is not.
• Practical consequence. The log shows when and why the direction shifted. A reader sees both what was believed and what replaced it, and the superseded record explains why the old design looks as it does.
• Versus the distractor. The tempting answer is to edit the old record so that it matches the current design. That leaves a log that shows only the current state and hides the reasoning.

Defend the design under challenge

Two examples. A researcher answering a reviewer’s comment states the claim, the evidence and what would count against it, and concedes what is right. A lawyer answering an objection cites the facts and the rule, and does not promise a different ruling because the objector is senior. A design review works the same way.

Challenges come in three kinds, and they need different answers. A different priority: the challenger weighs cost above waiting time. A cheaper option: the challenger proposes the nearest cheaper option. A new fact: something has changed since the decision. The first two are preferences, and the record already answers them, so the answer is to restate. The third is information, and it can reopen the decision.

The defence follows the record, in five beats that this chapter’s model answer names. State the requirement the decision serves. Name the alternative the challenger prefers and concede what it saves. State the trade-off: which requirement it would stop meeting or which cap it would break, with the numbers. State your confidence and what you do not know. Give the trigger: the fact that would change your mind, and say that a new record would supersede this one. The beats are the notes’ own pattern, drawn from the record fields and not quoted from a source.

A worked example, with invented numbers for the library assistant. The challenger says: “I would use the shared index with a filter. It costs 5 less.” Requirement: one branch’s staff must not read another branch’s documents, and the whole path may add at most 6 seconds. Alternative: the filter option does cost 5 less, and meets the isolation requirement. Trade-off: it adds 2 seconds and the design is already at 6, so the total would be 8, over the cap. Confidence: high while the cap is 6, and I have not measured the filter’s real delay. Trigger: if the cap is raised to 8, or if measurement shows the filter adds nothing, I would write a new record that supersedes this one.

• Applied use. Answer from the record. Concede what the challenger is right about, give the numbers for the rest, and name the fact that would change the decision.
• Signal. “Why not X?”, “I would have done Y” and “can you defend this?” ask for a defence. Stating a trigger shows the decision is not a matter of pride.
• Practical consequence. A defence that separates preference from new fact keeps the review moving. A new fact leads to a superseding record, and a preference leaves the record standing.
• Versus the distractor. Two wrong answers pull opposite ways: insisting the decision is closed because it was accepted, and changing it at once because the challenger is senior. The first ignores new facts and the second ignores the record.

Three situations you will meet, and what they mean for your design

1. A new engineer proposes the cheaper option for a decision, and nobody remembers why it was not taken.
2. A cap changes after a decision was accepted, and the cheaper option now fits.
3. A reviewer says “I would have chosen differently” and gives no new information.

What you see	What to do	Why
A cheaper option proposed, reasons forgotten (situation 1)	Read the record’s options and trade-offs, and answer from them	A record keeps the reasons that memory loses
A cap changed after acceptance (situation 2)	Write a new record that supersedes the old one, and set the old status to Superseded	An accepted record is not edited, and the log keeps the history
A preference with no new fact (situation 3)	Restate requirement, alternative, trade-off, confidence and trigger, and leave the record standing	A preference is answered by the record, and a new fact is what reopens it

Flashcards

Say the answer aloud before revealing it. Speaking it is what exposes the gaps that reading past a written answer hides.

What does an ADR contain at a minimum, according to the AWS guide?

The context of the decision, the decision itself and the consequences of the decision. This chapter adds the options considered, the trade-offs, the confidence level and the status, which the Azure guide lists.

Which decisions deserve a record?

Choices that affect the system’s structure or its key quality attributes, or that are difficult to reverse. In the design project that means the choices that carry a requirement or a cap, and not trivia such as naming.

Why should the context be value-neutral?

One early write-up says the context should simply describe facts. A context that argues for the outcome hides the facts a later reader needs to judge whether the decision still holds.

What happens to an accepted record when the decision changes?

It is not edited. A new record is proposed and accepted, it supersedes the old one, and the old record’s status becomes Superseded with a link to its replacement.

Name the four statuses the project uses, and what Proposed means.

Proposed, Accepted, Rejected and Superseded. Proposed means the owner has submitted the record and it is ready for review, and it stays Proposed while review actions are open.

Why record a confidence level?

Some significant decisions are made with relatively low confidence. Writing that down helps a later reader decide when to reconsider, because new information bears harder on an uncertain decision.

Why must a rejected record carry a reason?

The AWS guide says the owner adds the reason to prevent future discussions on the same topic. Without it the rejected idea is proposed again and argued from the start.

What are the three kinds of challenge, and which can reopen a decision?

A different priority, a cheaper option and a new fact. The first two are preferences, which the record already answers. A new fact is information and can lead to a superseding record.

Name the five beats of a defence.

Requirement, alternative, trade-off, confidence and trigger. State what the decision serves, concede what the alternative saves, say what it would lose or break with numbers, say how sure you are, and say what fact would change the decision.

What is a reversal trigger, and where does the term come from?

A stated fact whose change would lead you to reopen the decision, such as a cap being raised. It is this chapter’s own term, drawn from the record’s status and confidence fields, and it is not a field in the cited guides.

Model answer

The question: “A senior reviewer says: I would have chosen the cheaper option for that decision. Defend yours.” An architect meets it in a design review or a hiring loop. A strong answer hits five beats, in this order:

1. Requirement — what the decision serves, stated from the record’s context.
2. Alternative — the cheaper option the reviewer prefers, and what it saves.
3. Trade-off — which requirement it would stop meeting or which cap it would break, with numbers.
4. Confidence — how sure you are and what is still unknown.
5. Trigger — the fact that would change the decision, and that a new record would supersede this one.

“The decision serves two things: one branch’s staff must not read another branch’s documents, and the whole path may add at most 6 seconds. I agree the cheaper option saves 5 and meets the isolation requirement. Where it differs is time: it adds 2 seconds, and the design already uses all 6, so it would take the total to 8 and break the cap. I am confident while the cap is 6, though I have not measured the filter’s actual delay and that is the thing I do not know. If the cap were raised to 8, or measurement showed the filter adds nothing, I would write a new record that supersedes this one, and the old one would stay in the log marked Superseded.”

Missing a beat is a failure state. Without requirement the defence is a matter of taste. Without alternative the challenger has not been heard. Without trade-off the claim that the choice is better cannot be checked. Without confidence the answer claims certainty it does not have. Without trigger the decision looks like a matter of pride and there is no route to change it.

Why this matters

• Handovers. A new owner who finds the reasons in a record can keep or change the decision on evidence, and does not have to rebuild the reasons from memory.
• Design reviews. A reviewer may have chosen differently, and a defence that concedes what is true and names a trigger keeps the review on facts.
• Audits and customer questions. A customer or auditor who asks why a control exists can be shown the record, with its context and its date.
• Hiring loops. A portfolio piece that shows a design and its records, and not just a diagram, shows how you reason under challenge.

After this chapter

The project for this chapter is Decisions on the Record (project.md). You will check five draft records against a template, write a record for each decision from your Chapter 8 trade-off lines, and supersede one record when a cap changes.

This is the last chapter. The project ends by naming the files from the course that make up your portfolio, and what a reviewer can see in each.

Hands-on Project — Decisions on the Record

Objective

Check draft decision records against a template, write a record for each decision in a design from its trade-off line, and supersede one record when a cap changes, so that every choice in your capstone design carries its reason and can be defended and changed on evidence.

What this project needs: Python 3 and its standard library only. No task needs a cloud account, a model, an API key or an internet connection. Everything is printed below and every check is an exact comparison. The three tasks can be done in any order.

Your evidence file. Keep one plain-text file, chapter9-evidence.txt, of key: value lines with the key in lower case exactly as printed. Write what your code printed, not what you expected. Write a list comma-separated with no spaces, in the order given, and an empty list as none.

The data and the rules for all tasks

The case continues from Chapter 8: an insurer’s claims assistant. Copy this into your Python file.

# Chapter 8's tradeoffs.tsv as (decision, nearest cheaper option, saving, requirements lost, caps broken):
TRADEOFFS = [
    ("D1", "A", 7, "R1", "-"), ("D2", "A", 6, "R2", "-"), ("D3", "B", 2, "R4", "-"),
    ("D4", "B", 6, "-", "wait"), ("D5", "B", 4, "R6", "-"), ("D6", "C", 4, "R8", "-"),
]
DESIGN = {"D1": "B", "D2": "B", "D3": "C", "D4": "C", "D5": "C", "D6": "B"}   # Chapter 8's design.tsv

ALLOWED_STATUS = ["Proposed", "Accepted", "Rejected", "Superseded"]

DRAFTS = [
    {"id": "ADR-A", "title": "Use single-pass retrieval", "status": "Accepted",
     "context": "Answers must cite a document version.", "options": ["no retrieval", "single pass", "rerank"],
     "decision": "single pass", "consequences": "Adds 2 seconds and 12 cost units.", "superseded_by": ""},
    {"id": "ADR-B", "title": "Approve payments above a threshold", "status": "Accepted",
     "context": "Payments over 200 need a person.", "options": ["no gate", "gate above threshold"],
     "decision": "gate above threshold", "consequences": "", "superseded_by": ""},
    {"id": "ADR-C", "title": "Delegate identity to the user", "status": "Proposed",
     "context": "", "options": ["shared credential", "user delegation"],
     "decision": "user delegation", "consequences": "Adds 1 second.", "superseded_by": ""},
    {"id": "ADR-D", "title": "One index per tenant", "status": "Approved",
     "context": "Tenants must stay apart.", "options": ["shared index", "index per tenant"],
     "decision": "index per tenant", "consequences": "Costs 11 cost units.", "superseded_by": ""},
    {"id": "ADR-E", "title": "Route easy questions to a small model", "status": "Superseded",
     "context": "Cost is capped.", "options": ["large model only"],
     "decision": "small model", "consequences": "Saves 6 cost units.", "superseded_by": ""},
]

# decision -> option -> (cost per 1,000 requests, seconds added, requirements it meets)
OPTIONS = {
    "D1": {"A": (5, 1, set()),   "B": (12, 2, {"R1"}),  "C": (20, 4, {"R1"})},
    "D2": {"A": (0, 0, set()),   "B": (6, 2, {"R2"}),   "C": (9, 3, {"R2"})},
    "D3": {"A": (1, 0, set()),   "B": (2, 0, {"R7"}),   "C": (4, 1, {"R4", "R7"})},
    "D4": {"A": (2, 0, set()),   "B": (5, 2, {"R3"}),   "C": (11, 0, {"R3"})},
    "D5": {"A": (1, 0, set()),   "B": (3, 0, {"R5"}),   "C": (7, 1, {"R5", "R6"})},
    "D6": {"A": (14, 3, {"R8"}), "B": (8, 2, {"R8"}),   "C": (4, 1, set())},
}
REQUIREMENT_IDS = {"R1", "R2", "R3", "R4", "R5", "R6", "R7", "R8"}
CAP_COST = 50

A design picks one option for each of the six decisions. Its cost and its wait are the sums of the chosen options’ costs and seconds, and a requirement is covered when at least one chosen option lists it in its meets set. Cost is per 1,000 requests, and the rule that costs and seconds add is this case’s simplification.

A record is valid when it has none of the following problems, which are checked in this order, and a record lists each problem it has once:

1. missing_title, missing_context, missing_decision, missing_consequences: that field is absent or the empty text (checked in the order title, context, decision, consequences).
2. few_options: the options list has fewer than two entries.
3. decision_not_in_options: decision is not empty and is not one of the options entries.
4. bad_status: status is not in ALLOWED_STATUS.
5. no_successor: status is Superseded and superseded_by is empty.

A record line has six tab-separated fields. For the six decisions in TRADEOFFS order, the lines are built like this.

• The record id: ADR-1 for the first decision, ADR-2 for the second, and so on.
• The decision, for example D1.
• The chosen option, from DESIGN.
• The rejected option: the nearest cheaper option from TRADEOFFS.
• The trigger: if the caps-broken field is not -, the text raise: followed by the first cap named in it. Otherwise the text drop: followed by the first requirement named in the requirements-lost field.
• The status, which is Accepted for all six.

Task 1 — Check the draft records, and carry the trade-offs in

Carry-in: from TRADEOFFS, add up the savings, and list the decisions whose caps-broken field is not -. Then write problems(record) to return the list of problems of a record, and run it over DRAFTS.

t1_carried_saving: <sum of the six savings>
t1_carried_cap_blocked: <decisions whose caps-broken field is not "-", in order, comma-separated, or none>
t1_bad_ids: <ids of records with at least one problem, in order, comma-separated, or none>
t1_problem_total: <total number of problems across the five records>
t1_problems_adr_e: <the problems of ADR-E, in the order checked, comma-separated>

Output: one Python file with problems, and five evidence lines.

Task 2 — Write a record for each decision

Write adr-records.tsv: one line per decision in TRADEOFFS order, built by the record-line rules above.

t2_raise_triggers: <number of lines whose trigger starts with raise:>
t2_drop_requirements: <the requirement ids after drop:, in order, comma-separated, or none>
t2_rejected: <the six rejected options in order, comma-separated>

Output: adr-records.tsv of six lines, and three evidence lines.

Task 3 — Supersede a record when a cap changes

A new fact arrives: the wait cap is raised from 8 to 10 seconds. CAP_COST stays at 50. A design is valid when it covers all eight requirement ids, the cost is at most CAP_COST and the wait is at most 10. Try every combination of one option per decision and find the cheapest valid design. Compare it with DESIGN. For each decision whose option changed, in decision order, the log gets a new record: the next id after ADR-6, the decision, the new option as chosen, the old option as rejected, the trigger supersedes: followed by the id of the old record for that decision, and the status Accepted. The old record keeps every field except its status, which becomes Superseded.

Build the six record lines as the rules above describe, so this task does not need your Task 2 file. Write adr-records-final.tsv: those six lines with the changes applied, followed by any new lines.

t3_valid: <number of valid combinations under the new cap>
t3_new_design: <the options of the cheapest valid design in order D1 to D6, comma-separated>
t3_new_cost: <its cost>
t3_new_wait: <its wait>
t3_changed: <decisions whose option changed, in order, comma-separated, or none>
t3_statuses: <the status of each line of adr-records-final.tsv, in order, comma-separated>

Output: adr-records-final.tsv, and six evidence lines.

What carries forward

This is the last chapter, so what carries forward is your portfolio. Keep adr-records.tsv, adr-records-final.tsv and chapter9-evidence.txt with the files each earlier project told you to keep, (from this course and from the related courses Build and Evaluate RAG Applications and Building AI Agents), including placement.tsv, architecture.tsv, degradation.tsv, ingest-plan.tsv, budget-rules.tsv, agent-scopes.tsv, executed.tsv, oversight-policy.tsv, denied.tsv, audit-gaps.tsv, design.tsv and tradeoffs.tsv. Together they show a reviewer a design you built, the checks you ran on it, and the reasons you can defend.