Chapter 7 — Auditability

Course: Taking AI to Production

Prerequisites: You can read and run a short Python script. From earlier material you need three ideas, each restated in a sentence in the notes before it is used: an audit record can name both the user and the agent that acted for the user, an approved action carries an approval identifier, and a boundary can refuse a request. Trace, sequence number, hash chain, anchor, reference and gap are defined in the notes before they are used.

What the reader can do after this chapter:

• Check a set of audit records against a stated list of required fields and say which of them could not be used to reconstruct what happened.
• Verify a hash-chained log, say where an altered record or a deleted record shows up, and say what a rewritten chain needs to be compared against.
• Find the gaps in a numbered sequence of records and say what a sequence number can and cannot reveal.

Core question this chapter answers: After the fact, how can you prove what an AI system was asked, what it retrieved and what it did, and how do you know the record itself can be trusted?

Connections:

• Builds on: the refused requests from the tenant isolation chapter in Building AI Agents. It opens with the requests in denied.tsv and asks what would have to be written down to prove, afterwards, that they were refused and why.
• Leads into: Chapter 8, the capstone, which asks for a complete design. It opens with the gaps your project finds in the audit records, in audit-gaps.tsv, and asks which control in the design would prevent each kind.

Your answers are scored

This chapter is assessed. The quiz is marked against a key held separately from your materials. Questions 1–8 are multiple choice and are marked by comparing your letter with the key. Questions 9–10 ask for a short answer in an exact format, and each one is marked by a rule printed with the question.

• Nothing is judged by reading it for style. An answer either matches the rule or it does not.
• All three project tasks are scored from an evidence file of key: value lines and one tab-separated file. Every accepted value follows from what the task prints.

Taking AI to Production — Chapter 7: Auditability

Before you start

Nothing to install for the reading. For the project you need Python 3 and nothing beyond its standard library. You do not need an account with any cloud or model provider or an API key: every record, hash and sequence the project uses is printed in the project file.

Keep one question in mind while you read: if someone asks in a review what happened on this request, what would I point at, and why would they believe it?

Summary

An audit trail answers what a system was asked, what it retrieved, what it decided and what it did, and it only helps if it can be read back as a story and trusted as evidence. Each step of a request needs a record that names who acted, what was asked and what came back, tied together by a shared trace identifier. Prompts and answers often contain personal or sensitive data, so a record can hold references to stored content rather than the content itself. A record is trusted when tampering shows up, which a hash chain, kept with a copy of its latest value somewhere the writer cannot change, makes possible. Gaps in a numbered sequence, and refusals as well as actions, belong in the design because the missing record is the one nobody can examine.

Key concepts

Term	Definition
Audit record	An entry written for one step of a request, such as a retrieval, a model call, a tool call or a refusal. It says who acted, what the step was and what came of it.
Trace	A shared identifier carried by every record of one request, so that the records can be put back together in order.
Sequence number	A number each record carries within its trace, increasing by one, so that a missing record leaves a visible hole.
Reference	A stored identifier, such as a document id with its version or a pointer to a stored prompt, that lets someone retrieve the content without the log holding it.
Reconstruction	Rebuilding, from the records alone, what a request was asked, what was retrieved and what was done.
Hash	A fixed-length value computed from some text. Changing the text changes the value in practice.
Hash chain	A log where each record’s stored hash is computed from the previous record’s hash and its own text, so changing or removing a record breaks the check at that record.
Anchor	A copy of the chain’s latest hash kept in a separate store that the writer of the log cannot change.
Gap	A sequence number that should have a record and does not.
Denial record	An audit record of a request that was refused, with the reason.

Explanations

The failure, before the explanation

A company’s refund agent answers a customer, “Your refund was reviewed against the policy updated recently and approved.” The customer disputes it, and a reviewer opens the logs. The application log shows that a request arrived and that a response of some length went out. It shows no document identifier, so nobody can say which version of the policy the answer used. The log line for the tool call has the order number and the word “success”, with no approval identifier and no user, so nobody can say who approved it or for whom the agent acted. The model’s name is missing, and the reviewer cannot tell whether the model changed after the incident.

Then a second problem appears: a log file for part of the evening is not there, and nothing says whether it was never written or was removed. Nothing in the model misbehaved. The logs were written for debugging, which asks “did the request run?”, and not for evidence, which asks “what exactly happened, and can I prove it?”.

What a record must hold: reconstruction

Two examples first. A flight data recorder keeps the readings that let investigators replay a flight in order, including what the crew was told and what they did. A hospital chart records, for each intervention, who did it, what was given and when, and it is useless if a line reads only “treated”. A second way to see it is a receipt: “paid” is not a receipt, while an itemised one with a date, a till number and a cashier is.

For an agent, reconstruction needs a record per step, and each record has a common part and a part specific to the kind of step. The common part is the trace identifier, the sequence number, the user and the agent that acted for the user (the audit record in the identity chapter in Building AI Agents named both). The specific part depends on the step. A retrieval records which documents were returned, with their versions. A model call records the model and version used and references to its input and output. A tool call records the action, the approval identifier from the approval chapter in Building AI Agents, and the result. A denial records the action and the reason. OpenTelemetry’s generative-AI conventions point the same way for model calls: they define gen_ai.request.model, conditionally required when available, for the model a request is made to, and gen_ai.response.id, recommended, for the unique identifier of the completion (OpenTelemetry, GenAI semantic conventions, which carry a Development status marker, so they may change).

Regulation names recording as a requirement for some systems. The EU AI Act’s record-keeping article says high-risk AI systems “shall technically allow for the automatic recording of events (logs) over the lifetime of the system” (Regulation (EU) 2024/1689, Article 12(1), as published at artificialintelligenceact.eu). That applies to a defined class of systems. This chapter does not claim it applies to yours; it shows that automatic recording is named as a requirement rather than left to the developer’s choice.

• Applied use. Write the list of required fields per kind of step, make the code that writes records refuse a record that lacks one, and run a sample request end to end to see whether it can be reconstructed from the records.
• Signal. Wording such as “which version of the document”, “who approved it”, “which model produced this” and “for whom was the agent acting” points toward fields a record needs. “We have the request and the response” is the weak design.
• Practical consequence. A missing field cannot be filled in later. A reviewer asking which model answered an earlier request gets the answer from the record or not at all. Writing records costs storage, and a required-fields check at write time turns a silent hole into an error.
• Versus the distractor. The tempting wrong answer is “turn on debug logging”. Debug logs record what the developer thought to print, and they do not carry the trace, the sequence, the approval or the user.

References, not content

A library catalogue gives two examples. The catalogue card holds the title, shelf mark and edition, so you can find the exact book, and it does not hold the book. A bank’s transaction log holds an account reference and an amount, and not a copy of the customer’s file. A second way to see it is a shipping label: it identifies the parcel and where it came from without opening it.

Prompts and answers often hold personal data. OpenTelemetry’s conventions treat the content attributes (gen_ai.input.messages, gen_ai.output.messages and gen_ai.system_instructions) as “Opt-In”, with the warning that they are likely to contain sensitive information including personal data, and they state that instrumentations may provide a way to filter or truncate them (OpenTelemetry, GenAI semantic conventions). That is a convention for one telemetry standard, and its guidance applies to what a generic trace collects. For an audit design the same tension appears: the record needs enough to prove what happened, and the broad log should not become a second copy of every customer’s text.

The pattern is to keep the content in a store with tighter access and a stated retention, and to write into the audit record a reference to it: a document identifier with a version, a pointer to the stored prompt, and, where it helps, a hash of the content so that someone who retrieves a copy can check it is the one the record meant. A hash proves a given copy matches. It does not recover the content, and it does not say the content was correct.

• Applied use. Decide for each field whether it is stored in the record or referenced. Put identifiers, versions, decisions and reasons in the record, and put prompts and answers in an access-controlled store.
• Signal. “Logs are readable by all engineers”, “customer text in the log” and “we must be able to show which document” together point toward references with a protected content store.
• Practical consequence. A reference only helps while the content it points to is kept. If the content store expires prompts before the audit trail is needed, the reference leads nowhere, so retention of the two is set together.
• Versus the distractor. Two tempting wrong answers pull opposite ways: log the full text of everything in the general log, or log only counts and timings. The first spreads sensitive data, and the second cannot reconstruct anything.

Trusting the record: tamper evidence and the anchor

A sealed ledger gives two examples. A paper ledger with numbered pages, bound so that a torn-out page shows, makes removal visible. A wax seal on each letter, stamped over the previous letter’s seal, shows if one was opened. A second way to see it is a chain of shipping containers, each sealed with a tag that records the tag of the one before it: replace a container in the middle and the tags on either side no longer match.

A hash chain does the same for records. Each record stores a hash computed from the previous record’s hash and its own text. To verify, recompute each record’s hash from the stored previous hash and its text, and compare it with what is stored. If a record’s text was altered, the check fails at that record. If a record was deleted, the check fails at the record after it, because that record’s stored hash was computed from the deleted one. This is detection, not prevention: nothing stops the change, and the chain shows it.

One provider’s log service describes a production form. Its log-file integrity validation “creates a hash for every log file that it delivers”, and also delivers a digest file that references the log files for the last period and holds a hash of each. Each digest file is signed with a private key, and digest files are put in a folder separate from the logs “to enforce granular security policies”. Hashing uses SHA-256 and signing uses SHA-256 with RSA, and the page says this makes it computationally infeasible to modify, delete or forge log files without detection. Enabling the feature delivers digest files but “does not validate the integrity of the files”, which is a separate step (AWS, Validating CloudTrail log file integrity). That is one provider’s design. The claim there is about that service’s files, and not about every logging system.

There is a limit the simple chain does not remove. Someone who can write the whole log can alter a record and recompute every later hash, and the chain then checks out internally. The answer is the anchor: a copy of the latest hash kept somewhere the writer cannot change, such as a separate store with different permissions. A rewritten chain ends in a different hash from the anchor, so comparing the two exposes it. The same reason is behind the provider’s separate folder for digest files.

• Applied use. Chain the records, store the latest hash in a separate system with separate permissions, and run verification on a schedule. Make the agent’s own identity unable to modify or delete the log store.
• Signal. “An administrator could have edited the log”, “prove the log was not changed” and “the logs for one evening are missing” point toward tamper evidence. “The log is in the same bucket the application writes to” is the weak design.
• Practical consequence. Verification has to be run to give any assurance, and someone has to own the anchor store and the schedule. A chain proves the records are unchanged since writing, and does not prove they were correct when written.
• Versus the distractor. The tempting wrong answer is to use a longer hash. A longer hash does not help against someone who recomputes the chain. Separation does.

Gaps and refusals: completeness

A numbered cheque book gives two examples. If cheques 41, 42 and 44 were cashed, number 43 is unaccounted for. A hotel’s key-card log that lists only successful entries cannot show the three failed attempts at 3 am. A second way to see it is a register at a fire drill: a name missing from the list is a question, while a name never written down is a question nobody knows to ask.

A sequence number per trace lets a missing record show up: if the records for one trace are numbered 1, 2, 4, then something numbered 3 should exist. The provider’s validation page makes a related point at the file level: the process lets you assert positively that no log files were delivered during a given period. A sequence number has a limit: it reveals a hole in the middle, and it cannot show that records after the last one are missing, because nothing says how far the trace should have gone. Pairing it with the chain, or with a count recorded separately, covers more.

The other half of completeness is refusals. Requests that a boundary refuses are the evidence that the boundary works, and the first place an attacker’s attempts show. The tenant isolation chapter in Building AI Agents produced a list of refused requests with reasons, and those need records of their own: the request, who made it, what it asked for and which rule refused it. A trail that holds only the actions that ran would show nothing about the attempts that failed.

A related point comes from the Model Context Protocol’s guidance on forwarding client tokens straight to downstream services. It says the downstream server’s logs “may show requests that appear to come from a different source with a different identity”, which makes incident investigation and auditing “more difficult” (Model Context Protocol, Security Best Practices). An audit trail built on the downstream logs alone would then name the wrong party.

• Applied use. Number the records within each trace, write a denial record for each refusal, and run a check that lists gaps in each trace.
• Signal. “How many attempts were refused?”, “show me everything that happened on this request” and “were any records lost?” point toward denial records and sequence numbers.
• Practical consequence. A gap check finds holes in the middle of a trace and not a trace that stops early. Denial records add volume, and that volume is the cost of being able to show that a rule was enforced.
• Versus the distractor. The tempting wrong answer is to log only successes, because failures are noise. The failures are what shows whether a control was tested and held.

Three situations you will meet, and what they mean for your design

1. A customer disputes an answer from a retrieval-based assistant, and the team must show which document version it used.
2. A security team asks whether anyone altered the audit log after a suspected incident.
3. A trace shows records numbered 1, 2 and 4 for a request that moved a payment.

What you see	What to do	Why
A dispute over which document version (situation 1)	Record document ids and versions in the retrieval record, with a reference to the stored prompt and answer	The record proves which version; the content store holds the text
A question whether the log was altered (situation 2)	Verify the chain and compare the latest hash with the anchor kept elsewhere	A rewritten chain agrees with itself and disagrees with the anchor
A hole at 3 in a payment trace (situation 3)	Treat it as a lost or removed record, investigate, and check whether the trace stops early	A sequence number reveals a middle gap and says nothing of the end

Flashcards

Say the answer aloud before revealing it. Speaking it is what exposes the gaps that reading past a written answer hides.

What does each audit record need in common, and what is specific to the step?

In common: the trace identifier, the sequence number, the user and the agent acting for the user. Specific to the step: for a retrieval the documents and versions, for a model call the model and references to input and output, for a tool call the action, approval and result, and for a denial the action and the reason.

Why is "we have the request and the response" a weak audit trail?

It cannot say which document version, which model, who approved or for whom the agent acted. Those facts cannot be added after the event, so a reviewer cannot reconstruct the request.

What does the EU AI Act's Article 12(1) say, and what does this chapter claim about it?

High-risk AI systems shall technically allow for the automatic recording of events over the lifetime of the system. The notes only record that this is named as a requirement for that class of systems, and does not say it applies to a given system.

How do OpenTelemetry's conventions treat prompt and answer content?

As Opt-In attributes with a warning that they are likely to contain sensitive information including personal data, with a note that instrumentations may filter or truncate them. The conventions carry a Development status marker.

What does a reference let you do that a log line of full text does not?

It identifies the exact content, such as a document and version or a stored prompt, and lets someone retrieve it from a protected store, without spreading the text through the general log.

How does a hash chain show an altered record, and how a deleted one?

Verification recomputes each hash from the stored previous hash and the record’s text. An altered record fails at its own position. A deleted record makes the record after it fail, because its stored hash was computed from the deleted record’s hash.

What can someone who writes the whole log do to a chain, and what defeats it?

Alter a record and recompute every later hash, so the chain checks out internally. An anchor, the latest hash kept in a separate store the writer cannot change, defeats it: the rewritten chain ends in a different value.

What does AWS's log-file integrity validation do, and what does enabling it not do?

It delivers hashes and signed digest files that let you detect modified or deleted log files, using SHA-256 and SHA-256 with RSA. Enabling it delivers digest files and does not itself validate the files, which is a separate step.

What can a sequence number reveal, and what can it not?

A missing number in the middle of a trace. It cannot show that records after the last one are missing, since nothing says how far the trace should have gone.

Why record refusals as well as actions?

A refusal is evidence that a control was tested and held, and it is where an attacker’s attempts show first. A trail holding only successes shows neither.

Model answer

The question: “Something went wrong with the agent earlier. How would you show what it was asked, what it retrieved and what it did, and why should the reviewer trust your evidence?” An architect is asked this in an incident review or an audit. A strong answer hits five beats, in this order:

1. Records — a record per step with a trace, a sequence number, the user and the agent, and step-specific fields.
2. References — identifiers, versions and pointers in the record, with content in a protected store.
3. Chain — hash-chained records, so an alteration or deletion shows when verified.
4. Anchor — the latest hash kept in a separate store the writer cannot change.
5. Completeness — sequence numbers to find gaps, and records for refusals as well as actions.

“I would write a record for every step: retrieval, model call, tool call and refusal. Each carries a trace identifier, a sequence number, the user and the agent acting for the user, and fields for the step, such as the document ids and versions, the model, the approval identifier and the reason for a refusal. Prompts and answers would sit in a store with tighter access, and the record would hold a reference to them, because the content can be sensitive. I would chain the records by hash so a changed or removed record shows up on verification, and keep the latest hash in a separate system the log writer cannot modify, since someone who can write the log could otherwise recompute the chain. Sequence numbers would show holes in the middle of a trace, and I would note that they cannot show a trace that stops early. I would record denials too, because they show the controls working.”

Missing a beat is a failure state. Without records the story cannot be rebuilt. Without references the log either lacks evidence or spreads sensitive text. Without chain a record can change unseen. Without anchor a rewritten chain looks valid. Without completeness the missing and refused events are invisible.

Why this matters

• Disputes and incidents. A customer’s complaint or a security incident is settled by what the records show, and a missing field is missing for good.
• Reviews and audits. A review asks what the system did and how you know, and an answer that names the records, their protection and their gaps is stronger than a statement that logging is on.
• Privacy. A trail that copies every prompt into a broad log creates a second place for personal data to leak, so what goes in the record and what is referenced is a design decision.
• Talking to customers. When a customer asks whether you can show what the agent did with their data, an architect who can describe records, protection and gaps gives a better answer than “we log everything”.

After this chapter

The project for this chapter is Records You Can Trust (project.md). You will check seven audit records for required fields, verify three hash-chained logs against a stored anchor, and find the gaps in three numbered traces, finishing by writing audit-gaps.tsv.

Chapter 8 is the capstone: a complete design for a realistic organisation. It opens with the gaps in your audit-gaps.tsv and asks which control in the design would prevent each kind of gap.

Hands-on Project — Records You Can Trust

Objective

Check audit records for the fields a reviewer would need, verify hash-chained logs against a stored anchor, and find the gaps in numbered traces, so that trust in an audit trail rests on checks you ran and not on the fact that logging is switched on.

What this project needs: Python 3 and its standard library only. No task needs a cloud account, a model, an API key or an internet connection. The records, hashes and sequences are printed below, and every check is an exact comparison. All three tasks are required, and they can be done in any order.

Your evidence file. Keep one plain-text file, chapter7-evidence.txt. Each task tells you which lines to add. Every line has the form key: value, one per line, with the key in lower case exactly as printed. Write what your code printed, not what you expected it to print. Where a value is a list, write its items comma-separated with no spaces, in the order given. Write an empty list as none.

The data and the rules for all tasks

Copy this into your Python file.

import hashlib

# The denied.tsv from the tenant isolation chapter, as (request, reason):
DENIED = [("t2", "other_tenant"), ("t4", "no_role"), ("t5", "not_authenticated"), ("t6", "other_tenant")]

COMMON = {"trace", "seq", "user", "agent"}
KIND_FIELDS = {
    "retrieval":  {"doc_ids"},
    "model_call": {"model", "input_ref", "output_ref"},
    "tool_call":  {"action", "approval", "result"},
    "denial":     {"action", "reason"},
}

RECORDS = [
    {"id": "r1", "kind": "retrieval",  "trace": "tr1", "seq": 1, "user": "ann", "agent": "refund-agent",
     "doc_ids": "doc-7@v3"},
    {"id": "r2", "kind": "model_call", "trace": "tr1", "seq": 2, "user": "ann", "agent": "refund-agent",
     "model": "m-1", "input_ref": "in-a", "output_ref": "out-a"},
    {"id": "r3", "kind": "model_call", "trace": "tr1", "seq": 3, "user": "ann", "agent": "refund-agent",
     "model": "", "input_ref": "in-b", "output_ref": "out-b"},
    {"id": "r4", "kind": "tool_call",  "trace": "tr1", "seq": 4, "user": "ann", "agent": "refund-agent",
     "action": "refunds:create", "approval": None, "result": "ok"},
    {"id": "r5", "kind": "denial",     "trace": "tr1", "seq": 5, "user": "ann", "agent": "refund-agent",
     "action": "users:delete"},
    {"id": "r6", "kind": "retrieval",  "trace": "tr2", "seq": 1, "user": "ben", "doc_ids": ""},
    {"id": "r7", "kind": "tool_call",  "trace": "tr2", "seq": 2, "user": "ben", "agent": "report-agent",
     "action": "tickets:read", "approval": "none-needed", "result": "ok"},
]

def H(prev, text):
    return hashlib.sha256(f"{prev}|{text}".encode()).hexdigest()[:12]

# Each entry is (record text, stored hash). The first record's previous hash is the text "genesis".
LOG_A = [
    ("tr1|1|ann|retrieval|doc-7@v3", "783071191219"),
    ("tr1|2|ann|model_call|m-1|out-a", "7fb3aba92dbc"),
    ("tr1|3|ann|tool_call|refunds:create|p4", "294dfbab79d8"),
    ("tr1|4|ann|denial|users:delete|no_role", "5ff9dcfabf4e"),
    ("tr1|5|ann|model_call|m-1|out-b", "bc2bba307e3f"),
]
LOG_B = [
    ("tr1|1|ann|retrieval|doc-7@v3", "783071191219"),
    ("tr1|3|ann|tool_call|refunds:create|p1", "294dfbab79d8"),
    ("tr1|4|ann|denial|users:delete|no_role", "5ff9dcfabf4e"),
    ("tr1|5|ann|model_call|m-1|out-b", "bc2bba307e3f"),
]
LOG_C = [
    ("tr1|1|ann|retrieval|doc-7@v3", "783071191219"),
    ("tr1|2|ann|model_call|m-1|out-a", "7fb3aba92dbc"),
    ("tr1|3|ann|tool_call|refunds:create|p4", "06c127b64674"),
    ("tr1|4|ann|denial|users:delete|no_role", "2388664e396b"),
    ("tr1|5|ann|model_call|m-1|out-b", "94a1ff6250cd"),
]
ANCHOR = "bc2bba307e3f"   # the latest hash, saved in a separate store when the log was written

SEEN = {"tr1": [1, 2, 3, 5, 6, 9], "tr2": [1, 2, 3], "tr3": [2, 3, 4]}

Task 1 — Check the records for required fields

First, a carry-in. Count the entries in DENIED, and count those whose reason is other_tenant.

A record needs the fields in COMMON plus the fields in KIND_FIELDS for its kind. A field is missing when the key is absent from the record, or its value is None or the empty text. Write missing(record) to return the set of missing fields, and run it over all seven records.

Add to your evidence file:

t1_carried_denials: <number of entries in DENIED>
t1_carried_other_tenant: <number of entries in DENIED whose reason is other_tenant>
t1_incomplete_ids: <ids of records with at least one missing field, in order, comma-separated, no spaces>
t1_missing_total: <the total number of missing fields across the seven records>
t1_missing_r6: <the missing fields of r6, sorted alphabetically, comma-separated, no spaces>

Output: one Python file with missing, and five evidence lines.

Task 2 — Verify the chains

A log is chained as follows: for the record at position 0 the previous hash is the text genesis, and for every other position it is the stored hash of the entry before. The check at a position passes when H(previous_hash, record_text) equals that entry’s stored hash. Write verify(log) to return the list of positions (counting from 0) whose check fails. Then compare each log’s last stored hash with ANCHOR.

Add to your evidence file:

t2_bad_a: <positions that fail in LOG_A, comma-separated, no spaces, or none>
t2_bad_b: <positions that fail in LOG_B, comma-separated, no spaces, or none>
t2_bad_c: <positions that fail in LOG_C, comma-separated, no spaces, or none>
t2_a_head_matches_anchor: <true or false: LOG_A's last stored hash equals ANCHOR>
t2_c_head_matches_anchor: <true or false: LOG_C's last stored hash equals ANCHOR>

Output: one verify function, and five evidence lines.

Task 3 — Find the gaps

For each trace in SEEN, the records are expected to be numbered from 1 up to the highest number seen for that trace, with none left out. A gap is a number in that range that is not in the list. Write a function that finds the gaps, and apply it to the three traces.

Write audit-gaps.tsv: one line per trace, in the order of SEEN, with three fields separated by single tab characters: the trace, the number of gaps, and the gaps as a comma-separated list with no spaces, or - when there are none.

Add to your evidence file:

t3_total_gaps: <the total number of gaps across the three traces>
t3_traces_with_gaps: <traces with at least one gap, in order, comma-separated, no spaces>
t3_tr1_gaps: <the gaps of tr1, comma-separated, no spaces>

Output: an audit-gaps.tsv of three lines, and three evidence lines.

What carries forward

Keep audit-gaps.tsv and chapter7-evidence.txt. Chapter 8 is the capstone, a complete design. It opens with the gaps in your audit-gaps.tsv and asks which control in that design would prevent each kind of gap.