Chapter 6 — Provider Abstraction and Model Selection

Course: Getting Started with AI

Prerequisites: Chapters 1 to 5, with the ideas re-defined in the notes before they are used: a request has roles, a response ends with a stop reason and reports token counts, prices are quoted per million input and output tokens, and a reply is validated before anything uses it. You can read and run a short Python script. No account with a model provider and no API key are needed.

What the reader can do after this chapter:

• Put every model call behind one internal function that returns one normalized result, and map two different provider response shapes, including their stop reasons and token counts, onto it.
• Keep model identifiers in configuration, check them for pinning and for retired or deprecated status, and find the call sites that bypass the interface.
• Choose a model for a described workload by filtering on a measured quality threshold and then comparing cost at the workload’s real token mix.

Core question this chapter answers: If your application must work with more than one model, and any model can be retired, how do you structure the calls so that changing the model is one reviewed edit, and how do you decide which model to change to?

Connections:

• Builds on: Chapter 5’s request and its validated reply: the system message, examples, user message, token counts and the schema you wrote for the reply.
• Leads into: Build and Evaluate RAG Applications, which starts the grounding phase: the same request, now with text retrieved by similarity added to its input, and what similarity does and does not mean.

Your answers are scored

This chapter is assessed. The quiz is marked against a key held separately from your materials. Questions 1–8 are multiple choice and are marked by comparing your letter with the key. Questions 9–10 ask for an answer in an exact format, and each one is marked by a rule that the format printed with the question lets you check.

• Nothing is judged by reading it for style. An answer either matches the rule or it does not.
• All three project tasks are scored from an evidence file of key: value lines, and Task 3 also from the files it asks you to bring forward. Every accepted value, or the rule that decides it, follows from the data, formulas and rules printed in the project, so you can check your own work before you submit it.

Getting Started with AI — Chapter 6: Provider Abstraction and Model Selection

Before you start

Nothing to install for the reading. For the project you need Python 3, its standard library, and the tiktoken package for one counting step (the same package as before). You do not need an account with any model provider, an API key or any payment: the project’s providers and prices are invented for the exercise and every response in it is printed.

Keep one question in mind while you read: if the model I call today stopped being available tomorrow, how many places in my code would I have to change? By the end of the chapter you should be able to answer that with a number, and to bring it down to one.

Summary

Models are retired, replaced and repriced, and every provider describes its request, response, stop reasons and token counts in its own way. An application that calls a provider’s SDK from many places turns each of those changes into a search through the codebase, so the calls belong behind one internal interface that returns one normalized result, with the model identifiers held in configuration and pinned to a specific version. Providers publish compatibility layers that let one provider’s library talk to another, but the documentation lists differences, and unsupported fields are mostly ignored rather than rejected, so a compatibility layer is not an abstraction by itself. When a model must change, because it is retiring or because a better option exists, the decision rests on an evaluation set built from your own prompts and data, and on cost measured at your real token mix. A cheaper model that fails the quality bar is not a candidate, and the cheapest model that passes can change when the workload’s shape does.

Key concepts

Term	Definition
Provider	A company or service that serves models through an API, such as the ones whose documentation this chapter cites.
Call site	A place in your code that calls a provider’s API directly.
Adapter	A small piece of code that translates between your application’s own request and result types and one provider’s request and response formats.
Normalized result	Your application’s own type for the outcome of a model call, with fixed fields such as text, stop reason and token counts, whatever the provider’s response looked like.
Model identifier	The string sent in a request to say which model to use.
Snapshot	A specific, dated version of a model. Pinning means configuring a snapshot’s identifier, so the behaviour does not change unless you change the configuration.
Lifecycle status	One provider’s terms: active (fully supported), legacy (no more updates, may be deprecated), deprecated (still works, not recommended, with a retirement date) and retired (unavailable; requests fail).
Compatibility endpoint	A provider’s interface that accepts another provider’s library and request format, so existing code can call it by changing a few settings.
Evaluation set	A fixed collection of inputs with a way to score the outputs, built from your own use case, used to compare models and prompts.
Routing	Choosing, for each request, which provider and model handle it, usually from configuration.

Explanations

The failure, before the explanation

A team’s application was built quickly. Its model calls look like this, in many files:

# billing.py
resp = provider_sdk.create(model="model-a-2025-03-01", messages=msgs, max_tokens=300)
label = resp.choices[0].message.content
# support.py
resp = provider_sdk.create(model="model-a-2025-03-01", messages=other, max_tokens=300)
if resp.choices[0].finish_reason == "length": ...
# reports.py
resp = provider_sdk.create(model="model-a-2025-03-01", messages=third, max_tokens=800)

Then the provider announces that the model is being retired. One provider’s documentation defines a retired model plainly: “The model is no longer available for use. Requests to retired models will fail” (Anthropic, Model deprecations). Another says: “As we launch safer and more capable models, we regularly retire older models. Software relying on OpenAI models may need occasional updates to keep working” (OpenAI, Deprecations). The team has to find every call, change the model string, and check that each file’s reading of the response, choices[0].message.content and finish_reason, is also right for the replacement, which might be from a different provider with a different response shape. The model string was in three files in the example, but the knowledge of how to read a response was too, and that second kind of knowledge is the harder one to find. The rest of the chapter takes the pieces apart: one interface, what it must absorb, why a compatibility layer is not the same thing, how to pin and migrate, and how to choose.

One interface for every call

Start with the fix, in code. The application never calls a provider directly. It calls one function of its own, and that function is the only code that knows about providers:

result = complete(task="classify", messages=msgs, max_tokens=300)
# result.text, result.stop_reason, result.input_tokens, result.output_tokens

Behind complete, a configuration maps the task to a provider and a model, and one adapter per provider turns your request into that provider’s format and its response into your Result. Changing the model for a task is a change to the configuration; changing the provider is a new adapter and a configuration change; neither touches billing.py.

A second way to see it: a database layer in an application. Application code asks for “the customer with this id” and does not care whether the data sits in one engine or another; the driver absorbs the differences. The model call is the same kind of boundary. And a third picture, a travel plug adapter: the appliance is yours and does not change; the adapter handles the socket.

What does the interface have to absorb? The differences are concrete, and each appears in this chapter’s sources:

Difference	Example from the documentation	Why it matters to your code
Where standing instructions go	The Anthropic compatibility layer says that “system/developer messages are hoisted and concatenated to the beginning of the conversation, as Anthropic only supports a single initial system message”	An application that places system messages mid-conversation gets different input on one provider
Stop reason names	A stop_reason of max_tokens on one provider (Anthropic, Structured outputs) and a finish_reason of length on another (OpenAI, Structured model outputs)	Code that checks for truncation must know both words, or one normalized word
Token-count fields	cache_read_input_tokens on one provider (Anthropic, Prompt caching) and usage.input_tokens_details.cached_tokens on another (OpenAI, Prompt caching)	Cost logging reads different fields per provider
Supported parameters	The same layer lists temperature as “between 0 and 1 (inclusive)”, with values above 1 capped at 1, n as “must be exactly 1”, and seed as ignored	A setting that works on one provider can change meaning or be dropped on another

The interface’s job is to give your application one vocabulary for these: one word for “stopped because the limit was reached”, one pair of token-count names, one place where system text goes. Everything provider-specific stays inside an adapter. A good test of the boundary is a search: the name of a provider’s SDK appears in the adapter files and nowhere else.

Do not over-build it. A normalized result with four fields, a mapping table per provider for stop reasons, and a configuration file is enough to start. An interface that tries to expose every provider feature through one shape tends to become the lowest common denominator or a copy of one provider’s API, so keep provider-specific options in the adapter’s own configuration and add a field to the result only when your application needs it.

A compatibility layer is not an abstraction

Providers publish ways to reach their models with another provider’s library. One says its models “are accessible using the OpenAI libraries (Python and TypeScript / Javascript) along with the REST API, by updating three lines of code and using your Gemini API key” (Google, OpenAI compatibility); another documents an OpenAI SDK compatibility layer for its models (Anthropic, OpenAI SDK compatibility). That is useful for a quick test of whether a different model could do your task.

It is not the same as owning an interface, because the differences do not disappear. The Anthropic page calls the list of differences “the most substantial” ones, and includes: “The strict parameter for function calling is ignored, which means the tool use JSON is not guaranteed to follow the supplied schema”; response_format is “ignored”; “prompt caching is not supported, but it is supported in the Anthropic SDKs”; and “most unsupported fields are silently ignored rather than producing errors”. For guaranteed schema conformance the page points to the native API with its Structured Outputs feature. Silent ignoring is the dangerous part: code that asks for a feature and gets no error cannot tell that it did not get the feature. The Chapter 5 validator would catch a reply that does not match the schema, but it would not tell you that the cause was a field the layer ignored.

The same page adds a warning that bears on the next section: “If you’ve done lots of tweaking to your prompt, it’s likely to be well-tuned to OpenAI specifically. Consider reworking it for Claude.”

Use a compatibility layer when	Use the native interface (behind your adapter) when
You want a quick comparison of a task on another model	You depend on a feature the layer ignores or does not support, such as strict schemas or caching
The task uses only the fields the page lists as fully supported	A silent difference would go unnoticed in production
It is a throwaway test	The model will serve real traffic

Pinning and configuration: the model is data, not code

Picture a support-ticket classifier whose model string, an alias with no version, is typed into eleven files. One morning its labels shift, and nobody changed a line: the alias now points at a newer snapshot. Finding and testing the fix means editing eleven files. Where should the model identifier live? Not in the call sites. It lives in configuration that the adapter reads, as a pinned, specific version. The OpenAI guide says that “even different snapshots of models within the same family could produce different results” and recommends “pinning your production applications to specific model snapshots” with tests and evaluations to monitor behaviour when you change versions (OpenAI, Prompt engineering). Pinning keeps behaviour from changing under you without an edit; it also means that the edit, when it comes, is deliberate, reviewed and tested.

Pinning is not a way to avoid change. Models have a lifecycle. Anthropic’s documentation names four states: “Active: The model is fully supported and recommended for use. Legacy: The model will no longer receive updates and may be deprecated in the future. Deprecated: The model is still functional but no longer recommended. Anthropic provides a recommended replacement and assigns a retirement date. Retired: The model is no longer available for use.” It advises that, once a model is deprecated, you should “migrate all usage to a suitable replacement before the retirement date”, and “consider thorough testing of your applications with the new models well before the retirement date”. It also says partner-operated platforms set their own retirement schedules, so a model’s status and dates can differ by platform (Anthropic, Model deprecations). OpenAI’s page lists deprecations with recommended replacements and says it documents minimum notice periods before retirement (OpenAI, Deprecations).

Two habits follow. First, know where you stand: Anthropic describes an audit, an export from its Console usage page of API usage broken down by API key and model, to find any use of deprecated models. A configuration file you own gives you the same answer for free, because the models you use are listed in one place. Second, treat the lifecycle status as an input to your own checks: a configuration that points at a retired model, or at an identifier that carries no version, is a defect a script can find before a production call does.

What you see	Likely cause	What to do instead
A model string in many files	The identifier lives in code	Move it to one configuration read by the adapter
An identifier with no version	Behaviour can change without an edit	Pin a specific version and test before changing it
A model listed as deprecated	A retirement date is approaching	Plan the migration now, using your evaluation set

Migrating: a model change is an experiment

Think of a change of database engine. No team changes it because a blog post says the new one is faster; they run their own queries and their own data on it first. A model change is the same, and your evaluation set is your queries and your data.

When a model has to change, because it is retiring or because another looks better, treat the change as an experiment with a result. Anthropic’s guidance on choosing a model: “Create benchmark tests specific to your use case - having a good evaluation set is the most important step in the process. Test with your actual prompts and data.” It lists what to compare across models: accuracy of responses, response quality, handling of edge cases, and “weigh performance and cost tradeoffs” (Anthropic, Choosing the right model).

The steps are short. Keep the evaluation set, run it on the current model and record the scores, run it on the candidate with the same prompt, and read the differences. Then try the prompt changes the earlier warning suggests, and run again. Only then change the configuration. The Chapter 5 validator belongs in the loop: a candidate that returns the right label in the wrong shape fails the contract before it fails the quality bar.

The same page names a lever that is often cheaper than switching: for models that support an effort parameter, it says “tuning effort is often a better lever than switching models”, trading intelligence for latency and cost within one model.

Choosing a model: quality first, then cost

Say three models score 0.95, 0.91 and 0.84 on your evaluation set, and the third is by far the cheapest. If your bar is 0.90, the third is out before price is even read; the choice is between the first two, on cost. Which model to move to is a selection problem with two stages. The first stage is a filter. Run your evaluation set and keep only the models whose score meets the bar you set for the task, say 0.90 on the fraction of cases passed. A cheaper model that misses the bar is not a candidate, however cheap. The second stage is a comparison of cost among those that remain, at your actual token mix.

Here is why the mix matters, with an invented price table used by this chapter’s project. Prices are per million tokens, input then output, with each model’s measured score on a 50-case evaluation set:

Model	Input	Output	Score
alpha-lite	0.30	1.00	0.84
alpha-heavy	0.40	12.00	0.93
beta-mini	0.90	3.00	0.91
beta-pro	5.00	15.00	0.97

With a bar of 0.90, alpha-lite is out. Among the other three, the cheapest depends on the shape of the request. For a request with 1,000 input tokens and 300 output tokens, beta-mini costs 1,000 × 0.90 ÷ 1,000,000 + 300 × 3.00 ÷ 1,000,000 = $0.0018, against $0.0040 for alpha-heavy. For a request with 20,000 input tokens and 100 output tokens, alpha-heavy costs $0.0092 and beta-mini $0.0183. The winner flipped because one model is cheap on input and dear on output and the other is the reverse. And alpha-lite, $0.0061 on the second request, is cheaper than both and still not a candidate, because it fails the bar.

Providers give starting points, and they are starting points. One provider describes two approaches: start efficiency-first with a faster, more cost-effective model, test thoroughly and “upgrade only if necessary for specific capability gaps”; or start capability-first with the strongest model for a complex task and then “consider increasing efficiency by lowering effort or downgrading models” (Anthropic, Choosing the right model). Its selection matrix pairs “the lowest latency and price” with its Haiku 4.5 model. Whether that matches your task is what your evaluation set measures. The same page also describes multi-model strategies that pair a lower-cost model with a frontier model, an executor that escalates hard decisions to an advisor or an orchestrator that delegates bulk work to lower-cost workers, so that most tokens are billed at the lower rate. All of these are choices your interface should make easy: if the model is configuration, trying another is one edit.

Requirement in the request	Best choice	It is the distractor when	Trade-off
Strict latency, high volume, simple task	A fast, low-cost model that clears the quality bar	The task has capability gaps the evaluation set exposes	Lower cost and latency, risk of failing hard cases
Hard reasoning where accuracy outweighs cost	A stronger model	The evaluation set shows a cheaper model passes	Higher cost per request
Mixed difficulty	A cheaper model with escalation to a stronger one	Every request is hard, so escalation adds latency	Two models to configure and evaluate

Three situations, and what they mean for your design

Before the table, three situations (illustrations, not measurements):

1. A team finds 40 files that call a provider’s SDK directly. The provider has announced the retirement of the model in all of them.
2. A team switches to another provider through its compatibility endpoint. Their JSON replies start arriving with prose around them, though the same prompt worked before and they set the same response format option.
3. A team picks the cheapest model on a price list. After launch, its answers on hard cases are wrong far more often than before.

What you see	Which part of the design	What to do
40 direct call sites (situation 1)	No interface	Add one complete function and adapters; move call sites behind it one at a time; keep the SDK name in adapter files only
A setting has no effect (situation 2)	A compatibility layer that may ignore the field	Check the layer’s list of supported fields; use the native interface, or the native structured-output feature, behind an adapter; validate replies
A cheap model fails hard cases (situation 3)	Selection by price alone	Build an evaluation set from your own cases, set a quality bar and choose the cheapest model that clears it

Flashcards

Say the answer aloud before revealing it. Speaking it is what exposes the gaps that reading past a written answer hides.

Why is calling a provider's SDK from many places a trap?

A model’s retirement, a price change or a provider switch becomes an edit at every call site, and each file also contains its own knowledge of how to read that provider’s response. One interface with adapters keeps both in one place.

What does the single internal interface return, and why?

One normalized result with fixed fields such as text, stop reason and token counts, whatever the provider’s response looked like, so the rest of the application has one vocabulary.

Name three differences between providers that an adapter has to absorb.

Where system instructions go, the names of stop reasons and token-count fields, and which request parameters are supported or ignored. Each appears in the documentation cited in the notes.

Why is a compatibility endpoint not an abstraction?

It lets one provider’s library call another’s models, but the documentation lists differences, such as ignored parameters, and most unsupported fields are silently ignored rather than rejected. You still need your own boundary and checks.

What does pinning mean, and what does it not do?

It means configuring a specific model snapshot so behaviour does not change without your edit. It does not stop retirement: a pinned model can still be deprecated and then retired, so you still plan migrations.

What are one provider's four lifecycle states?

Active, legacy, deprecated and retired. Deprecated models still work but come with a recommended replacement and a retirement date; requests to retired models fail.

Where should the model identifier live, and why?

In configuration read by the adapter, as a pinned version, so changing it is one reviewed edit and the models in use can be listed from one place.

What does the provider guidance say is the most important step in deciding whether to change models?

Create benchmark tests specific to your use case: a good evaluation set, run with your actual prompts and data, comparing accuracy, quality, edge cases, performance and cost.

Why can a prompt tuned for one provider need rework for another?

The provider’s own compatibility page warns that a heavily tweaked prompt is likely tuned to the original provider, and suggests reworking it. Run your evaluation set again after any change.

What are the two stages of choosing a model in this chapter?

First filter by measured quality against a bar you set, then compare cost among the models that pass, at your actual input and output token mix.

A cheaper model is cheapest on your workload but misses the quality bar. What do you do?

Drop it from consideration. Cost only ranks the models that clear the bar; consider lower effort or a combination of models if you need to reduce cost further.

Model answer

The question: “Our application calls one model through a provider’s SDK in dozens of places, and that model is being retired. How should it be structured, and how do you pick the replacement?” This is the question a lead engineer asks in a design review after the failure above. A strong answer hits five beats, in this order:

1. Interface — every call goes through one internal function that returns a normalized result.
2. Adapters — each provider’s request and response differences live in one adapter, and a compatibility layer is not a substitute.
3. Configuration — model identifiers are pinned, kept in configuration, and checked against lifecycle status.
4. Evaluation — a candidate is judged on an evaluation set built from your own prompts and data.
5. Selection — filter by the quality bar, then compare cost at your real token mix.

“I’d put every model call behind one function of our own that returns a normalized result: the text, a stop reason in our own vocabulary, and input and output token counts. Each provider gets an adapter that translates our request and its response, so the SDK name appears only there, and I wouldn’t treat a compatibility endpoint as the interface, because the documentation lists ignored parameters and silently ignored fields. The model identifier would be pinned in configuration, not in code, and a script would flag any identifier that is unversioned, deprecated or retired. For the replacement I’d build an evaluation set from our own prompts and data, run the current model and each candidate, rework the prompt if a provider’s guidance suggests it, and check the replies against our schema. Then I’d drop every candidate below the quality bar and compare the cost of the rest at our actual input and output sizes, because the cheapest can change with the shape of the request. The change itself is then one reviewed edit to the configuration.”

Missing a beat is a failure state. Without interface a retirement is a hunt through the codebase. Without adapters the differences leak back in. Without configuration the model is a string in code and nobody knows which are in use. Without evaluation the replacement is a guess. Without selection the answer picks by price list — and that is the beat reviewers most often find missing.

Why this matters

• Continuity. Providers document that older models are retired. An application whose model identifier is one reviewed edit survives that; one whose identifier is scattered does not.
• Bargaining and options. With one interface, trying another provider or a cheaper model is an experiment, not a rewrite.
• Cost. The cheapest model for one workload shape is not the cheapest for another, and the cheapest that fails the quality bar is not a saving.
• Operations. A list of the models in use, with their lifecycle status, is something an on-call engineer can read before a retirement date, not after it.

After this chapter

The project for this chapter is One Interface, Many Models (project.md). You will normalize two providers’ response shapes into one result, audit a configuration for unpinned, deprecated and retired models and count a codebase’s direct call sites, and finish by choosing a model for your own request from a price and quality table.

Build and Evaluate RAG Applications starts the grounding phase with the same request. It adds text retrieved by similarity to the request’s input, and asks what similarity does and does not mean, and what goes wrong when it is assumed to mean more than it does.

Hands-on Project — One Interface, Many Models

Objective

Normalize two providers’ response shapes into one result, audit a model configuration for unpinned, deprecated and retired models and count the direct call sites in a small codebase, and finish by choosing a model for your own request from a price and quality table, so that a change of model is a configuration edit and the choice of replacement is a calculation.

What this project needs: Python 3 with the standard library only for Tasks 1 and 2. Task 3 counts tokens and so also needs the tiktoken package (pip install tiktoken, which needs an internet connection). The package, its cl100k_base encoding and the enc.encode call are the ones in the OpenAI Cookbook, How to count tokens with tiktoken. No task needs an account with a model provider, an API key or any payment, and no task calls a model. The two providers in this project, alpha and beta, their models and their prices are invented for the exercise; they are not real services or real prices. All three tasks are required.

Your evidence file. Keep one plain-text file, chapter6-evidence.txt. Each task tells you which lines to add. Every line has the form key: value, one per line, with the key in lower case exactly as printed. Write what your code printed, not what you expected it to print.

Task 1 — Normalize two response shapes

The two invented providers return responses in different shapes. Copy these six canned responses exactly:

ALPHA = [
    {"output": {"text": "bug", "stop": "end"}, "usage": {"in": 73, "out": 1}},
    {"output": {"text": "cut off here", "stop": "cutoff"}, "usage": {"in": 120, "out": 300}},
    {"output": {"text": "I cannot help with that.", "stop": "declined"}, "usage": {"in": 95, "out": 7}},
]
BETA = [
    {"choices": [{"message": {"content": "question"}, "finish_reason": "stop"}],
     "usage": {"prompt_tokens": 80, "completion_tokens": 1}},
    {"choices": [{"message": {"content": "partial"}, "finish_reason": "length"}],
     "usage": {"prompt_tokens": 150, "completion_tokens": 400}},
    {"choices": [{"message": {"content": ""}, "finish_reason": "tool_calls"}],
     "usage": {"prompt_tokens": 60, "completion_tokens": 0}},
]

Write normalize_alpha(resp) and normalize_beta(resp). Each returns a tuple (text, stop_reason, input_tokens, output_tokens), where the stop reason is one of your application’s own words, mapped like this:

Alpha stop	Beta finish_reason	Your stop_reason
end	stop	done
cutoff	length	length
declined	content_filter	refusal
anything else	anything else	unknown

Alpha’s token counts are usage.in and usage.out; Beta’s are usage.prompt_tokens and usage.completion_tokens. Beta’s text is choices[0].message.content; Alpha’s is output.text.

Print each of the six results as one line, text|stop_reason|input_tokens|output_tokens, in the order above (the first three from ALPHA, then the three from BETA). Then print the sum of all six input token counts and the sum of all six output token counts.

Add to your evidence file. The text part of a line may be empty:

t1_a1: <line for ALPHA[0]>
t1_a2: <line for ALPHA[1]>
t1_a3: <line for ALPHA[2]>
t1_b1: <line for BETA[0]>
t1_b2: <line for BETA[1]>
t1_b3: <line for BETA[2]>
t1_total_in: <sum of input tokens>
t1_total_out: <sum of output tokens>

Output: one Python file with two functions, and eight evidence lines. Stop there: two functions and one mapping, not a client library.

Task 2 — Audit a configuration and a codebase

Here is a routing configuration, a lifecycle table for the invented models, and a tiny codebase held as a dictionary of file names to source text. Copy them exactly:

CONFIG = {
    "classify": {"provider": "alpha", "model": "alpha-1-2026-01-15"},
    "summarize": {"provider": "beta", "model": "beta-latest"},
    "extract": {"provider": "beta", "model": "beta-2-2025-06-30"},
    "route": {"provider": "alpha", "model": "alpha-0-2025-03-01"},
}

LIFECYCLE = {
    "alpha-1-2026-01-15": "active",
    "beta-latest": "active",
    "beta-2-2025-06-30": "deprecated",
    "alpha-0-2025-03-01": "retired",
}

FILES = {
    "billing.py": 'def f(x):\n    return alpha_call({"m": "alpha-1-2026-01-15", "p": x})\n',
    "support.py": 'r = beta_call(payload)\nr2 = beta_call(payload2)\n',
    "reports.py": 'def g(x):\n    return complete("summarize", x)\n',
    "adapter.py": 'def complete(task, x):\n    return alpha_call(x)\n',
}

This project’s pinning rule, which is an exercise rule: a model identifier is pinned if it ends in a dash followed by a date written YYYY-MM-DD, that is, it matches the regular expression -\d{4}-\d{2}-\d{2}$. Write these three functions using only the standard library:

1. resolve(task, config) — returns (provider, model) for the task.
2. audit(config, lifecycle) — returns three sorted lists of task names: the tasks whose model is not pinned, the tasks whose model is deprecated, and the tasks whose model is retired. A task can be in more than one list.
3. direct_calls(files) — counts call sites, meaning occurrences of the text alpha_call( or beta_call(, in every file except adapter.py, which is the one place allowed to call providers. Return the total and the sorted list of file names that have at least one.

Run them and print the results: resolve("classify", CONFIG), the three audit lists, the number of distinct tasks that appear in at least one list, and the direct-call result.

Add to your evidence file. Write lists as names separated by commas with no spaces, or none if empty:

t2_resolve_classify: <the provider and the model, joined by a slash>
t2_unpinned: <tasks not pinned>
t2_deprecated: <tasks on a deprecated model>
t2_retired: <tasks on a retired model>
t2_tasks_needing_action: <number of distinct tasks in at least one of the three lists>
t2_direct_call_sites: <total call sites outside adapter.py>
t2_files_with_direct_calls: <file names, sorted>

Output: one Python file with three functions, and seven evidence lines.

Task 3 — Choose a model for your own request

Use the three files you wrote in Chapter 3 and carried since: system.txt, examples.json and user.txt. If you do not have them, make stand-ins now: a system.txt of three to six lines, an examples.json that is a JSON list of three objects each with input and output strings, and a user.txt holding one input. Only token counts are marked.

1. Count tokens with tiktoken.get_encoding("cl100k_base") and len(enc.encode(text)), on the contents exactly as open(name, encoding="utf-8").read() returns them: system.txt; user.txt; and the examples, counting the input string and the output string of every example separately. Add the three counts. This is the request’s input tokens.

2. Here are four invented models, with prices in dollars per million tokens and each model’s score on a 50-case evaluation set. Copy the table exactly:

   MODELS = {
       "alpha-lite":  {"input": 0.30, "output": 1.00,  "score": 0.84},
       "alpha-heavy": {"input": 0.40, "output": 12.00, "score": 0.93},
       "beta-mini":   {"input": 0.90, "output": 3.00,  "score": 0.91},
       "beta-pro":    {"input": 5.00, "output": 15.00, "score": 0.97},
   }

   The quality bar is a score of at least 0.90. A model below it is not a candidate. The cost of a request is input_tokens × input_price ÷ 1,000,000 + output_tokens × output_price ÷ 1,000,000.

3. Scenario A: your request’s input tokens and 300 output tokens. Print the cost for each of the four models to 6 decimal places, and the cheapest model among those at or above the bar.

4. Scenario B: a request with 20,000 input tokens and 100 output tokens. Print the cost for alpha-lite to 6 decimal places, the cheapest model among those at or above the bar, and that model’s cost to 6 decimal places.

5. Write models.json: a JSON object with one key, my_request, whose value has provider, set to alpha or beta to match your Scenario A choice’s name, and model, set to your Scenario A choice followed by -2026-06-01, for example beta-mini-2026-06-01. Check that your audit function from Task 2 reports it as pinned.

Add to your evidence file. Write model names exactly as in MODELS, and the eligible list sorted, with commas and no spaces:

t3_input_tokens: <system + examples + user>
t3_eligible: <models at or above the bar, sorted>
t3_a_alpha_lite: <cost, 6 decimal places>
t3_a_alpha_heavy: <cost, 6 decimal places>
t3_a_beta_mini: <cost, 6 decimal places>
t3_a_beta_pro: <cost, 6 decimal places>
t3_a_choice: <cheapest eligible model for scenario A>
t3_b_alpha_lite: <cost, 6 decimal places>
t3_b_choice: <cheapest eligible model for scenario B>
t3_b_choice_cost: <cost, 6 decimal places>

Output: models.json and ten evidence lines. Submit the Python files from Tasks 1 and 2, the four files (system.txt, examples.json, user.txt, models.json) and the evidence file, so the counts can be re-checked.

What carries forward

Keep system.txt, examples.json, user.txt, models.json and chapter6-evidence.txt. The next course, Build and Evaluate RAG Applications, starts from the same request and adds retrieved text to its input, so the token counts and the model you chose here are what that text will have to fit beside.