Session 1 of 8

What a Language Model Actually Computes

Chapter 1 — What a Language Model Actually Computes

Course: Getting Started with AI

Prerequisites: You can read and run a short Python script, and you have called an HTTP API from code or from the command line. You have used a chat assistant at least once. No machine-learning background is assumed: tokens, context windows, probabilities and sampling are all defined in the notes before they are used.

What the reader can do after this chapter:

Explain, for a given prompt and settings, why a model returned the output it did, naming the tokens it read, the next-token probabilities it produced and the sampling rule that picked from them.
Predict which candidate tokens survive a temperature, top-k or top-p setting when given the model’s next-token probabilities, and say how the choice of setting changes the variety of output.
Diagnose a response that was cut short or that changed between two identical requests, and name the cause from the response’s own fields: its stop reason and its token counts.

Core question this chapter answers: When you send the same input to a language model twice and get two different answers, what exactly did the model compute each time, and which part of that computation decided the difference?

Connections:

Builds on: ordinary API work — a request goes in, a response comes back with a body and a few metadata fields. This chapter shows what happens between the two.
Leads into: Chapter 2, which takes the token counts you measure in this chapter’s project and turns them into two numbers a production design has to answer for: what a request costs and how long the reader waits for it.

Your answers are scored

This chapter is assessed. The quiz is marked against a key held separately from your materials. Questions 1–8 are multiple choice and are marked by comparing your letter with the key. Questions 9–10 ask you to write a short answer in an exact format, and each one is marked by a rule printed with the question, so you can check that your answer has the right shape before you submit it.

Nothing is judged by reading it for style. An answer either matches the rule or it does not.
All three project tasks are scored from an evidence file of key: value lines. Every line’s accepted value, or the rule that decides it, follows from what the task prints, so you can check your own file before you submit it.

Getting Started with AI — Chapter 1: What a Language Model Actually Computes

Before you start

Nothing to install for the reading. For the project you need Python 3 and the ability to install one package with pip. You do not need an account with any model provider, and you do not need an API key, unless you choose to do the optional third task.

Keep one question in mind while you read: if I sent this exact request again, would I get this exact response back, and how would I know? By the end of the chapter you should be able to answer it for any request, from the request’s settings and the response’s own fields.

Summary

A language model does not read words, store facts in a database or plan a whole answer before writing it. It reads a sequence of tokens, computes a probability for every token that could come next, and a sampling rule picks one of them; the chosen token is appended to the sequence and the loop runs again until a stop condition is met. Everything the model can use while doing this must fit inside its context window, which holds the input and the output it is writing, and is measured in tokens rather than in characters or words. Most of the variety you see between two runs of the same prompt comes from the sampling step, not a fault in the model: temperature reshapes the probabilities, and top-k and top-p limit which candidates the step may draw from. This mental model — tokens in, a probability distribution out, a sampling rule between them, all inside a fixed window — is what every later chapter on prompting, retrieval, agents and evaluation is built on.

Key concepts

Term	Definition
Token	The unit a model reads and writes. A token can be a whole word, part of a word, a space plus a word, or a punctuation mark. Text is converted to a list of token ids before the model sees it.
Tokenizer	The fixed procedure, and its vocabulary, that turns text into token ids and back. Different model families use different tokenizers, so the same text can have different token counts on different models.
Context window	Everything the model can reference when producing a response: the instructions, the conversation so far, any documents or tool results, and the response being written. Its size is a token limit.
Next-token distribution	The list of probabilities, one for every token in the vocabulary, that the model computes for the next position. The probabilities add up to 1.
Autoregressive generation	Producing text one token at a time, where each new token is chosen using all the tokens before it, and then becomes part of the input for the next one.
Greedy decoding	A sampling rule that always picks the single most probable next token. Given the same distribution, it always makes the same choice.
Sampling	Picking the next token at random, in proportion to its probability, so a token with probability 0.3 is chosen about three times in ten.
Temperature	A setting that sharpens or flattens the distribution before sampling. Below 1 it makes the likeliest tokens more likely still; above 1 it spreads probability towards less likely tokens.
Top-k	A filter that keeps only the k most probable tokens, then redistributes the probability among them before sampling.
Top-p (nucleus)	A filter that keeps the smallest set of most-probable tokens whose probabilities add up to at least p, then redistributes the probability among them.
Stop reason	A field in the response that says why generation ended: the model finished, a length limit was hit, a stop sequence appeared, or the model called a tool.
Context rot	The documented loss of accuracy and recall as the number of tokens in the context grows, even while the context is still inside its limit.

Explanations

The failure, before the explanation

Here is a support bot running in a staging environment. The engineer who built it sends it the same ticket text twice, back to back, with the same settings:

Ticket: “Deploy to staging failed. Log ends with: Killed after npm run build.”

Reply 1: “The build process was most likely killed by the operating system for running out of memory. Increase the memory limit for the build container and retry.”

Reply 2: “This usually means the build step exceeded a resource limit. Check whether your CI runner enforces a timeout or a memory cap, and look for an OOM message in the system log.”

Neither reply is wrong, and neither is a copy of the other. The engineer’s first instinct — the one a backend or QA background trains into you — is that a function given the same input should return the same output, so something must be broken: a cache, a load balancer routing to two different model versions, a race. Sometimes one of those is true. But the replies can differ even when none of them is, because the difference is designed in. To see why, you need to know what the model computed on each request.

Tokens: what the model actually reads

Start with a concrete case. The tokenizer that one widely used model family uses, called cl100k_base, turns the string tiktoken is great! into six tokens. Decoded back to text, they are t, ik, token, is, great and ! (OpenAI Cookbook, How to count tokens with tiktoken; the split was also checked by running the tiktoken package on 2026-10-03). Two things stand out. The package name tiktoken became three pieces, because this tokenizer’s vocabulary has no single token for it. And the space before is and great is part of the token — the model sees “ is“, not “is”.

A second way to picture it: a token is to a model what a byte is to a network buffer. You do not reason about a TCP payload in “words”; you reason about bytes, because that is what the size limits and the transfer costs are counted in. A model’s context window and its usage-based price are counted in tokens, so that is the unit you estimate them in.

Not every limit is a token limit, though. One provider’s documentation describes two kinds of limit on an account: “Spend limits set a maximum monthly cost an organization can incur for API usage” and “Rate limits set the maximum number of API requests an organization can make over a defined period of time” (Anthropic, Rate limits). A request-per-minute cap or a monthly spend cap can stop a request that fits comfortably in the window. Tokens tell you whether a request fits and what it costs; they do not tell you whether the account will accept it right now.

For rough planning, one provider’s documentation gives two rules of thumb for its own models: a token is about four characters, and 100 tokens is about 60–80 English words (Google, Gemini API — Understand and count tokens). Treat these as estimates for English prose on that model family only. Text with many rare words, code, numbers or a non-Latin script usually needs more tokens per character, and a different model family’s tokenizer gives a different count for the same text. When the number matters, count it with the tokenizer of the model you are actually calling.

Why this matters immediately: the OpenAI Cookbook names the two reasons to count tokens — whether the text “is too long for a text model to process” and “how much an … API call costs (as usage is priced by token)”. Both of those are decisions you will make as an engineer, and both are made in tokens.

The context window: the model’s whole world for one request

Picture a whiteboard of a fixed size. Before the model writes anything, everything it is allowed to use — your instructions, the conversation so far, the pasted log file, the retrieved documents, the tool results — must be written on that whiteboard. Then the model writes its answer on the same whiteboard, in the space that is left. When the board is full, nothing more fits.

That is the context window. One provider defines it as “all the text a language model can reference when generating a response, including the response itself”, and contrasts it with the training data: it is a “working memory” for the model (Anthropic, Context windows). Another provider says the same thing in one line: “the context window defines the combined limit of input and output tokens” (Google, Gemini API — Understand and count tokens).

Three consequences follow, and each one shows up in production:

The model has no memory between requests except what is put back on the board. In a chat built on an API that keeps no conversation state, the “memory” of earlier turns exists because the application sends the earlier turns again with each request. Each user message and assistant response accumulates in the window (Anthropic, Context windows). Some APIs and products store the conversation for you; they still have to put it back into the window to use it.
Output competes with input for the same space. A huge pasted log leaves less room for the answer. One provider documents a stop reason for exactly this case: generation that fills the model’s context window stops with model_context_window_exceeded, and an input that already exceeds the window is rejected with an error before anything is generated (Anthropic, Context windows).
Fitting is not the same as being used well. The same documentation warns that “as token count grows, accuracy and recall degrade”, calls this context rot, and concludes that what you put in the context matters as much as how much room there is. A larger window is not a licence to paste everything.

The next-token distribution: what the model computes

Here is the core of the chapter, shown with numbers before it is stated as a rule.

Suppose the sequence so far is The deploy failed because the and the model is computing the next token. It does not produce one answer. It produces a probability for every token in its vocabulary. Most are close to zero. Here are five of the candidates, with probabilities from a small worked example used throughout this chapter and its project:

Candidate next token	Probability
`server`	0.445
`build`	0.270
`network`	0.164
`config`	0.099
`cat`	0.022

(For this teaching example the five probabilities have been rescaled to add up to 1 on their own, as if these were the only tokens. A real vocabulary has many thousands of entries.)

That table is the model’s entire output for this position. Which token actually appears in the response is decided by the next step, sampling, which the model itself does not do. Then the chosen token is appended to the sequence, the model computes a fresh distribution for the next position, and the loop repeats.

This loop has a name. A widely used introduction to text generation describes it like this: “the probability distribution of a word sequence can be decomposed into the product of conditional next word distributions” (Hugging Face, How to generate text). In plain terms: an answer is a chain of next-token choices, each made with the full sequence before it in view and nothing after it. Another course describes the same idea for this family of models: the output “depends on the past and present inputs, but not the future ones” (Hugging Face LLM Course, How do Transformers work?).

A second way to picture it: autocomplete on your phone, run in a loop, with a far better model of which word comes next. Your keyboard offers three suggestions; the model scores every possible next token. Your thumb picks one; a sampling rule picks one. The picked word changes what the keyboard suggests next. Nothing in that loop plans the end of the sentence before it reaches it.

One consequence is easy to miss. Because each token is chosen with only the earlier ones in view, a model that picks an unlucky token early must continue from it. If cat were picked above, every later token would be computed to follow The deploy failed because the cat, and the model would do its best to make that sentence coherent. It does not go back.

Sampling: where the variety comes from

Now return to the two support-bot replies. The model computed a distribution at each position; the difference between the two replies was decided by how a token was picked from it.

Greedy decoding always picks the most probable token: server, every time. The Hugging Face guide defines it as selecting “the word with the highest probability as its next word at each timestep”. Given the same distribution, greedy decoding makes the same choice. The same guide warns that it tends to produce repetitive text.

Sampling picks “randomly … according to its conditional probability distribution” (Hugging Face, How to generate text). With the table above, server is picked about 44 or 45 times in a hundred, build about 27 times, and cat about twice. Run the same request twice with sampling and the first token can differ — and once one token differs, every later distribution is computed from a different sequence, so the two replies drift apart. That is the whole explanation of the support bot: two runs, two draws, two different but reasonable replies.

Three settings shape the draw. Each one is shown on the same table.

Temperature sharpens or flattens the distribution before the draw. Lowering it below 1 “makes high-probability words more likely while reducing low-probability candidates” (Hugging Face, How to generate text). On the worked example, temperature 0.5 raises server from 0.445 to about 0.643 and pushes cat down to about 0.002; temperature 2.0 lowers server to about 0.325 and raises cat to about 0.072. Picture a dimmer switch for surprise: turn it down and the model nearly always takes the obvious word; turn it up and long shots get picked more often.

Top-k keeps the k most probable tokens and redistributes the probability among them (Hugging Face, How to generate text). With k = 2, only server and build remain, and their shares become about 0.62 and 0.38. Top-k uses a fixed pool size whatever the distribution looks like.

Top-p, also called nucleus sampling, “chooses from the smallest set of words whose cumulative probability exceeds probability p” (Hugging Face, How to generate text). Walk down the table adding as you go: 0.445, then 0.715, then 0.879. With p = 0.8, the running total first reaches 0.8 at network, so the pool is server, build and network. The pool size adapts: when the model is confident, a few tokens reach p quickly; when it is unsure, many tokens are needed.

Settings interact. Apply temperature 0.5 first and the table becomes 0.643, 0.237, 0.087, 0.032, 0.002; now the running total reaches 0.8 at build (0.643 + 0.237 = 0.880), so the same top-p of 0.8 keeps only two tokens instead of three. Lowering temperature made the nucleus smaller.

Why “set it to deterministic” is not a full answer

The obvious fix for the support bot is to remove the randomness. That helps, but it is not a guarantee, and providers say so in writing.

One provider offers a seed parameter that makes “a best effort to sample deterministically”, and states plainly: “Determinism is not guaranteed”. It adds that there is “a small chance that responses differ even when request parameters and system_fingerprint match”, and asks you to watch the system_fingerprint field to see when the serving backend has changed (OpenAI Cookbook, How to make your completions outputs consistent with the new seed parameter).
Another provider’s newer models do not let you change temperature: its API reference marks the parameter deprecated for models released after a named model version and says that only a value of 1.0 is accepted on them (Anthropic, Messages API reference).
A third provider’s API describes seed as a way to get consistent outputs across requests, beside the usual temperature, topP and topK (Google, Gemini API — Generating content).

So the knobs differ by provider and even by model version. What does not differ is the structure: a distribution is computed, and something picks from it. When an engineer needs repeatable output, the dependable tools are the ones that do not rely on the draw at all: test the property you need (the reply names a memory limit) rather than the exact string, record the settings and the backend identifier with every response, and constrain the output’s form — topics later chapters cover.

Stop reasons: reading why a response ended

A response that ends mid-sentence looks like a bug in the model. Often it is a setting. Both providers documented below report why generation stopped, in a field on the response, and when your provider has such a field it is the first thing to read.

One provider’s documented values include (Anthropic, Handling stop reasons):

Stop reason	What it means
`end_turn`	The model finished its response naturally.
`max_tokens`	The response reached the `max_tokens` limit you set.
`stop_sequence`	The model produced one of the stop sequences you supplied.
`tool_use`	The model is asking to call a tool.
`model_context_window_exceeded`	The response filled the model’s context window.

The same documentation says that when the reason is max_tokens or model_context_window_exceeded, “the response is truncated”. Another provider reports the same facts under different names: STOP for a natural finish or a stop sequence, and MAX_TOKENS when the output limit was reached first (Google, Gemini API — Generating content). Learn the idea, then look up your provider’s spelling.

max_tokens is a ceiling, not a target. The reference says the model “may stop before reaching this maximum” — it only sets the absolute maximum (Anthropic, Messages API reference). A low ceiling silently truncates; a high one does not make answers longer.

Three situations you will meet, and what they mean for your design

Before the table, three situations:

A QA engineer writes an automated test that asserts the model’s reply equals a stored string. It passes on Monday and fails on Tuesday with no code change.
A DevOps engineer pastes a complete build log into a prompt “so the model has everything”. The reply is vague about the actual error, which is near the start of a very long log.
A backend engineer sees JSON responses from a model occasionally fail to parse. Every failing response ends without a closing brace.

What you see	What the model computed	What to do instead
Same input, different output (situation 1)	A fresh draw from the same distribution, or a backend change	Assert properties, not exact strings; log settings and backend id with every response
A long input, a vague answer (situation 2)	The answer was conditioned on a crowded window; recall degrades as tokens grow	Put only what the task needs into the context; extract the relevant part of the log first
Output cut off mid-structure (situation 3)	Generation stopped at the token ceiling	Read the stop reason; treat a length stop as a failure, not as an answer

Flashcards

Say the answer aloud before revealing it. Speaking it is what exposes the gaps that reading past a written answer hides.

What does a language model output at each step of generation?

A probability for every token in its vocabulary — the next-token distribution. It does not output a single word; a separate sampling rule picks one token from that distribution.

Why can a single English word become more than one token?

The tokenizer has a fixed vocabulary. A word that is not common enough to have its own entry is split into smaller pieces that are in the vocabulary — tiktoken becomes t, ik and token.

What counts against the context window?

Everything the model can reference for the request — instructions, earlier turns, documents, tool results — plus the output it is generating. Input and output share the same limit.

How does a chat model "remember" earlier messages?

The model itself keeps nothing between requests. With an API that keeps no conversation state, the application sends the earlier turns again as part of each new request, so they are on the whiteboard each time; if they are not sent, the model does not see them.

If a context still fits in the window, is more context always better?

No. Accuracy and recall degrade as the token count grows — context rot. What goes in matters as much as how much room there is.

What does greedy decoding do, and what is its known weakness?

It always picks the single most probable next token, so the same distribution always gives the same choice. It tends to produce repetitive text.

What does lowering temperature below 1 do to the distribution?

It makes high-probability tokens more likely and low-probability tokens less likely, so the draw almost always takes the obvious token. On the worked example, server rises from 0.445 to about 0.643.

How is top-p different from top-k?

Top-k keeps a fixed number of tokens. Top-p keeps the smallest set of most-probable tokens whose probabilities add up to at least p, so the pool shrinks when the model is confident and grows when it is unsure.

Does fixing a seed guarantee identical outputs?

No. One provider describes its seed as a best effort, says determinism is not guaranteed, and asks you to watch a backend fingerprint field for changes that can alter outputs.

A JSON response from a model ends without its closing brace. What do you check first?

The stop reason. A length stop (max_tokens, MAX_TOKENS) means the output was truncated by the token ceiling, not that the model chose to stop there.

Model answer

The question: “In production, our assistant gave two different answers to the exact same ticket. Walk me through what the model computed and why that can happen.” This is the question an interviewer for a role building with models will ask, and a lead engineer will ask after an incident. A strong answer hits five beats, in this order:

Tokens — the input was converted to tokens by the model’s tokenizer.
Window — all of it, plus the output, had to fit in the context window.
Distribution — at each position the model computed a probability for every possible next token.
Sampling — a sampling rule, shaped by temperature, top-k or top-p, picked one token, and the loop repeated with that token appended.
Evidence — you confirm which of these caused the difference from the request settings, the stop reason and the backend identifier, not by guessing.

“The model never saw the ticket as words: the tokenizer turned it into a sequence of tokens, and that sequence — the instructions, the ticket and the reply being written — all had to fit in the context window. At each position the model computed a probability for every possible next token. It didn’t choose one: a decoding rule did. If that rule was sampling rather than greedy decoding, it drew in proportion to those probabilities — temperature only reshapes them first — so two runs can take different tokens at the first uncertain position. From that point each reply is computed from a different sequence, so they drift apart while both staying reasonable. To confirm that’s what happened rather than a fault, I’d read the logged settings, the stop reason on both responses, and the provider’s backend identifier — if the settings used sampling and the backend didn’t change, the difference is most likely the draw — though the provider notes outputs can differ even when every one of those fields matches. If we need repeatability, I’d test properties of the reply instead of exact strings, because even a fixed seed is documented as best effort.”

Missing a beat is a failure state. Without tokens the answer cannot explain limits or cost. Without window it cannot explain truncation or forgotten turns. Without distribution “randomness” has nothing to be random over. Without sampling the difference is blamed on the wrong component. Without evidence it is a theory with no way to check it — and that is the beat interviewers most often find missing.

Why this matters

Cost and capacity planning. The context window and usage-based prices are counted in tokens, so an estimate of either in words or characters is an estimate in the wrong unit. Request-rate and spend caps sit beside them and are planned separately.
Debugging. “The model is flaky” is not a diagnosis. Truncation, a crowded window and a sampling draw are three different causes, with three different fixes, and the response’s own fields tell you which one you have.
Testing. Engineers moving in from QA and backend work bring exact-match assertions with them. Knowing where the variety comes from tells you which assertions can hold and which cannot.
Talking to customers. When a customer reports that “it gave a different answer yesterday”, the engineer who can explain the distribution and the draw in two sentences keeps their trust.

After this chapter

The project for this chapter is Look Inside One Request (project.md). You will count tokens with a real tokenizer, write a small sampler that applies temperature, top-k and top-p to the worked example’s next-token values and draws from it, so that you can watch where the variety comes from, and finish by sizing one real request from your own work in tokens.

Chapter 2 starts from the request you size in the last task. It turns tokens into money and waiting time, and shows how to estimate both before you build.

Hands-on Project — Look Inside One Request

Objective

Count tokens with a real tokenizer, build a small sampler that turns a model’s next-token values into probabilities and draws from them, and size one real request from your own work, so that you can see where the variety between two identical requests comes from and hold a token budget the next chapter turns into cost and waiting time.

What this project needs: Python 3 and permission to install one package with pip, which needs an internet connection. No task needs an account with a model provider, an API key or any payment. All three tasks are required.

Your evidence file. Keep one plain-text file, chapter1-evidence.txt. Each task tells you which lines to add. Every line has the form key: value, one per line, with the key in lower case exactly as printed. Write what your code printed, not what you expected it to print.

Task 1 — Count tokens with a real tokenizer

Install the tokenizer package: pip install tiktoken.
Run this script:
```
import tiktoken

enc = tiktoken.get_encoding("cl100k_base")
ids = enc.encode("tiktoken is great!")
print(len(ids))
print("|".join(enc.decode([i]) for i in ids))
```
The package, the cl100k_base encoding and this example string all come from the OpenAI Cookbook, How to count tokens with tiktoken, which installs the package with pip install tiktoken.
Record the two printed lines. Spaces are part of a token, so keep any space that appears before a piece exactly as printed.
Now pick one sentence of your own from your real work — a line from a log, a commit message, a ticket title. Count its characters with len(text) and its tokens with len(enc.encode(text)), and divide characters by tokens.

Stop there: one sentence, not a dataset. If you want to look further, try the same sentence with a long identifier or a non-English word added and see how the count moves, but record only the first sentence.

Add to your evidence file:

t1_example_tokens: <the first printed number>
t1_example_pieces: <the second printed line, exactly>
t1_own_chars: <characters in your sentence>
t1_own_tokens: <tokens in your sentence>
t1_own_ratio: <characters ÷ tokens, one decimal place>

Output: a script that prints token counts, and five evidence lines. You can check the first two yourself: the notes give the count and the pieces for this exact string.

Task 2 — Build a sampler

You are given the model’s raw scores (called logits) for five candidate next tokens, at the position after The deploy failed because the:

Token	Logit
`server`	2.0
`build`	1.5
`network`	1.0
`config`	0.5
`cat`	-1.0

A model turns logits into probabilities with the softmax function: divide each logit by the temperature, take exp() of each result, then divide each by the sum of all of them. The results add up to 1. With temperature 1 you should get the probabilities used in the notes.

Write one Python file with no packages beyond the standard library (math and random) that does the following. Each step can be checked against the notes before you move on.

softmax(logits, temperature) — returns a probability per token. Print the probability of server at temperature 1.0, 0.5 and 2.0, rounded to 3 decimal places. Check the 1.0 and 0.5 values against the notes before going further.
top_k(probs, k) — keeps the k most probable tokens and rescales them to add up to 1. Print the kept token names for k = 2.
top_p(probs, p) — sorts tokens from most to least probable, keeps adding them until the running total is at least p, and rescales what it kept. Print the kept token names for p = 0.8 at temperature 1. Check it against the worked example in the notes.
greedy(probs) — returns the most probable token. Call it 1,000 times at temperature 1 and print how many distinct tokens it returned.
sample(probs, u) — draws one token in proportion to its probability, given a number u between 0 and 1. Sort the tokens from most to least probable, add their probabilities one at a time, and return the first token whose running total is greater than u. Print the tokens it returns at temperature 1 for u = 0.10, 0.50, 0.80, 0.95 and 0.99, in that order.
Draw many times. Create rng = random.Random(2026) and call sample(probs, rng.random()) 10,000 times at temperature 1. Print the share of draws that were server, to 3 decimal places. Fixing the seed makes your run repeatable, so it can be marked exactly.

One job per function, five functions, nothing more. You do not need a real model, a class hierarchy or a command-line interface.

Add to your evidence file, writing token names without spaces, most probable first, comma-separated:

t2_p_server_t1: <probability, 3 decimal places>
t2_p_server_t05: <probability, 3 decimal places>
t2_p_server_t2: <probability, 3 decimal places>
t2_topk2: <token names>
t2_topp08_t1: <token names>
t2_greedy_distinct: <number>
t2_sample_picks: <the five tokens from step 5, in the order of the u values>
t2_seeded_server_share: <share from step 6, 3 decimal places>

Before you submit, change the seed in step 6 to any other number and run it again. The share moves a little and stays near the server probability from step 1; that movement between two seeds is the same thing that made the support bot in the notes give two different replies. If your share is near 1.0, your sampler is greedy; if it is near 0.2, it is ignoring the probabilities. Record only the share for seed 2026.

Output: one Python file with five functions, and eight evidence lines.

Task 3 — Size one real request

Pick one request you would really send to a model in your work: a short instruction (a few lines at most) plus one piece of input it works on — a log excerpt, a ticket or a diff. One request, one input, about one screen of text at most.

Save the instruction in instruction.txt and the input in input.txt.
Count the tokens in each file with cl100k_base, as in Task 1.
Choose an output ceiling: the most tokens you would allow the reply to use, as a whole number. Think about how long a useful reply to this request would be; the number is yours to choose.
Add the two input counts and the ceiling to get the request’s total.
For this exercise, assume a context window of 8,192 tokens. This is a round teaching figure, not the window of any particular model. Divide your total by 8,192 to get the share of the window the request would use, to 3 decimal places.