Tutorial: Measure Prompt Caching and Batch Savings in a Cache Lab

Ask questions about one long document, watch cache writes turn into cheap reads, see a timestamp silently break the cache, and run the work as a half-price batch.

Intermediate18 min readUpdated October 19, 2026

On this page· What you will build

Why this matters

Prompt caching and batches explains the two biggest cost levers that require no change in quality: reuse a long prefix at a fraction of the price, and send work that can wait at half price. Both are easy to get almost right, and "almost" costs more than doing nothing: a cache that is written on every request and never read is more expensive than no cache at all. This tutorial builds a small lab that makes each case visible in a table, so you can recognize the healthy pattern, and the broken one, in your own usage numbers.

What you will build#

A script that asks five questions about one long document, three ways:

Stable prefix: the document is a cached system prompt. The first request writes it; the next four read it.
Timestamp at the top: the same, with the current time prepended. The prefix changes every request, so the cache never hits.
Batch (with --batch): the same five requests through the Message Batches API, at half price.

For each request it prints uncached, written, and read tokens, the actual cost, and what the request would have cost without caching.

Hover or tap a component to see what it does.

Prompt cache lab: questions about one long document share a cached prefix, so the first request writes the cache and the rest read it; a batch runs the same work at half price.

Show text description

Caching pays when a long prefix stays byte-identical across requests; batching pays when nobody is waiting.

Questions: Five questions, each sent after the same long, stable system prompt.
Claude: Reports uncached, written, and read tokens on every response.
Batches API: The same requests, processed asynchronously; results are matched by custom_id.
Prompt cache: Written once (1.25x input price), then read at 0.05x on this model until it expires.

You need Node.js 22.18 or later, an Anthropic API key, and a long Markdown document of at least a few thousand tokens (a long README, a design doc, or a page from this wiki's source). A full run costs a few cents.

1. Set up the project#

mkdir cache-lab && cd cache-lab
npm init -y
npm pkg set type=module
npm install @anthropic-ai/sdk
npm install -D typescript @types/node
mkdir src test
npm pkg set scripts.lab="node --env-file=.env src/lab.ts" scripts.test="node --test" scripts.typecheck="tsc"

Copy the tsconfig.json from Foundations Tutorial 1 and add your key to .env.

2. Cost, with and without the cache#

src/cost.ts

import type Anthropic from "@anthropic-ai/sdk";
 
// USD per million tokens for claude-opus-5-5 at the time of writing. Check the pricing page.
export const PRICE = { input: 4, cacheWrite: 5, cacheRead: 0.2, output: 20 }; // 5-minute cache writes: 1.25x input
export const BATCH_DISCOUNT = 0.5;
 
type BilledUsage = Pick<Anthropic.Usage, "input_tokens" | "output_tokens" | "cache_creation_input_tokens" | "cache_read_input_tokens">;
 
/** What a response actually cost. */
export function costUSD(u: BilledUsage, { batch = false } = {}): number {
  const usd =
    (u.input_tokens * PRICE.input +
      (u.cache_creation_input_tokens ?? 0) * PRICE.cacheWrite +
      (u.cache_read_input_tokens ?? 0) * PRICE.cacheRead +
      u.output_tokens * PRICE.output) /
    1_000_000;
  return batch ? usd * BATCH_DISCOUNT : usd;
}
 
/** What the same response would have cost with no caching at all: every prompt token at the input price. */
export function uncachedUSD(u: BilledUsage): number {
  const prompt = u.input_tokens + (u.cache_creation_input_tokens ?? 0) + (u.cache_read_input_tokens ?? 0);
  return (prompt * PRICE.input + u.output_tokens * PRICE.output) / 1_000_000;
}

The three input counts in usage never overlap: input_tokens is only the part processed fresh, cache_creation_input_tokens was written to the cache (at 1.25× the input price), and cache_read_input_tokens was served from it (at $0.20 per million on this model, 5% of the input price). Their sum is the full prompt, which is what uncachedUSD prices as if caching did not exist.

test/cost.test.ts

import assert from "node:assert/strict";
import { test } from "node:test";
import { costUSD, uncachedUSD } from "../src/cost.ts";
 
const usage = (input: number, write: number, read: number, output: number) => ({
  input_tokens: input,
  cache_creation_input_tokens: write,
  cache_read_input_tokens: read,
  output_tokens: output,
});
const close = (a: number, b: number) => assert.ok(Math.abs(a - b) < 1e-12, `${a} != ${b}`);
 
test("a cache write costs 1.25x input, a cache read 0.05x on this model", () => {
  close(costUSD(usage(0, 1_000_000, 0, 0)), 5);
  close(costUSD(usage(0, 0, 1_000_000, 0)), 0.2);
});
 
test("without caching, every prompt token is billed at the input price", () => {
  close(uncachedUSD(usage(100, 9_000, 0, 50)), (9_100 * 4 + 50 * 20) / 1e6);
});
 
test("two requests sharing a 10K-token prefix: caching wins from the second request", () => {
  const first = usage(20, 10_000, 0, 100);
  const second = usage(20, 0, 10_000, 100);
  const cached = costUSD(first) + costUSD(second);
  const uncached = uncachedUSD(first) + uncachedUSD(second);
  assert.ok(cached < uncached);
  close(cached, (40 * 4 + 10_000 * 5 + 10_000 * 0.2 + 200 * 20) / 1e6);
});
 
test("batch results cost half", () => {
  close(costUSD(usage(1_000, 0, 0, 1_000), { batch: true }), costUSD(usage(1_000, 0, 0, 1_000)) / 2);
});

The third test is the break-even rule in numbers: with a 5-minute cache, a prefix pays for itself from its second use.

3. The lab#

src/lab.ts

import { readFile } from "node:fs/promises";
import { setTimeout as sleep } from "node:timers/promises";
import Anthropic from "@anthropic-ai/sdk";
import { costUSD, uncachedUSD } from "./cost.ts";
 
const client = new Anthropic();
const MODEL = "claude-opus-5-5";
const MIN_CACHEABLE = 512; // tokens; shorter prefixes silently are not cached (check the docs for your model)
 
const file = process.argv[2];
if (!file) throw new Error("usage: node src/lab.ts <long-document.md> [--batch]");
const doc = await readFile(file, "utf8");
 
const QUESTIONS = [
  "Summarize this document in two sentences.",
  "List the three most important terms it defines.",
  "What is the single most common mistake it warns about?",
  "Who is this document written for?",
  "Suggest one question a reader might still have after reading it.",
];
 
const instructions = "Answer questions about the document in <document>. Be brief: at most 60 words.";
const text = (m: Anthropic.Message) => m.content.flatMap((b) => (b.type === "text" ? [b.text] : [])).join("").trim();
 
// The document is the stable prefix, marked as cacheable. Questions vary, so they come after it.
const system = (prefix = ""): Anthropic.TextBlockParam[] => [
  { type: "text", text: `${prefix}${instructions}\n\n<document>\n${doc}\n</document>`, cache_control: { type: "ephemeral" } },
];
 
const { input_tokens } = await client.messages.countTokens({ model: MODEL, system: system(), messages: [{ role: "user", content: "x" }] });
console.log(`${file}: about ${input_tokens.toLocaleString("en-US")} prompt tokens`);
if (input_tokens < MIN_CACHEABLE) console.log(`warning: under ${MIN_CACHEABLE} tokens, nothing will be cached; use a longer document`);
 
async function run(label: string, makeSystem: () => Anthropic.TextBlockParam[]) {
  console.log(`\n${label}`);
  console.log(`  ${"#".padEnd(3)}${"uncached".padStart(10)}${"written".padStart(10)}${"read".padStart(10)}${"cost".padStart(11)}${"without cache".padStart(15)}`);
  let actual = 0;
  let baseline = 0;
  for (const [i, question] of QUESTIONS.entries()) {
    const message = await client.messages.create({
      model: MODEL,
      max_tokens: 1024,
      output_config: { effort: "low" },
      system: makeSystem(),
      messages: [{ role: "user", content: question }],
    });
    const u = message.usage;
    actual += costUSD(u);
    baseline += uncachedUSD(u);
    console.log(
      `  ${String(i + 1).padEnd(3)}${String(u.input_tokens).padStart(10)}${String(u.cache_creation_input_tokens ?? 0).padStart(10)}` +
        `${String(u.cache_read_input_tokens ?? 0).padStart(10)}${`$${costUSD(u).toFixed(4)}`.padStart(11)}${`$${uncachedUSD(u).toFixed(4)}`.padStart(15)}`,
    );
  }
  const change = Math.round((1 - actual / baseline) * 100);
  console.log(`  total $${actual.toFixed(4)} vs $${baseline.toFixed(4)} without caching (${change >= 0 ? `${change}% saved` : `${-change}% MORE: cache writes with no reads`})`);
}
 
// 1. A stable prefix: the first request writes the cache, the rest read it.
await run("Stable prefix", () => system());
 
// 2. The classic mistake: something that changes on every request at the top of the prompt.
await run("Timestamp at the top (cache never hits)", () => system(`Current time: ${new Date().toISOString()}\n`));
 
// 3. The same questions as a batch: half price, results within 24 hours (usually much sooner).
if (process.argv.includes("--batch")) {
  const batch = await client.messages.batches.create({
    requests: QUESTIONS.map((question, i) => ({
      custom_id: `q${i + 1}`, // results can arrive in any order; match them by ID
      params: { model: MODEL, max_tokens: 1024, output_config: { effort: "low" }, system: system(), messages: [{ role: "user", content: question }] },
    })),
  });
  console.log(`\nBatch ${batch.id} submitted; polling...`);
 
  let status = batch;
  while (status.processing_status !== "ended") {
    await sleep(Number(process.env.POLL_MS ?? 30_000));
    status = await client.messages.batches.retrieve(batch.id);
    console.log(`  ${status.processing_status}: ${JSON.stringify(status.request_counts)}`);
  }
 
  let total = 0;
  for await (const item of await client.messages.batches.results(batch.id)) {
    if (item.result.type !== "succeeded") {
      console.log(`  ${item.custom_id}: ${item.result.type}, resubmit it`);
      continue;
    }
    const usd = costUSD(item.result.message.usage, { batch: true });
    total += usd;
    console.log(`  ${item.custom_id} $${usd.toFixed(4)}  ${text(item.result.message).slice(0, 70)}`);
  }
  console.log(`  batch total $${total.toFixed(4)} (50% of standard prices)`);
}

The structure of system() is the whole technique: stable content first, marked with cache_control; variable content (the question) after it. The cache matches prefixes byte for byte, so anything that changes must come after the last cached block.

Token counting runs first because caching has a minimum prefix length, 512 tokens on claude-opus-5-5. A shorter prefix is not an error: it is silently not cached, and the "written" column stays at zero.

4. Run it#

npm run lab -- ../docs/long-guide.md

against the local mock, with a document of about 4,700 tokens

Stable prefix
  #    uncached   written      read       cost  without cache
  1          13      4695         0    $0.0245        $0.0198
  2          15         0      4695    $0.0020        $0.0198
  3          17         0      4695    $0.0020        $0.0198
  4          11         0      4695    $0.0019        $0.0197
  5          20         0      4695    $0.0021        $0.0199
  total $0.0324 vs $0.0991 without caching (67% saved)
 
Timestamp at the top (cache never hits)
  #    uncached   written      read       cost  without cache
  1          14      4705         0    $0.0245        $0.0198
  2          16      4705         0    $0.0246        $0.0199
  3          18      4705         0    $0.0246        $0.0199
  4          12      4705         0    $0.0245        $0.0198
  5          20      4705         0    $0.0247        $0.0200
  total $0.1228 vs $0.0993 without caching (24% MORE: cache writes with no reads)

How to read the two tables:

Stable prefix. Request 1 costs more than it would without caching, because writing costs 1.25× the input price. Every later request reads the same prefix at 5% of the price. By the second request caching is ahead, and over five it saves about two thirds.
Timestamp at the top. Every request writes a new cache entry and none is ever read. You pay the 25% write premium five times for nothing: worse than not caching at all. In production this shows up as cache_creation_input_tokens on every request and cache_read_input_tokens always zero. Typical causes are a timestamp, a request ID, a user's name, or an unsorted object serialized into the system prompt.

Run the lab twice within five minutes. On the second run, even request 1 of the stable table reads from the cache: the entry written by the first run is still alive. That is the 5-minute TTL at work, and it is why traffic that arrives in bursts caches so well.

5. Batch it#

npm run lab -- ../docs/long-guide.md --batch

excerpt, against the local mock

Batch msgbatch_… submitted; polling...
  in_progress: {"processing":5,"succeeded":0,"errored":0,"canceled":0,"expired":0}
  ended: {"processing":0,"succeeded":5,"errored":0,"canceled":0,"expired":0}
  q1 $0.0010  …
  …
  batch total $0.0049 (50% of standard prices)

Three rules for batches, all visible in the code:

Match by custom_id, never by position: results can come back in any order.
Handle every result type. errored, canceled, and expired items need resubmitting; the loop says so for each one.
Poll from a scheduled job in production, not a process kept alive for hours. Most batches finish well within the 24-hour limit, often in minutes.

Caching and batching combine: batch requests that share a prefix can also read from the cache, but reuse inside a batch is best-effort, so do not count on it when estimating.

Common mistakes#

Variable text before the cached content. A timestamp, user name, or request ID at the top of the system prompt invalidates the cache on every request.
Caching a prefix under the minimum size. Nothing fails; nothing is cached. Count tokens and check cache_creation_input_tokens.
Caching content used once. A write costs 1.25×; without a second read inside the TTL, caching costs more than it saves.
Batching user-facing work. Batches are for jobs nobody is waiting on: nightly classification, backfills, evals.
Matching batch results by position. Use custom_id.

Try it

Extend the lab

Move the timestamp after the document (at the start of the user message instead) and confirm that the stable table's pattern returns.
Add a second cache breakpoint: a short fixed instruction block before the document, each with its own cache_control. When would two breakpoints help?
Switch to the 1-hour cache (cache_control: { type: "ephemeral", ttl: "1h" }, writes at 2× the input price). How many reads does it take to break even?
Add a --model claude-haiku-5-5 option with its prices, and compare the cost of the stable table across models.

Check your understanding

Key takeaways

Put stable content first with cache_control, and everything that varies after it.
Watch usage: healthy caching shows one write followed by reads; writes on every request mean a broken prefix.
A write costs more than an uncached request, so caching only pays when the prefix is reused within its TTL.
Batch work nobody is waiting for at half price, match results by custom_id, and resubmit failed items.

Next steps#

Workflows vs agents

LLM App Engineering done: start AI Agents and Orchestration.

Cost engineering and model routing

Combine caching, batching, and routing in production.

Finished this article? Track it to see your progress on the roadmap.