Choosing LLMs and Providers: Capability vs Cost vs Latency

The three axes, plus the fine print#

Then check the fine print that can rule a model out regardless of quality: context window size, input types (images, PDFs), rate limits for your account tier, data handling (retention, training use, regional processing), and availability on the cloud platform your company already uses.

The current Claude lineup#

At the time of writing (October 2026), these are the models you will choose between. Prices are list prices per million tokens; check the official pricing and models pages before you commit, because lineups and prices change.

This wiki's examples default to claude-opus-5-5. In production you will often use more than one: a capable model where errors are expensive and a smaller one for high-volume, well-defined steps.

Effort: a second dial on the same model#

Current models expose an effort setting (low through max) that controls how much the model reasons and how thorough it is. Effort lets you trade cost for quality within one model, and it is often the first lever to try: a capable model at low effort can beat a smaller model at high effort, and costs less than the capable model at high effort.

The shape in the chart is typical: quality rises with model size and effort, while cost rises faster at the top end. The right point depends entirely on your task, which is why you measure instead of reading leaderboards.

Measure on your own tasks#

This harness runs the same small task set across several configurations and reports accuracy, latency, and cost. Replace the tasks with twenty real examples from your product and you have the start of an eval suite.

import Anthropic from "@anthropic-ai/sdk";
 
const client = new Anthropic();
 
// USD per million tokens at the time of writing. Check the pricing page before relying on these.
const PRICES = {
  "claude-opus-5-5": { input: 4, output: 20 },
  "claude-sonnet-5-5": { input: 2, output: 10 },
  "claude-haiku-4-5": { input: 1, output: 5 },
} as const;
 
type Config = { model: keyof typeof PRICES; effort?: "low" | "medium" | "high" };
 
const configs: Config[] = [
  { model: "claude-haiku-4-5" }, // Haiku 4.5 does not take an effort setting
  { model: "claude-sonnet-5-5", effort: "low" },
  { model: "claude-sonnet-5-5", effort: "high" },
  { model: "claude-opus-5-5", effort: "medium" },
];
 
const tasks = [
  { prompt: "Sentiment of: 'The update broke my login again.' Answer with one word: positive, negative, or neutral.", expect: /negative/i },
  { prompt: "A train leaves at 14:35 and arrives at 17:10. How many minutes is the trip? Answer with the number only.", expect: /\b155\b/ },
  { prompt: "Extract the invoice number from: 'Ref INV-2026-0412, due Oct 31.' Answer with the number only.", expect: /INV-2026-0412/ },
  { prompt: "Is 391 a prime number? Answer yes or no, then give the factorization if no.", expect: /\bno\b[\s\S]*17[\s\S]*23/i },
];
 
for (const config of configs) {
  let correct = 0;
  let cost = 0;
  const started = performance.now();
 
  for (const task of tasks) {
    const response = await client.messages.create({
      model: config.model,
      max_tokens: 2048,
      ...(config.effort ? { output_config: { effort: config.effort } } : {}),
      messages: [{ role: "user", content: task.prompt }],
    });
    const text = response.content.flatMap((b) => (b.type === "text" ? [b.text] : [])).join("");
    if (task.expect.test(text)) correct++;
    const price = PRICES[config.model];
    cost += (response.usage.input_tokens * price.input + response.usage.output_tokens * price.output) / 1_000_000;
  }
 
  const seconds = ((performance.now() - started) / 1000 / tasks.length).toFixed(1);
  console.log(
    `${config.model.padEnd(18)} ${(config.effort ?? "-").padEnd(6)} ` +
      `accuracy ${correct}/${tasks.length}  avg ${seconds}s  cost $${cost.toFixed(4)}`,
  );
}

Four tasks are enough to learn the workflow, not enough to decide. Real decisions need a few dozen representative cases per feature, ideally including the hard ones that made you consider a bigger model in the first place.

Route instead of compromising#

When one feature has both easy and hard requests, you do not have to pick one model for all of them. A router sends each request to the cheapest model that handles it well.

Start with rules (request type, input length, customer tier); they are free, fast, and debuggable. Add a model-based classifier only when rules leave too much on the table, and keep evals on sampled traffic so a degraded route is noticed. The cost engineering article builds this end to end.

Discover models programmatically#

The Models API lists the models available to your key with their limits, which is useful for configuration checks and for catching a typo in a model ID at startup rather than on the first user request.

import Anthropic from "@anthropic-ai/sdk";
 
const client = new Anthropic();
 
for await (const model of client.models.list()) {
  console.log(`${model.id.padEnd(22)} ${model.display_name}`);
}

Providers and platforms#

The same Claude models are available directly from Anthropic's API and through cloud platforms such as Amazon Bedrock, Google Cloud Vertex AI, and Microsoft Foundry. Choose the platform by organizational constraints more than by model:

• Direct API: new features and models first, one bill, simplest setup.
• Cloud platforms: existing cloud contracts and credits, data residency and compliance requirements, private networking, and unified access control. Feature availability can lag the direct API, so check the features you rely on.

Keep your code portable without over-abstracting: put model IDs and settings in configuration, keep prompts in files, and wrap the client creation in one module. A heavy "works with every provider" layer usually hides the very features that make a model worth choosing.

Common mistakes#

• Choosing by benchmark headline. Public benchmarks measure someone else's tasks. Your eval set is the only benchmark that matters.
• Using the most capable model everywhere. It is the right default while you build; it is rarely the right final answer for high-volume steps.
• Using the smallest model everywhere. Saving cents per request while failing users is the most expensive outcome.
• Never revisiting the choice. New models ship often. Rerun your evals on each release; a model swap can cut cost or raise quality with a one-line change.
• Hard-coding model IDs across the codebase. Keep them in configuration so a switch is reviewed in one place.

Next steps#