Tutorial: Workflow vs Agent Bake-off for Lead Qualification

Build the same lead qualifier as a workflow and as a tool-using agent, run both on labelled leads, and compare accuracy, consistency, calls, tokens, and latency.

Intermediate20 min readUpdated October 19, 2026

On this page· What you will build

Why this matters

Workflows vs agents argues that you should climb to an agent only when the task needs it. Arguments are easy to agree with and hard to apply. This tutorial turns the question into a measurement: you build one lead qualifier twice, once as a workflow where code gathers the facts, and once as an agent that decides which tools to call, then run both over the same labelled leads. The table at the end, with accuracy, consistency, model calls, tokens, and latency, is the kind of evidence that should settle the decision on a real project.

What you will build#

approach    accuracy  consistent  calls/lead  tokens/lead  median ms
workflow        100%         8/8         1.0          255        184
agent            94%         7/8         3.8        2,717        416
    l8: expected existing_customer, got hot (Rule 2: Globex has 2000 employees ...)

A labelled dataset of eight inbound leads, each with the verdict a sales lead would give it.
One policy and output schema, shared by both approaches so the comparison is fair.
A workflow: code looks up the company and the CRM, and one structured call makes the judgment.
An agent: the model calls lookup_company and check_customer as it sees fit and submits its verdict through a record_qualification tool.
A bake-off script that runs both twice and prints the table, plus tests that run the real SDK with no network.

Hover or tap a component to see what it does.

Workflow vs agent bake-off: the same labelled leads, policy, and schema go through both architectures, and a report compares quality and cost.

Show text description

Holding everything but the architecture constant is what makes the comparison mean something.

Labelled leads: Each lead carries the verdict a person would give it, so every answer can be scored.
Workflow: Code looks up the company and the CRM for every lead, then makes one structured call.
Agent: The model calls lookup_company and check_customer as it chooses, then record_qualification.
Claude: Both approaches share the rules and the output shape, so only the architecture differs.
Bake-off table: Accuracy, consistency across rounds, model calls, tokens, and median latency per approach.

You need Node.js 22.18 or later and an Anthropic API key. A full run makes about 80 small requests.

1. Set up the project#

mkdir lead-bakeoff && cd lead-bakeoff
npm init -y
npm pkg set type=module
npm install @anthropic-ai/sdk zod
npm install -D typescript @types/node
mkdir src test data
npm pkg set scripts.compare="node --env-file=.env src/compare.ts" scripts.test="node --test" scripts.typecheck="tsc"

Copy the tsconfig.json from Foundations Tutorial 1 and add your key to .env.

2. The labelled leads#

An evaluation is only as good as its labels. Each lead below carries the verdict a person applying the policy would give.

data/leads.json

[
  { "id": "l1", "email": "maria@initech.com", "message": "We're evaluating tools for our 400-person support org. Can we get a demo next week?", "expected": "hot" },
  { "id": "l2", "email": "bob@globex.com", "message": "Question about adding five seats to our plan.", "expected": "existing_customer" },
  { "id": "l3", "email": "sam@gmail.com", "message": "Just curious how this works, I'm a student writing a paper.", "expected": "cold" },
  { "id": "l4", "email": "lee@tinyco.io", "message": "What's the pricing for a 5-person team?", "expected": "warm" },
  { "id": "l5", "email": "priya@umbrella.com", "message": "We need to replace our current vendor before Q1. Please send pricing for 300 seats.", "expected": "hot" },
  { "id": "l6", "email": "alex@hooli.com", "message": "Where can I find your API documentation?", "expected": "warm" },
  { "id": "l7", "email": "kim@yahoo.com", "message": "Do you have any job openings?", "expected": "cold" },
  { "id": "l8", "email": "dana@globex.com", "message": "Our team wants a demo of the new analytics add-on.", "expected": "existing_customer" }
]

Eight leads is enough to learn the method, not to make a decision. On a real project, collect fifty or more from your inbox, including the awkward ones.

3. The data sources#

src/crm.ts

// Stand-ins for a company-data API and your CRM.
export interface Company {
  domain: string;
  name: string;
  employees: number;
  industry: string;
}
 
const COMPANIES: Company[] = [
  { domain: "initech.com", name: "Initech", employees: 450, industry: "software" },
  { domain: "globex.com", name: "Globex", employees: 2_000, industry: "manufacturing" },
  { domain: "tinyco.io", name: "TinyCo", employees: 6, industry: "design agency" },
  { domain: "umbrella.com", name: "Umbrella Corp", employees: 1_200, industry: "healthcare" },
  { domain: "hooli.com", name: "Hooli", employees: 300, industry: "internet" },
];
const CUSTOMER_DOMAINS = new Set(["globex.com"]);
 
export const domainOf = (email: string) => email.split("@")[1]?.toLowerCase() ?? "";
 
export async function lookupCompany(domain: string): Promise<Company | null> {
  return COMPANIES.find((c) => c.domain === domain) ?? null; // free email providers are not companies
}
 
export async function isCustomer(domain: string): Promise<boolean> {
  return CUSTOMER_DOMAINS.has(domain);
}

4. One policy, one output shape#

Both approaches get the same rules and must produce the same structure. If they differ in anything but the architecture, the comparison measures the wrong thing.

src/policy.ts

import { z } from "zod";
 
// One policy and one output shape for both approaches, so the comparison is fair.
export const POLICY = `You qualify inbound sales leads. Apply these rules in order:
1. existing_customer: the sender's company is already a customer (route to their account manager).
2. hot: a company of 200 or more employees AND the message shows buying intent (pricing, a demo, a timeline, replacing a vendor).
3. warm: a known company that does not meet the hot rule.
4. cold: no company information (for example a free email address), or no commercial intent (students, job seekers).`;
 
export const Qualification = z.object({
  verdict: z.enum(["existing_customer", "hot", "warm", "cold"]),
  reason: z.string().describe("One sentence naming the rule that applied and the facts behind it"),
});
export type Qualification = z.infer<typeof Qualification>;
 
export interface Lead {
  id: string;
  email: string;
  message: string;
}
 
export interface RunReport {
  verdict: Qualification["verdict"] | null; // null when the run produced no valid answer
  reason: string;
  modelCalls: number;
  tokens: number;
  ms: number;
}
 
export const leadBlock = (lead: Lead) => `<lead>\nFrom: ${lead.email}\n${lead.message}\n</lead>`;

The reason field is not decoration: when a verdict is wrong, it tells you why the system thought it was right, which is the first thing you need when debugging.

5. The workflow#

Code decides which facts matter and fetches all of them, in parallel. The model sees everything at once and makes one judgment.

src/workflow.ts

import Anthropic from "@anthropic-ai/sdk";
import { zodOutputFormat } from "@anthropic-ai/sdk/helpers/zod";
import { domainOf, isCustomer, lookupCompany } from "./crm.ts";
import { type Lead, leadBlock, POLICY, Qualification, type RunReport } from "./policy.ts";
 
/** Code gathers every fact first; the model makes one structured judgment. */
export async function qualifyWorkflow(lead: Lead, client = new Anthropic()): Promise<RunReport> {
  const started = performance.now();
  const domain = domainOf(lead.email);
  const [company, customer] = await Promise.all([lookupCompany(domain), isCustomer(domain)]);
 
  const message = await client.messages.parse({
    model: "claude-opus-5-5",
    max_tokens: 2048,
    output_config: { effort: "low", format: zodOutputFormat(Qualification) },
    system: POLICY,
    messages: [{ role: "user", content: `${leadBlock(lead)}\n<crm>\n${JSON.stringify({ domain, company, existingCustomer: customer })}\n</crm>` }],
  });
 
  return {
    verdict: message.parsed_output?.verdict ?? null,
    reason: message.parsed_output?.reason ?? `no structured output (stop_reason: ${message.stop_reason})`,
    modelCalls: 1,
    tokens: message.usage.input_tokens + message.usage.output_tokens,
    ms: performance.now() - started,
  };
}

The client parameter defaults to a real client but can be replaced, which is what makes the tests in step 8 possible.

6. The agent#

The agent gets the same policy and the same data, as tools. It chooses what to look up. Its answer arrives through a submit tool, so the verdict is validated by the same Zod schema.

src/agent.ts

import Anthropic from "@anthropic-ai/sdk";
import { betaZodTool } from "@anthropic-ai/sdk/helpers/beta/zod";
import { z } from "zod";
import { isCustomer, lookupCompany } from "./crm.ts";
import { type Lead, leadBlock, POLICY, Qualification, type RunReport } from "./policy.ts";
 
/** The model decides which facts it needs and calls tools to get them, then records its verdict. */
export async function qualifyAgent(lead: Lead, client = new Anthropic()): Promise<RunReport> {
  const started = performance.now();
  let recorded: Qualification | null = null;
 
  const tools = [
    betaZodTool({
      name: "lookup_company",
      description: "Look up a company by email domain. Returns name, employee count, and industry, or null if unknown.",
      inputSchema: z.object({ domain: z.string().describe("Domain only, such as example.com") }),
      run: async ({ domain }) => JSON.stringify(await lookupCompany(domain.toLowerCase())),
    }),
    betaZodTool({
      name: "check_customer",
      description: "Check whether a domain belongs to an existing customer.",
      inputSchema: z.object({ domain: z.string() }),
      run: async ({ domain }) => JSON.stringify({ existingCustomer: await isCustomer(domain.toLowerCase()) }),
    }),
    // The agent's answer arrives as a tool call, so it is structured and validated like any other input.
    betaZodTool({
      name: "record_qualification",
      description: "Record your final verdict. Call it exactly once, after gathering the facts you need.",
      inputSchema: Qualification,
      run: async (input) => {
        recorded = input;
        return "Recorded.";
      },
    }),
  ];
 
  const runner = client.beta.messages.toolRunner({
    model: "claude-opus-5-5",
    max_tokens: 4096,
    max_iterations: 6,
    output_config: { effort: "low" },
    system: `${POLICY}\nUse the tools to gather the facts you need, then call record_qualification exactly once.`,
    tools,
    messages: [{ role: "user", content: leadBlock(lead) }],
  });
 
  let modelCalls = 0;
  let tokens = 0;
  for await (const message of runner) {
    modelCalls++;
    tokens += message.usage.input_tokens + message.usage.output_tokens;
  }
 
  const result = recorded as Qualification | null;
  return {
    verdict: result?.verdict ?? null,
    reason: result?.reason ?? "the agent finished without recording a verdict",
    modelCalls,
    tokens,
    ms: performance.now() - started,
  };
}

Three details carry the design:

max_iterations: 6 bounds the loop. A lead needs at most four calls; anything more is a confused run, and it should stop.
The submit tool replaces free text. If the agent answers in prose instead, recorded stays null and the run counts as a failure, rather than you guessing a verdict from a sentence.
Every response's usage is summed. Each loop iteration resends the whole conversation, so the agent's token cost grows with every step.

7. The bake-off#

src/compare.ts

import { readFile } from "node:fs/promises";
import { qualifyAgent } from "./agent.ts";
import type { Lead, RunReport } from "./policy.ts";
import { qualifyWorkflow } from "./workflow.ts";
 
type Labeled = Lead & { expected: RunReport["verdict"] };
const leads = JSON.parse(await readFile("data/leads.json", "utf8")) as Labeled[];
const ROUNDS = 2; // run everything twice: consistency is part of quality
 
const approaches = { workflow: qualifyWorkflow, agent: qualifyAgent };
const median = (xs: number[]) => [...xs].sort((a, b) => a - b)[Math.floor(xs.length / 2)] ?? 0;
 
console.log(`${"approach".padEnd(10)}${"accuracy".padStart(10)}${"consistent".padStart(12)}${"calls/lead".padStart(12)}${"tokens/lead".padStart(13)}${"median ms".padStart(11)}`);
for (const [name, qualify] of Object.entries(approaches)) {
  const runs: RunReport[][] = [];
  for (let round = 0; round < ROUNDS; round++) runs.push(await Promise.all(leads.map((lead) => qualify(lead))));
 
  const all = runs.flat();
  const correct = runs.flatMap((r) => r.map((report, i) => report.verdict === leads[i]!.expected)).filter(Boolean).length;
  const consistent = leads.filter((_, i) => runs.every((r) => r[i]!.verdict === runs[0]![i]!.verdict)).length;
  console.log(
    `${name.padEnd(10)}${`${Math.round((correct / all.length) * 100)}%`.padStart(10)}${`${consistent}/${leads.length}`.padStart(12)}` +
      `${(all.reduce((s, r) => s + r.modelCalls, 0) / all.length).toFixed(1).padStart(12)}` +
      `${Math.round(all.reduce((s, r) => s + r.tokens, 0) / all.length).toLocaleString("en-US").padStart(13)}` +
      `${Math.round(median(all.map((r) => r.ms))).toLocaleString("en-US").padStart(11)}`,
  );
  // Show the leads this approach got wrong, with its own explanation.
  runs[0]!.forEach((report, i) => {
    if (report.verdict !== leads[i]!.expected) console.log(`    ${leads[i]!.id}: expected ${leads[i]!.expected}, got ${report.verdict} (${report.reason})`);
  });
}

Run it:

npm run compare

approach    accuracy  consistent  calls/lead  tokens/lead  median ms
workflow        100%         8/8         1.0          255        184
agent            94%         7/8         3.8        2,717        416
    l8: expected existing_customer, got hot (Rule 2: Globex has 2000 employees and the message shows buying intent.)

8. Read the table#

Each column answers a different question:

Accuracy is the share of runs that matched the label. The agent's miss is instructive: Dana's message reads like a hot lead, so the agent decided it had seen enough and never checked whether Globex was already a customer. The workflow cannot make that mistake, because code always fetches both facts.
Consistent counts leads that got the same verdict in both rounds. An agent chooses its path every time, so the same input can take a different route and get a different answer. Inconsistent verdicts are often worse than consistently wrong ones, because they are harder to notice and to fix.
Calls and tokens are the cost. The agent needs several round trips, and each one resends the conversation so far, so it spends roughly ten times the tokens here.
Latency follows the number of round trips.

For this task the facts needed are the same for every lead, so code can fetch them, and the workflow wins on every column. That is the signal from the article: when you can write down the steps, write them down. The agent earns its cost when the path really does depend on what it finds, such as an open-ended research question, where you cannot list the lookups in advance.

9. Test without the network#

The tests use a real Anthropic client whose fetch replays scripted responses. The SDK's parsing, tool runner, and Zod validation all run for real; nothing leaves your machine.

test/scripted.ts

import Anthropic from "@anthropic-ai/sdk";
 
type Block = Record<string, unknown>;
 
/**
 * A real Anthropic client whose HTTP layer replays scripted responses. Everything above HTTP,
 * including parse() and the tool runner, runs for real, with no network and no cost.
 */
export function scriptedClient(responses: { content: Block[]; stop_reason?: string }[]) {
  const requests: Anthropic.MessageCreateParams[] = [];
  const client = new Anthropic({
    apiKey: "test-key",
    maxRetries: 0,
    fetch: async (_url, init) => {
      requests.push(JSON.parse(String(init?.body)) as Anthropic.MessageCreateParams);
      const next = responses.shift();
      if (!next) return Response.json({ type: "error", error: { type: "api_error", message: "script exhausted" } }, { status: 500 });
      return Response.json({
        id: `msg_${requests.length}`,
        type: "message",
        role: "assistant",
        model: "claude-opus-5-5",
        stop_reason: next.stop_reason ?? (next.content.some((b) => b.type === "tool_use") ? "tool_use" : "end_turn"),
        stop_sequence: null,
        usage: { input_tokens: 100, output_tokens: 20 },
        content: next.content,
      });
    },
  });
  return { client, requests };
}
 
export const text = (t: string) => ({ type: "text", text: t });
export const toolUse = (id: string, name: string, input: unknown) => ({ type: "tool_use", id, name, input });

test/bakeoff.test.ts

import assert from "node:assert/strict";
import { test } from "node:test";
import { qualifyAgent } from "../src/agent.ts";
import { qualifyWorkflow } from "../src/workflow.ts";
import { scriptedClient, text, toolUse } from "./scripted.ts";
 
const lead = { id: "l1", email: "maria@initech.com", message: "Can we get a demo next week for our 400-person team?" };
 
test("the workflow gathers CRM facts in code and makes one model call", async () => {
  const { client, requests } = scriptedClient([{ content: [text('{"verdict":"hot","reason":"Rule 2: 450 employees and a demo request"}')] }]);
  const report = await qualifyWorkflow(lead, client);
  assert.equal(report.verdict, "hot");
  assert.equal(report.modelCalls, 1);
  assert.equal(requests.length, 1);
  assert.match(JSON.stringify(requests[0]!.messages), /\\"employees\\":450/); // the facts were in the prompt
});
 
test("the agent calls tools, then records its verdict", async () => {
  const { client, requests } = scriptedClient([
    { content: [toolUse("t1", "lookup_company", { domain: "initech.com" })] },
    { content: [toolUse("t2", "record_qualification", { verdict: "hot", reason: "Rule 2" })] },
    { content: [text("Done.")] },
  ]);
  const report = await qualifyAgent(lead, client);
  assert.equal(report.verdict, "hot");
  assert.equal(report.modelCalls, 3);
  // The tool result sent back on the second request contains the company data.
  assert.match(JSON.stringify(requests[1]!.messages.at(-1)), /Initech/);
});
 
test("an agent that never records a verdict is a failed run, not a guess", async () => {
  const { client } = scriptedClient([{ content: [text("I think this lead is hot.")] }]);
  const report = await qualifyAgent(lead, client);
  assert.equal(report.verdict, null);
});
 
test("an invalid verdict is rejected by the tool's schema and sent back as an error", async () => {
  const { client, requests } = scriptedClient([
    { content: [toolUse("t1", "record_qualification", { verdict: "lukewarm", reason: "?" })] },
    { content: [text("Sorry.")] },
  ]);
  const report = await qualifyAgent(lead, client);
  assert.equal(report.verdict, null);
  assert.match(JSON.stringify(requests[1]!.messages.at(-1)), /is_error/);
});

npm test   # 4 passing, no API calls

The scripted client is reusable: the remaining tutorials in this section test their agents the same way.

Common mistakes#

Comparing approaches with different prompts or schemas. Then you are measuring the prompts. Share the policy and the output shape.
Running each lead once. A single run hides inconsistency, which is one of the main costs of autonomy.
Counting only the last response's tokens. An agent's cost is the sum over every step of the loop.
Parsing the agent's verdict from prose. Use a submit tool or structured output, and treat a missing answer as a failure.
Deciding from eight examples. Use them to build the harness, then grow the dataset from real traffic before you commit.

Try it

Extend the bake-off

Add a third approach: the workflow with claude-haiku-5-5 instead of Opus, and compare cost against accuracy.
Add ten tricky leads, such as a customer writing from a personal address or a subsidiary of a customer, and see which approach degrades first.
Raise ROUNDS to five and report, for each lead, how many distinct verdicts each approach gave.
Give the agent a search_news tool that only matters for some leads, and check whether autonomy starts to pay off.

Check your understanding

Key takeaways

Decide between a workflow and an agent by measuring both on the same labelled inputs, with the same policy and output schema.
Compare accuracy, consistency, model calls, tokens, and latency; autonomy costs on every one of them.
When the facts a task needs are known in advance, fetch them in code: a workflow cannot skip a step.
Test agents offline with a real SDK client whose fetch replays scripted responses.

Next steps#

Agent design patterns

Next article: chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer.

Workflows vs agents

Revisit the article this tutorial puts to the test.

Finished this article? Track it to see your progress on the roadmap.