Skip to content
AI Full-Stack Wiki

Multi-Turn Chat and Conversation Memory with the Claude API

Build multi-turn chat in Node.js: keep history, replay content blocks correctly, and control cost with sliding windows, summaries, compaction, and long-term memory.

Difficulty: Intermediate16 min readUpdated
On this page· History is just an array you resend

Why this matters

The model forgets everything between requests, so "memory" is something your application builds. Done naively, every turn resends the whole conversation and both cost and latency grow without limit. Done carelessly, you trim the wrong message and the model loses the user's name, a decision from ten minutes ago, or half of a tool call, and the API rejects the request. Conversation memory is a design decision with clear trade-offs.

History is just an array you resend#

Each request resends the growing conversationThe user sends three messages. For each one, the app sends the model every earlier message plus the new one, and saves the reply; request size grows with every turn.UserYour appLLM API“Hi, I'm Ada.”messages:[user₁] (1message)“Hi Ada!”“What's myname?”messages:[user₁,assistant₁,user₂] (3)“Your name isAda.”“Summarize ourchat.”messages: [5messages] (5)summary

Step 1 of 9

“Hi, I'm Ada.”

The first message of a new conversation.

Stateless chat: the model remembers nothing between requests, so your app stores the conversation and resends the growing history on every turn.
Show text description

Three turns of a conversation. Each request to the model contains the entire conversation so far. Participants, left to right: User, Your app, LLM API.

  1. User → Your app: “Hi, I'm Ada.” · The first message of a new conversation.
  2. Your app → LLM API: messages: [user₁] (1 message) · The request contains only the first message.
  3. LLM API → Your app: “Hi Ada!” · The app saves the reply as assistant₁ in its database. The model keeps nothing.
  4. User → Your app: “What's my name?” · A follow-up that depends on the first turn.
  5. Your app → LLM API: messages: [user₁, assistant₁, user₂] (3) · The app sends the full history. Without it, the model could not know the name.
  6. LLM API → Your app: “Your name is Ada.” · Correct only because the history was included.
  7. User → Your app: “Summarize our chat.” · A third turn.
  8. Your app → LLM API: messages: [5 messages] (5) · The request grows every turn. Long chats eventually need trimming or summarizing, and prompt caching to keep cost down.
  9. LLM API → Your app: summary · Each answer costs input tokens for the whole history, not just the new message.

The messages array follows a few rules: it starts with a user turn, turns alternate between user and assistant (consecutive turns of the same role are merged), and each assistant turn is stored exactly as returned: the full response.content, including reasoning and tool blocks.

lib/anthropic.ts
import Anthropic from "@anthropic-ai/sdk";
 
export const anthropic = new Anthropic({ timeout: 60_000, maxRetries: 2 });
export const MODEL = "claude-opus-5-5";
chat-cli.ts
import { createInterface } from "node:readline/promises";
import type Anthropic from "@anthropic-ai/sdk";
import { anthropic, MODEL } from "./lib/anthropic.ts";
 
const rl = createInterface({ input: process.stdin, output: process.stdout });
const messages: Anthropic.MessageParam[] = [];
console.log('Chat with the model. Type "exit" to quit.');
 
while (true) {
  const input = (await rl.question("\nyou › ")).trim();
  if (!input || input === "exit") break;
 
  messages.push({ role: "user", content: input });
  const response = await anthropic.messages.create({
    model: MODEL,
    max_tokens: 4096,
    output_config: { effort: "low" },
    system: "You are a friendly, concise assistant.",
    messages,
  });
  messages.push({ role: "assistant", content: response.content }); // store the blocks, not just the text
 
  const text = response.content.flatMap((b) => (b.type === "text" ? [b.text] : [])).join("");
  console.log(`\nassistant › ${text}`);
  console.log(`  (${messages.length} messages so far, ${response.usage.input_tokens} input tokens this turn)`);
}
rl.close();

Watch the input token count grow with every turn. That number is your cost per turn, and it is the reason the rest of this article exists.

In a web app, the array lives in your database, not in memory: load the conversation's messages, append the new user turn, call the model, and save both turns in one transaction. The Databases article shows getHistory and saveTurn for exactly this.

Four memory strategies#

Long conversations need a policy for what the model sees. Each strategy trades cost against what the model can still remember.

  • System prompt
  • Summary of older turns
  • Retrieved memories
  • Recent turns
Conversation memory strategies compared at turn 40 of a chat: input tokens per request for full history, a sliding window, a running summary, and retrieval of relevant memories.
Show text description

Input tokens sent on turn 40 of a conversation averaging 250 tokens per turn, with an 800-token system prompt. Fewer tokens mean lower cost and latency; the trade-off is what the model can no longer see.

Input tokens per request at turn 40
StrategySystem promptSummary of older turnsRetrieved memoriesRecent turnsTotal
Full history (all 40 turns)8000010,00010,800
Sliding window (last 10 turns)800002,5003,300
Summary + recent (summary + last 6 turns)80045001,5002,750
Retrieval memory (relevant facts + last 6 turns)80006001,5002,900
StrategyKeepsLosesUse when
Full historyEverythingNothing, until the context limitShort conversations, high-stakes tasks
Sliding windowThe last N turns verbatimEverything olderCasual chat where old turns rarely matter
Summary + recentA running summary plus recent turnsExact wording of older turnsLong support or coaching sessions
Retrieval memoryRelevant stored facts plus recent turnsFacts that were never storedAssistants that span many conversations

With prompt caching (covered later), the repeated history prefix becomes cheap, so full history stays affordable much longer than the raw token counts suggest. Trim when you approach your latency budget or the context limit, not at the first sign of growth.

Sliding window, without breaking tool calls#

The naive messages.slice(-10) can start the array with an assistant turn, or keep a tool_result whose matching tool_use was cut off. Both make the request invalid. Cut only at a real user turn:

lib/memory.ts
import type Anthropic from "@anthropic-ai/sdk";
import { anthropic, MODEL } from "./anthropic.ts";
 
const textOf = (m: Anthropic.MessageParam) =>
  typeof m.content === "string" ? m.content : m.content.flatMap((b) => (b.type === "text" ? [b.text] : [])).join(" ");
 
const isToolResultTurn = (m: Anthropic.MessageParam) =>
  m.role === "user" && Array.isArray(m.content) && m.content.some((b) => b.type === "tool_result");
 
/** Keep roughly the last `keep` messages, starting at a real user turn so tool_use/tool_result pairs stay together. */
export function slidingWindow(messages: Anthropic.MessageParam[], keep: number): Anthropic.MessageParam[] {
  if (messages.length <= keep) return messages;
  let start = messages.length - keep;
  while (start < messages.length && (messages[start].role !== "user" || isToolResultTurn(messages[start]))) start++;
  return messages.slice(start);
}
 
/** Replace older turns with a short summary, keeping recent turns verbatim. */
export async function compactHistory(messages: Anthropic.MessageParam[], keepRecent = 6): Promise<Anthropic.MessageParam[]> {
  const recent = slidingWindow(messages, keepRecent);
  const older = messages.slice(0, messages.length - recent.length);
  if (older.length === 0) return messages;
 
  const summary = await anthropic.messages.create({
    model: MODEL,
    max_tokens: 1024,
    output_config: { effort: "low" },
    system:
      "Summarize this conversation for your own future reference. Keep names, decisions, user preferences, " +
      "facts the user shared, and open questions. Write under 150 words.",
    messages: [{ role: "user", content: older.map((m) => `${m.role}: ${textOf(m)}`).join("\n") }],
  });
  const text = summary.content.flatMap((b) => (b.type === "text" ? [b.text] : [])).join("");
 
  // A user turn holding the summary, then the recent turns (which also start with a user turn; the two are merged).
  return [{ role: "user", content: `Summary of our earlier conversation:\n${text}` }, ...recent];
}

Server-side compaction#

Instead of summarizing yourself, you can let the API do it. With the compaction beta enabled, the API summarizes older context automatically when a conversation approaches a threshold and returns a compaction block that replaces the older turns. The rule from earlier matters even more here: append response.content every turn, so the compaction block is sent back.

compaction.ts
import Anthropic from "@anthropic-ai/sdk";
 
const client = new Anthropic();
const messages: Anthropic.Beta.BetaMessageParam[] = [];
 
async function chat(userMessage: string): Promise<string> {
  messages.push({ role: "user", content: userMessage });
  const response = await client.beta.messages.create({
    betas: ["compact-2026-01-12"],
    model: "claude-opus-5-5",
    max_tokens: 16000,
    messages,
    context_management: { edits: [{ type: "compact_20260112" }] },
  });
  messages.push({ role: "assistant", content: response.content }); // keeps compaction blocks
  return response.content.flatMap((b) => (b.type === "text" ? [b.text] : [])).join("");
}
 
console.log(await chat("Help me plan a three-day trip to Lisbon."));
console.log(await chat("Make day two about food markets."));

Long-term memory across conversations#

Everything above is memory within one conversation. Remembering a user across conversations ("I'm vegetarian", "our stack is Postgres") is a separate feature: extract durable facts, store them per user, and retrieve the relevant ones into the context of new conversations. That is retrieval memory, built on the same embedding search as RAG and covered in the agent memory article. Always let users see and delete what is stored about them.

Common mistakes#

  • Saving only the assistant's text. Reasoning and tool blocks are lost and later turns break or degrade. Save response.content.
  • Slicing history at an arbitrary index. You can start with an assistant turn or orphan a tool_result. Cut at a real user turn.
  • Editing old turns to "clean up" history. Every edit is a cache miss and drops carried-over reasoning. Append; compact rarely.
  • Putting the summary in the system prompt and updating it every turn. The system prompt is the start of the cached prefix; changing it invalidates everything after it.
  • Storing memories users cannot see. Long-term memory is personal data: make it visible, editable, and deletable.

Try it

Compare memory strategies on a real conversation

  1. Run chat-cli.ts for twelve turns: introduce yourself, state two preferences, then ask unrelated questions.
  2. Apply slidingWindow(messages, 6) and ask "What did I say my preferences were?" Note the answer.
  3. Reset, replay the same conversation, apply compactHistory(messages, 6) instead, and ask again. Compare the answers and the input token counts.

Check your understanding

Why must the sliding window start at a user turn that is not a tool_result?
What is a hidden cost of summarizing history on every turn?
A user says 'I'm vegetarian' and expects every future conversation to know it. Which strategy fits?

Key takeaways

  • Your app owns memory: store the conversation and resend it, replaying each assistant turn's content blocks unchanged.
  • Choose a strategy per feature: full history, sliding window, summary plus recent turns, or retrieval memory.
  • Trim only at real user turns so tool calls stay paired; compact rarely and append otherwise.
  • Server-side compaction can summarize for you; long-term memory across conversations is retrieval, with user control.

Next steps#

Finished this article? Track it to see your progress on the roadmap.