Multi-Turn Chat and Conversation Memory with the Claude API
Build multi-turn chat in Node.js: keep history, replay content blocks correctly, and control cost with sliding windows, summaries, compaction, and long-term memory.
On this page· History is just an array you resend
Why this matters
The model forgets everything between requests, so "memory" is something your application builds. Done naively, every turn resends the whole conversation and both cost and latency grow without limit. Done carelessly, you trim the wrong message and the model loses the user's name, a decision from ten minutes ago, or half of a tool call, and the API rejects the request. Conversation memory is a design decision with clear trade-offs.
History is just an array you resend#
Step 1 of 9
“Hi, I'm Ada.”
The first message of a new conversation.
Show text descriptionHide text description
Three turns of a conversation. Each request to the model contains the entire conversation so far. Participants, left to right: User, Your app, LLM API.
- User → Your app: “Hi, I'm Ada.” · The first message of a new conversation.
- Your app → LLM API: messages: [user₁] (1 message) · The request contains only the first message.
- LLM API → Your app: “Hi Ada!” · The app saves the reply as assistant₁ in its database. The model keeps nothing.
- User → Your app: “What's my name?” · A follow-up that depends on the first turn.
- Your app → LLM API: messages: [user₁, assistant₁, user₂] (3) · The app sends the full history. Without it, the model could not know the name.
- LLM API → Your app: “Your name is Ada.” · Correct only because the history was included.
- User → Your app: “Summarize our chat.” · A third turn.
- Your app → LLM API: messages: [5 messages] (5) · The request grows every turn. Long chats eventually need trimming or summarizing, and prompt caching to keep cost down.
- LLM API → Your app: summary · Each answer costs input tokens for the whole history, not just the new message.
The messages array follows a few rules: it starts with a user turn, turns alternate between user and assistant (consecutive turns of the same role are merged), and each assistant turn is stored exactly as returned: the full response.content, including reasoning and tool blocks.
import Anthropic from "@anthropic-ai/sdk";
export const anthropic = new Anthropic({ timeout: 60_000, maxRetries: 2 });
export const MODEL = "claude-opus-5-5";import { createInterface } from "node:readline/promises";
import type Anthropic from "@anthropic-ai/sdk";
import { anthropic, MODEL } from "./lib/anthropic.ts";
const rl = createInterface({ input: process.stdin, output: process.stdout });
const messages: Anthropic.MessageParam[] = [];
console.log('Chat with the model. Type "exit" to quit.');
while (true) {
const input = (await rl.question("\nyou › ")).trim();
if (!input || input === "exit") break;
messages.push({ role: "user", content: input });
const response = await anthropic.messages.create({
model: MODEL,
max_tokens: 4096,
output_config: { effort: "low" },
system: "You are a friendly, concise assistant.",
messages,
});
messages.push({ role: "assistant", content: response.content }); // store the blocks, not just the text
const text = response.content.flatMap((b) => (b.type === "text" ? [b.text] : [])).join("");
console.log(`\nassistant › ${text}`);
console.log(` (${messages.length} messages so far, ${response.usage.input_tokens} input tokens this turn)`);
}
rl.close();Watch the input token count grow with every turn. That number is your cost per turn, and it is the reason the rest of this article exists.
In a web app, the array lives in your database, not in memory: load the conversation's messages, append the new user turn, call the model, and save both turns in one transaction. The Databases article shows getHistory and saveTurn for exactly this.
Four memory strategies#
Long conversations need a policy for what the model sees. Each strategy trades cost against what the model can still remember.
- System prompt
- Summary of older turns
- Retrieved memories
- Recent turns
Show text descriptionHide text description
Input tokens sent on turn 40 of a conversation averaging 250 tokens per turn, with an 800-token system prompt. Fewer tokens mean lower cost and latency; the trade-off is what the model can no longer see.
| Strategy | System prompt | Summary of older turns | Retrieved memories | Recent turns | Total |
|---|---|---|---|---|---|
| Full history (all 40 turns) | 800 | 0 | 0 | 10,000 | 10,800 |
| Sliding window (last 10 turns) | 800 | 0 | 0 | 2,500 | 3,300 |
| Summary + recent (summary + last 6 turns) | 800 | 450 | 0 | 1,500 | 2,750 |
| Retrieval memory (relevant facts + last 6 turns) | 800 | 0 | 600 | 1,500 | 2,900 |
| Strategy | Keeps | Loses | Use when |
|---|---|---|---|
| Full history | Everything | Nothing, until the context limit | Short conversations, high-stakes tasks |
| Sliding window | The last N turns verbatim | Everything older | Casual chat where old turns rarely matter |
| Summary + recent | A running summary plus recent turns | Exact wording of older turns | Long support or coaching sessions |
| Retrieval memory | Relevant stored facts plus recent turns | Facts that were never stored | Assistants that span many conversations |
With prompt caching (covered later), the repeated history prefix becomes cheap, so full history stays affordable much longer than the raw token counts suggest. Trim when you approach your latency budget or the context limit, not at the first sign of growth.
Sliding window, without breaking tool calls#
The naive messages.slice(-10) can start the array with an assistant turn, or keep a tool_result whose matching tool_use was cut off. Both make the request invalid. Cut only at a real user turn:
import type Anthropic from "@anthropic-ai/sdk";
import { anthropic, MODEL } from "./anthropic.ts";
const textOf = (m: Anthropic.MessageParam) =>
typeof m.content === "string" ? m.content : m.content.flatMap((b) => (b.type === "text" ? [b.text] : [])).join(" ");
const isToolResultTurn = (m: Anthropic.MessageParam) =>
m.role === "user" && Array.isArray(m.content) && m.content.some((b) => b.type === "tool_result");
/** Keep roughly the last `keep` messages, starting at a real user turn so tool_use/tool_result pairs stay together. */
export function slidingWindow(messages: Anthropic.MessageParam[], keep: number): Anthropic.MessageParam[] {
if (messages.length <= keep) return messages;
let start = messages.length - keep;
while (start < messages.length && (messages[start].role !== "user" || isToolResultTurn(messages[start]))) start++;
return messages.slice(start);
}
/** Replace older turns with a short summary, keeping recent turns verbatim. */
export async function compactHistory(messages: Anthropic.MessageParam[], keepRecent = 6): Promise<Anthropic.MessageParam[]> {
const recent = slidingWindow(messages, keepRecent);
const older = messages.slice(0, messages.length - recent.length);
if (older.length === 0) return messages;
const summary = await anthropic.messages.create({
model: MODEL,
max_tokens: 1024,
output_config: { effort: "low" },
system:
"Summarize this conversation for your own future reference. Keep names, decisions, user preferences, " +
"facts the user shared, and open questions. Write under 150 words.",
messages: [{ role: "user", content: older.map((m) => `${m.role}: ${textOf(m)}`).join("\n") }],
});
const text = summary.content.flatMap((b) => (b.type === "text" ? [b.text] : [])).join("");
// A user turn holding the summary, then the recent turns (which also start with a user turn; the two are merged).
return [{ role: "user", content: `Summary of our earlier conversation:\n${text}` }, ...recent];
}Server-side compaction#
Instead of summarizing yourself, you can let the API do it. With the compaction beta enabled, the API summarizes older context automatically when a conversation approaches a threshold and returns a compaction block that replaces the older turns. The rule from earlier matters even more here: append response.content every turn, so the compaction block is sent back.
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const messages: Anthropic.Beta.BetaMessageParam[] = [];
async function chat(userMessage: string): Promise<string> {
messages.push({ role: "user", content: userMessage });
const response = await client.beta.messages.create({
betas: ["compact-2026-01-12"],
model: "claude-opus-5-5",
max_tokens: 16000,
messages,
context_management: { edits: [{ type: "compact_20260112" }] },
});
messages.push({ role: "assistant", content: response.content }); // keeps compaction blocks
return response.content.flatMap((b) => (b.type === "text" ? [b.text] : [])).join("");
}
console.log(await chat("Help me plan a three-day trip to Lisbon."));
console.log(await chat("Make day two about food markets."));Long-term memory across conversations#
Everything above is memory within one conversation. Remembering a user across conversations ("I'm vegetarian", "our stack is Postgres") is a separate feature: extract durable facts, store them per user, and retrieve the relevant ones into the context of new conversations. That is retrieval memory, built on the same embedding search as RAG and covered in the agent memory article. Always let users see and delete what is stored about them.
Common mistakes#
- Saving only the assistant's text. Reasoning and tool blocks are lost and later turns break or degrade. Save
response.content. - Slicing history at an arbitrary index. You can start with an assistant turn or orphan a
tool_result. Cut at a real user turn. - Editing old turns to "clean up" history. Every edit is a cache miss and drops carried-over reasoning. Append; compact rarely.
- Putting the summary in the system prompt and updating it every turn. The system prompt is the start of the cached prefix; changing it invalidates everything after it.
- Storing memories users cannot see. Long-term memory is personal data: make it visible, editable, and deletable.
Try it
Compare memory strategies on a real conversation
- Run
chat-cli.tsfor twelve turns: introduce yourself, state two preferences, then ask unrelated questions. - Apply
slidingWindow(messages, 6)and ask "What did I say my preferences were?" Note the answer. - Reset, replay the same conversation, apply
compactHistory(messages, 6)instead, and ask again. Compare the answers and the input token counts.
Check your understanding
Key takeaways
- Your app owns memory: store the conversation and resend it, replaying each assistant turn's content blocks unchanged.
- Choose a strategy per feature: full history, sliding window, summary plus recent turns, or retrieval memory.
- Trim only at real user turns so tool calls stay paired; compact rarely and append otherwise.
- Server-side compaction can summarize for you; long-term memory across conversations is retrieval, with user control.
Next steps#
Finished this article? Track it to see your progress on the roadmap.
Related articles
- Your First LLM API Call in Node.js: Request and Response AnatomyMake your first Claude API call from Node.js and TypeScript, and learn every part of the request and response: messages, system, max_tokens, stop_reason, usage.
- Agent Memory, State, and Durable Execution with PostgresGive agents memory that lasts and runs that survive crashes: memory tiers, checkpointed steps in Postgres, leases, and idempotent side effects.