How to Build a Claude-Powered AI Agent With Memory

Building a Claude-powered AI agent with memory means designing a system that remembers what happened in previous turns (and ideally previous sessions) and feeds that context back into every new request. Claude itself is stateless — each API call only knows what's in the messages array you send it. Memory is something you build on top: a storage layer, a retrieval strategy, and a way to keep the context window from overflowing.

This guide walks through the three layers of memory you need — short-term (conversation), mid-term (summarized), and long-term (retrieval-based) — plus a working code pattern you can adapt to any stack.

Why "memory" isn't built into the model

Every Claude API call is independent. If you want the agent to remember a user's name, a decision made three turns ago, or a preference from last week, you have to:

1. Store the conversation somewhere (a database, not just client state)
2. Decide what to re-send on the next call
3. Compress or filter that history so you don't blow past token limits or pay for redundant context on every request

Get these three right and the agent "feels" like it has memory, even though the underlying model doesn't.

Layer 1: Short-term memory (the message window)

The simplest form of memory is just passing the running conversation back to Claude on every call. This is what most chatbots do and it works fine for short interactions.

{
  "model": "claude-sonnet-4-5",
  "max_tokens": 1024,
  "messages": [
    { "role": "user", "content": "My name is Priya and I prefer concise answers." },
    { "role": "assistant", "content": "Got it, Priya. I'll keep things brief." },
    { "role": "user", "content": "What's the capital of France?" }
  ]
}

The problem: as conversations grow, the message array grows with them. Eventually you hit context limits, latency increases, and cost per call rises because you're resending the same tokens repeatedly.

Layer 2: Mid-term memory (summarization)

Once a conversation passes a certain length — say 15-20 turns — summarize older turns into a compact block and drop the raw messages. A common pattern:

async function compressHistory(messages, summarizeFn) {
  if (messages.length <= 20) return messages;

  const older = messages.slice(0, -10);
  const recent = messages.slice(-10);

  const summary = await summarizeFn(older);

  return [
    { role: "user", content: `Conversation summary so far: ${summary}` },
    ...recent
  ];
}

The summarizeFn is itself a Claude call — you ask the model to compress the older messages into a few sentences covering facts, decisions, and preferences, then prepend that summary before the most recent raw turns. This keeps token usage flat regardless of how long the conversation runs.

Layer 3: Long-term memory (retrieval)

For an agent that needs to remember things across sessions — user preferences, past projects, prior decisions — you need persistent storage with retrieval, not just a rolling window. The standard pattern:

1. After each session, extract durable facts ("user prefers TypeScript", "project uses Postgres") and store them with embeddings in a vector database (or even a simple keyword-indexed table for smaller use cases)
2. On a new session, embed the incoming query and retrieve the top-k relevant facts
3. Inject those facts into the system prompt before sending to Claude

const memories = await vectorStore.query(userQuery, { topK: 5 });

const systemPrompt = `You are a helpful assistant.
Known facts about this user:
${memories.map(m => `- ${m.text}`).join("\n")}`;

const response = await claudeClient.messages.create({
  model: "claude-sonnet-4-5",
  system: systemPrompt,
  max_tokens: 1024,
  messages: [{ role: "user", content: userQuery }]
});

You don't need a dedicated vector database to start — a Postgres table with pgvector, or even a simple relevance-scored keyword search, works for most agents handling a few thousand users. Reach for a managed vector store only once retrieval quality or scale demands it.

Adding tool use for active memory management

A more advanced pattern is letting the agent decide when to save or recall memory itself, using Claude's tool-use capability. Define tools like save_memory and recall_memory, and let the model call them mid-conversation:

{
  "tools": [
    {
      "name": "save_memory",
      "description": "Store a fact about the user for future conversations",
      "input_schema": {
        "type": "object",
        "properties": { "fact": { "type": "string" } },
        "required": ["fact"]
      }
    }
  ]
}

When Claude decides a statement is worth remembering ("I'm allergic to shellfish"), it calls save_memory with the extracted fact, your backend writes it to storage, and you return a tool result confirming the save. This is more reliable than blind post-hoc extraction because the model is reasoning about relevance in real time. SubToAPI's /docs/tools endpoint documentation covers the request/response shape for tool calls if you're wiring this through an API key rather than calling Claude directly.

Where SubToAPI fits

If you're building this agent as part of a product — not just a personal script — you'll eventually need application-level API keys, streaming responses for a responsive UI, and usage visibility per user or per customer. SubToAPI wraps your existing Claude access into a standard HTTPS API (sub_live_... keys) so your agent's memory-management code talks to one stable endpoint:

curl https://api.subtoapi.app/v1/messages \
  -H "Authorization: Bearer $SUBTOAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-5",
    "max_tokens": 1024,
    "system": "Known facts: user prefers concise answers, works in Python.",
    "messages": [{"role": "user", "content": "How do I memoize a function?"}]
  }'

This matters most once you're running a memory-enabled agent for multiple users or teams — you get per-key usage metadata to see which customers are driving token consumption, which is exactly the kind of data you need to tune your summarization thresholds. See /docs/quickstart and /docs/messages for the full request format, and /docs/streaming if you want the agent's responses to stream into your frontend instead of waiting for a full completion.

Putting it together

A production-grade memory architecture typically combines all three layers:

• Short-term: raw messages for the current session, capped at ~20 turns
• Mid-term: a running summary that replaces older turns once the cap is hit
• Long-term: a retrieval store queried at session start and updated via explicit save_memory tool calls

Start simple — short-term plus summarization covers most chatbot and support-agent use cases. Add the retrieval layer only when users genuinely need cross-session recall, since it adds infrastructure (embeddings, a vector store, extraction jobs) you'll need to maintain. You can prototype the whole thing with a free trial at /signup and check /pricing once you're ready to scale past a single Claude account.

Questions

Does Claude have built-in memory across sessions? No. Each API call is stateless — the model only sees what's in the messages array you send. Any persistence across sessions has to be built by your application using storage and retrieval.

How do I stop the context window from filling up in long conversations? Summarize older turns into a compact block once the conversation passes a length threshold (commonly 15-20 turns), and keep only recent turns plus the summary in subsequent calls.

Do I need a vector database for agent memory? Not necessarily. A Postgres table with keyword or pgvector search handles most small-to-medium agents fine. Move to a dedicated vector store only when retrieval accuracy or query volume demands it.