How to Build an AI Agent Orchestrator

An AI agent orchestrator is the layer that decides which agent handles a task, passes context between steps, manages tool calls, and handles retries when something fails. Building one means solving four problems: task routing, state management, tool execution, and failure recovery. You don't need a heavyweight framework to do this — a well-structured orchestrator can be a few hundred lines of code sitting on top of a solid LLM API.

This guide walks through the core components you need, in the order you'll actually build them, with working code you can adapt.

What an Orchestrator Actually Does

Before writing code, it helps to separate "agent" from "orchestrator." An agent is a single LLM call (or loop) with a specific role — a researcher, a summarizer, a code reviewer. The orchestrator is the controller that:

• Decides which agent runs next based on the current state
• Passes outputs from one agent as inputs to another
• Executes tool/function calls the model requests
• Tracks conversation and task state across steps
• Retries or falls back when a step fails

If you skip the orchestrator and just chain prompts manually, you end up re-implementing all of this ad hoc, badly, inside your application logic. Building it properly once pays off fast.

Step 1: Define Your Agent Interface

Every agent should implement the same contract so the orchestrator can treat them interchangeably:

class Agent {
  constructor(name, systemPrompt, tools = []) {
    this.name = name;
    this.systemPrompt = systemPrompt;
    this.tools = tools;
  }

  async run(messages, client) {
    return client.messages.create({
      model: "claude-sonnet-4",
      system: this.systemPrompt,
      messages,
      tools: this.tools,
      max_tokens: 1024,
    });
  }
}

This keeps each agent dumb and swappable. All the routing intelligence lives in the orchestrator, not scattered across agent classes.

Step 2: Build the Router

The router decides which agent handles the next step. For simple pipelines, a fixed sequence works. For dynamic workflows, use a classifier call or a rules engine:

function routeTask(task, agents) {
  if (task.type === "research") return agents.researcher;
  if (task.type === "code") return agents.coder;
  if (task.type === "review") return agents.reviewer;
  return agents.generalist;
}

For anything beyond trivial branching, let a lightweight model call do the classification instead of hardcoding rules — it scales better as you add agent types.

Step 3: Manage Shared State

The orchestrator needs a state object that persists across agent calls: conversation history, intermediate outputs, tool results, and task status.

class OrchestratorState {
  constructor(taskId) {
    this.taskId = taskId;
    this.history = [];
    this.outputs = {};
    this.status = "running";
  }

  record(agentName, output) {
    this.outputs[agentName] = output;
    this.history.push({ role: "assistant", content: output });
  }
}

This is also where you log timestamps and token usage per step — essential once you're running multi-agent pipelines in production and need to debug why a task took 40 seconds or cost more than expected.

Step 4: Handle Tool Calls in the Loop

Most real orchestrators run a loop: call the model, check if it wants to use a tool, execute the tool, feed the result back, repeat until the model returns a final answer.

async function runAgentLoop(agent, state, client) {
  let messages = [...state.history];

  while (true) {
    const response = await agent.run(messages, client);

    if (response.stop_reason === "tool_use") {
      const toolCall = response.content.find(b => b.type === "tool_use");
      const result = await executeTool(toolCall.name, toolCall.input);

      messages.push({ role: "assistant", content: response.content });
      messages.push({
        role: "user",
        content: [{ type: "tool_result", tool_use_id: toolCall.id, content: result }],
      });
      continue;
    }

    state.record(agent.name, response.content);
    return response;
  }
}

This is the heart of the orchestrator. Everything else — routing, retries, logging — wraps around this loop. See /docs/tools for details on the tool-call contract if you're using SubToAPI as the backend.

Step 5: Add Retries and Fallbacks

Agents fail — rate limits, malformed tool arguments, timeouts. Wrap each agent call with retry logic and a fallback agent if the primary one fails repeatedly:

async function runWithRetry(agent, state, client, maxAttempts = 3) {
  for (let i = 0; i < maxAttempts; i++) {
    try {
      return await runAgentLoop(agent, state, client);
    } catch (err) {
      if (i === maxAttempts - 1) throw err;
      await new Promise(r => setTimeout(r, 500 * (i + 1)));
    }
  }
}

For production systems, log the failure reason and attempt count per task. This becomes invaluable when you're debugging why a pipeline stalled at 2am.

Step 6: Pick the Right API Layer Underneath

Your orchestrator is only as reliable as the API it calls. If you're using Claude under a personal or team subscription rather than a pay-per-token API account, you need a layer that exposes it as a proper HTTPS endpoint with streaming, tool support, and usage metadata — which is exactly what SubToAPI does. You get a sub_live_... application key, standard request/response shapes, and per-key usage tracking so you can see which agent or task is consuming the most tokens.

Setup is a signup, an API key, and a base URL swap — see /docs/quickstart. The orchestrator code above works unchanged against it; you're just pointing client at https://api.subtoapi.app/v1/messages with your key in the Authorization header. Streaming support (/docs/streaming) matters here too — for long multi-agent runs, you want partial output as each agent completes its step, not a blocking wait until the whole pipeline finishes.

Putting It Together

A minimal orchestrator loop looks like:

async function orchestrate(task, agents, client) {
  const state = new OrchestratorState(task.id);
  let currentTask = task;

  while (state.status === "running") {
    const agent = routeTask(currentTask, agents);
    const result = await runWithRetry(agent, state, client);
    currentTask = determineNextStep(result, state);
    if (!currentTask) state.status = "done";
  }

  return state;
}

Start simple: one router, two or three agents, one shared state object. Add retries once you see real failures, not before. Add dynamic routing once fixed sequences stop covering your use cases. Most orchestrator complexity should be earned, not pre-built.

questions

Do I need LangChain or a similar framework to build an orchestrator? No. Frameworks help with boilerplate but the core logic — routing, state, tool loop, retries — is straightforward to write yourself and easier to debug when you own it.

How many agents should an orchestrator manage before it gets complex? Complexity comes from branching logic, not agent count. Three agents with dynamic routing can be harder to reason about than ten agents in a fixed pipeline.

What's the best way to expose Claude for a multi-agent system? Use a proper API layer with streaming and usage metadata rather than raw subscription access. See /pricing for options if you're evaluating SubToAPI for this.