How to Build a Voice-to-Text Pipeline With Claude

What a voice-to-text pipeline actually needs

A voice-to-text pipeline for Claude is a system that converts spoken audio into text, then passes that text to Claude for understanding, summarization, extraction, or action. It's not a single API call — it's two distinct stages chained together: speech-to-text (STT) and language processing. Most tutorials conflate these and leave you guessing how the handoff between them should actually work.

The short answer: you transcribe audio with a speech recognition engine (Whisper, Deepgram, AssemblyAI, or a browser's native Web Speech API), then send the resulting transcript as a plain text prompt to Claude through a standard messages request. Claude never touches audio directly — it only ever sees text. Everything below walks through the pipeline stage by stage, with code you can adapt to your stack.

Stage 1: capture and transcribe the audio

You have three realistic options for the transcription layer:

• Browser-native: the Web Speech API (SpeechRecognition) works for quick prototypes and live demos, but accuracy and browser support are inconsistent.
• Whisper (OpenAI) or open-source Whisper models: good accuracy, works offline if self-hosted, slightly higher latency.
• Deepgram / AssemblyAI: purpose-built for streaming transcription with low latency, better for live voice apps.

For a server-side pipeline, a typical transcription call looks like this (using Whisper's API as an example):

const formData = new FormData();
formData.append("file", audioBlob, "input.webm");
formData.append("model", "whisper-1");

const response = await fetch("https://api.openai.com/v1/audio/transcriptions", {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.WHISPER_KEY}` },
  body: formData,
});

const { text: transcript } = await response.json();

That transcript variable is the handoff point. Everything downstream is text processing, which is where Claude comes in.

Stage 2: send the transcript to Claude

Once you have clean text, treat it exactly like any other user input. This is where SubToAPI is useful: it turns your existing Claude access into a normal HTTPS API with application keys (sub_live_...), so your pipeline doesn't need separate provider accounts or billing for the language-processing half.

curl https://api.subtoapi.app/v1/messages \
  -H "Authorization: Bearer $SUBTOAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4",
    "max_tokens": 500,
    "messages": [
      {
        "role": "user",
        "content": "Here is a voice memo transcript. Summarize it into three action items:\n\n'"$transcript"'"
      }
    ]
  }'

For details on request structure and parameters, see the messages docs. If you're starting from zero, the quickstart guide walks through getting an API key and making your first call before you plug in the transcription stage.

Stage 3: handle real-time or long-form audio differently

Pipelines split into two shapes depending on what you're building:

Batch pipeline (voice memos, meeting recordings, uploaded files): transcribe the whole file first, then send the full transcript to Claude in one request. Simple, reliable, no concurrency issues.

Live pipeline (voice assistants, live captioning with follow-up commands): transcribe in chunks as the user speaks, and stream Claude's response back as each chunk completes. This requires streaming on both ends — streaming STT output and streaming Claude's reply. SubToAPI supports server-sent events for this; the streaming docs cover the event format and how to parse partial tokens as they arrive.

A minimal streaming consumer looks like this:

const res = await fetch("https://api.subtoapi.app/v1/messages", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.SUBTOAPI_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "claude-sonnet-4",
    max_tokens: 400,
    stream: true,
    messages: [{ role: "user", content: transcript }],
  }),
});

const reader = res.body.getReader();
const decoder = new TextDecoder();
let output = "";

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  output += decoder.decode(value);
  // parse SSE chunks here and render incrementally
}

Stage 4: make the transcript do something useful

Raw transcripts are messy — filler words, false starts, run-on sentences. A good prompt pattern is to ask Claude to clean the transcript and extract structure in the same call, rather than running separate cleanup and extraction steps:

{
  "model": "claude-sonnet-4",
  "max_tokens": 600,
  "messages": [
    {
      "role": "user",
      "content": "Clean up this raw voice transcript (remove filler words, fix punctuation) and extract any tasks mentioned as a list:\n\n[transcript]"
    }
  ]
}

If your pipeline needs to trigger actions based on what was said — creating a calendar event, filing a ticket, sending a message — pair the transcript with tool definitions so Claude can call functions instead of just returning text. The tools documentation covers schema design for this pattern.

Where to run this pipeline

For most teams, the transcription stage runs as a short-lived serverless function (triggered by file upload or a WebSocket chunk), and the Claude call runs right after it in the same function or a queued job. Keep the two stages decoupled in code even if they run back-to-back — it makes it easy to swap transcription providers later without touching your Claude logic.

If you're already paying for Claude access through a subscription, SubToAPI lets you reuse that access as the API layer for this exact use case, with per-user usage tracking and team seats if more than one person is building against it. Plans start at €9/month for solo use, with team and scale tiers for shared projects — see pricing for the breakdown, or sign up to get an API key and test the pipeline end to end.

questions

Does Claude transcribe audio directly? No. Claude processes text only. You need a separate speech-to-text engine (Whisper, Deepgram, AssemblyAI, or browser APIs) to produce a transcript before sending anything to Claude.

What's the best STT engine to pair with Claude? It depends on latency needs. Whisper is accurate for batch/offline use; Deepgram and AssemblyAI are better for live, low-latency streaming transcription feeding into a real-time Claude pipeline.

Can I stream both transcription and Claude's response at once? Yes, but they're independent streams. Transcribe audio in chunks, then stream each completed chunk (or the full transcript) to Claude using server-sent events — see the streaming docs for the SSE format.