Claude API Vision: Image Input Tutorial

Claude's API supports vision: you can send one or more images alongside text in a single request and get back a text response that describes, analyzes, or reasons about the image content. This works through the same messages endpoint you use for text, just with an additional content block type for images. No separate vision endpoint, no special model unless you want the fastest/cheapest option for simple tasks.

This tutorial covers the two supported ways to send images (base64 and URL), the request format, common pitfalls like file size and format limits, and how to send multiple images in one call for comparison tasks.

How image input works in the Claude API

A standard text-only message looks like this:

{
  "role": "user",
  "content": "What is in this image?"
}

For vision, the content field becomes an array of blocks, mixing image and text types:

{
  "role": "user",
  "content": [
    {
      "type": "image",
      "source": {
        "type": "base64",
        "media_type": "image/jpeg",
        "data": "<base64-encoded-image-data>"
      }
    },
    {
      "type": "text",
      "text": "What is in this image?"
    }
  ]
}

Claude reads the image block(s) first, then the accompanying text as the instruction or question. Order matters for clarity in prompts with multiple images, but Claude will still process images and text regardless of position.

Supported image formats and limits

• Formats: JPEG, PNG, GIF (non-animated), WEBP
• Size: keep individual images under ~5MB for best results; larger files get resized or rejected depending on the client
• Resolution: very large images are automatically downscaled before Claude processes them, so there's no benefit to sending ultra-high-res originals
• Multiple images: you can include several image blocks in one message — useful for comparisons, before/after analysis, or multi-page documents

Step 1: Encode your image as base64

Most implementations send images as base64 strings. Here's how to do it in a shell script:

IMAGE_BASE64=$(base64 -i photo.jpg)

In Node.js:

import fs from "fs";

const imageBuffer = fs.readFileSync("photo.jpg");
const imageBase64 = imageBuffer.toString("base64");

Then build the request payload with that string as the data field, matching the media_type to your actual file type (image/jpeg, image/png, etc.). Mismatched media types are a common source of errors — a PNG saved with a .jpg extension will still fail if you report the wrong media type.

Step 2: Send the request

If you're calling the Claude API directly, the request goes to the messages endpoint with your model, image block, and text prompt. If you're using SubToAPI as your access layer, the format is nearly identical — same message structure, same image blocks — but authenticated with your sub_live_... key instead of a provider-specific credential:

curl https://api.subtoapi.app/v1/messages \
  -H "Authorization: Bearer $SUBTOAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet",
    "max_tokens": 1024,
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "image",
            "source": {
              "type": "base64",
              "media_type": "image/jpeg",
              "data": "'"$IMAGE_BASE64"'"
            }
          },
          {
            "type": "text",
            "text": "Describe what is happening in this image."
          }
        ]
      }
    ]
  }'

The response comes back in the same shape as a text-only call — a content array with the model's analysis as text, plus usage metadata so you can track token consumption on image-heavy requests (images count toward your input tokens too).

Sending multiple images in one request

For comparison tasks — spotting differences between two screenshots, checking a document against a reference, or summarizing a multi-page scan — stack multiple image blocks in the same content array:

{
  "role": "user",
  "content": [
    { "type": "image", "source": { "type": "base64", "media_type": "image/png", "data": "<image1>" } },
    { "type": "image", "source": { "type": "base64", "media_type": "image/png", "data": "<image2>" } },
    { "type": "text", "text": "What changed between these two screenshots?" }
  ]
}

Claude processes them in the order given, so if sequence matters (e.g., "before" then "after"), keep the images in that order and say so explicitly in your text prompt to avoid ambiguity.

Practical use cases worth building

• Document extraction — pull structured data (invoice totals, dates, line items) from scanned receipts or PDFs converted to images
• UI review — feed a screenshot and ask Claude to flag accessibility issues, layout bugs, or inconsistent styling
• Content moderation — classify or describe uploaded images before they go live
• Diagram and chart reading — ask Claude to summarize a chart's trend or extract numbers from a graph image

For any of these, combine vision with Claude's tool use so the model can call a function with the extracted data instead of just returning prose — see /docs/tools for the format.

Common mistakes to avoid

• Forgetting the media_type field — it's required and must match the actual file, not just the extension
• Sending images too large — resize before encoding if you're hitting payload size limits
• Not setting max_tokens high enough — detailed image descriptions can run long; a low cap truncates the response mid-sentence
• Skipping streaming on long analyses — if you're asking for detailed multi-image comparisons, streaming the response improves perceived latency; see /docs/streaming

If you're building on top of this, /docs/messages has the full request/response schema and /docs/quickstart walks through getting a key and making your first call. Signing up at /signup includes a free trial so you can test vision requests before committing to a plan — pricing details are at /pricing.

questions

Can Claude API read text inside an image, like a scanned document? Yes. Claude can extract and transcribe text from images, including receipts, forms, and screenshots, though very low-resolution or heavily skewed scans reduce accuracy. For high-volume document extraction, test with a representative sample first.

Do images count toward token usage and cost? Yes, images are converted into input tokens based on their size and resolution before processing. Larger or higher-resolution images consume more tokens, so resizing to a reasonable resolution before sending reduces cost without hurting accuracy for most tasks.

Can I send an image by URL instead of base64? Some implementations support a URL-based image source alongside base64. Check your specific client's documentation for the exact field name, since base64 is the most universally supported format across API integrations.