How to Fine Tune Prompts for Tool Use

Fine tuning prompts for tool use means adjusting your tool descriptions, system prompt, and examples until the model reliably picks the right tool, fills in correct parameters, and knows when not to call a tool at all. It's not about retraining weights — it's prompt-level engineering that directly affects whether your tool-calling pipeline is production-ready or flaky.

Most tool-calling failures aren't model failures. They're prompt failures: ambiguous tool names, vague parameter descriptions, missing edge-case guidance, or a system prompt that never tells the model when a tool is unnecessary. This guide walks through the specific levers you can pull to fix that, with examples you can apply immediately.

Why Tool Use Prompts Need Separate Tuning

Regular prompt optimization focuses on output quality — tone, length, accuracy. Tool use prompting has an extra failure surface: the model has to decide whether to call a tool, which tool, and what arguments to pass, all before it generates any visible text. Small wording changes in a tool's description field often matter more than changes to your system prompt.

If you're new to tool calling with Claude, read through /docs/tools first — the rest of this article assumes you already have a working tool schema and are trying to make it more reliable.

Start With the Tool Description, Not the System Prompt

The single highest-leverage place to fine tune is the description field on each tool. The model uses this text, almost verbatim, to decide if a tool matches the user's intent. Vague descriptions cause two common failures: the model calls a tool it shouldn't, or it ignores a tool it should have used.

Weak:

{
  "name": "get_weather",
  "description": "Gets weather",
  "input_schema": { "type": "object", "properties": { "location": { "type": "string" } } }
}

Stronger:

{
  "name": "get_weather",
  "description": "Returns current weather conditions (temperature, precipitation, wind) for a specific city or ZIP code. Use this only when the user asks about present or near-term weather, not historical climate data.",
  "input_schema": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "City name or ZIP code, e.g. 'Austin, TX' or '94103'"
      }
    },
    "required": ["location"]
  }
}

Three changes did most of the work:

• Scope boundary — "only when... not historical" prevents misuse.
• Concrete examples in the parameter description — reduces malformed inputs.
• Explicit output contents — helps the model decide if this tool actually answers the user's question.

Tune the System Prompt for Decision Logic

Your tool descriptions tell the model what each tool does. Your system prompt should tell it how to decide between tools, or between a tool and a plain text answer. Without this, models tend to over-call tools even for questions they could answer directly.

A useful pattern:

You have access to tools for weather, currency conversion, and order lookup.
Call a tool only when the user's request requires real-time or account-specific
data that you cannot answer from general knowledge. If the user asks a general
question, answer directly without calling any tool. If a request is ambiguous
about which tool applies, ask a clarifying question instead of guessing.

This single paragraph fixes three common issues: unnecessary tool calls, wrong-tool selection on ambiguous input, and silent guessing when parameters are missing.

Use Few-Shot Examples for Edge Cases, Not Happy Paths

Few-shot examples in tool-use prompts are most valuable when they demonstrate edge cases the model tends to get wrong — not the obvious case it already handles correctly. If your logs show the model calling get_weather when a user asks about climate trends, add one example showing the correct non-call or a different tool choice.

User: What's the weather like in Chicago right now?
Assistant: [calls get_weather with location="Chicago, IL"]

User: Has Chicago gotten warmer over the last decade?
Assistant: I don't have a tool for historical climate trends, so I'll answer
from general knowledge: ...

Keep examples short and limit them to 2–4 per tool. Long few-shot blocks increase token cost and can dilute the effect of your system prompt instructions.

Iterate With Real Transcripts, Not Guesses

Fine tuning tool prompts is an empirical process. Pull actual failed tool calls from your logs — wrong tool selected, missing required parameter, hallucinated parameter value — and use each failure as the seed for a prompt fix. Guessing at improvements without transcript evidence usually just shifts the failure mode rather than fixing it.

A simple test loop:

const testCases = [
  { input: "What's 50 USD in EUR?", expectedTool: "convert_currency" },
  { input: "Is it going to rain tomorrow in Boston?", expectedTool: "get_weather" },
  { input: "What's the capital of France?", expectedTool: null }
];

for (const tc of testCases) {
  const res = await fetch("https://api.subtoapi.app/v1/messages", {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${process.env.SUBTOAPI_KEY}`,
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      model: "claude-sonnet-4",
      max_tokens: 512,
      system: SYSTEM_PROMPT,
      tools: TOOL_DEFINITIONS,
      messages: [{ role: "user", content: tc.input }]
    })
  });
  const data = await res.json();
  const toolCall = data.content.find(c => c.type === "tool_use");
  console.log(tc.input, "->", toolCall?.name ?? null, "expected:", tc.expectedTool);
}

Run this set after every prompt change. It takes a few minutes and catches regressions before they reach production. If you're building this on top of SubToAPI, the same /v1/messages endpoint used in your app works directly in this kind of test harness — see /docs/messages for the full request format and /docs/streaming if your tool-calling flow also streams partial responses.

Common Mistakes to Fix First

• Overlapping tool descriptions — if two tools could plausibly handle the same request, the model will pick inconsistently. Narrow the scope of each.
• No fallback instruction — always tell the model what to do when no tool fits (ask a question, answer from knowledge, or say it can't help).
• Parameter descriptions without format hints — "date" is worse than "date in YYYY-MM-DD format."
• Testing only happy-path inputs — the real value of tuning shows up on ambiguous or adversarial inputs.
• Changing multiple things at once — adjust one tool description or one system prompt sentence per test cycle so you can attribute the effect correctly.

If you're setting up tool-calling access for the first time, /docs/quickstart walks through issuing an API key and sending your first request before you get into prompt tuning. A free trial gives you enough room to run these test loops without worrying about cost, and /pricing has the plan details once you're ready to scale usage across a team.

How long does it take to fine tune a tool-use prompt?

Expect a few iteration cycles — most teams land on a stable version within a day of testing against real transcripts, since each change is small and the feedback loop (run test cases, read output) is fast.

Do I need few-shot examples for every tool?

No. Add them only where logs show the model making mistakes. Tools with clear, non-overlapping use cases often work well with just a strong description and no examples.

Can prompt tuning fully eliminate wrong tool calls?

It significantly reduces them but won't guarantee zero errors on highly ambiguous input. Pair tuned prompts with validation of returned parameters and a fallback path for low-confidence tool selections.