Best Local Model for Function Calling in 2025

If you're picking a local model for function calling, the short answer is: Qwen2.5 (7B–32B), Llama 3.1/3.3 (70B+), and Hermes-3/Firefunction-v2 fine-tunes are the strongest open options right now. Qwen2.5 gives the best accuracy-to-size ratio if you're running on a single GPU. Llama 3.1 70B or larger is more reliable for multi-step tool chains but needs serious hardware. Mistral's function-calling checkpoints are usable but lag behind both on strict JSON schema adherence.

The harder question is what "best" means for your setup. Function calling isn't one skill — it's several: deciding when to call a tool, picking the right tool among several similar ones, formatting arguments that match a JSON schema exactly, and handling the result correctly in a follow-up turn. A model can be good at one of these and bad at another, which is why benchmark leaderboards often don't match what you'll see in your own app.

What actually matters in a function-calling model

Before comparing models, it helps to know what to test for, because raw parameter count is a weak predictor here.

• Schema adherence — does the model emit valid JSON matching your tool's parameter types, or does it drift (strings instead of numbers, missing required fields, extra commentary before the JSON)?
• Tool selection accuracy — with 5–15 tools available, does it pick the correct one, or does it default to the first/most common tool in the list?
• Parallel and multi-step calls — can it call two tools in one turn, or chain a tool call → result → another tool call without losing context?
• Refusal to hallucinate calls — does it invoke a tool when none is needed, just because tools are present in the prompt?
• Quantization sensitivity — function calling degrades faster than plain chat quality under aggressive quantization (Q4 and below), because argument formatting needs precision.

The models worth running

Qwen2.5-7B/14B/32B-Instruct

Qwen2.5's instruct models are trained with native tool-use formatting and are noticeably better than similarly sized Llama models at strict JSON output. The 14B variant is a good default for a single 24GB GPU — it handles 5–10 tool schemas without much confusion. The 32B version closes most of the gap with much larger models on tool selection accuracy.

Llama 3.1 / 3.3 (70B+)

Meta's larger Llama models have solid native function-calling support via their built-in tool format. They're more forgiving with ambiguous or overlapping tool descriptions than smaller models, and they handle longer multi-turn tool conversations better. The tradeoff is hardware: realistically you need 2×A100/H100 or a quantized 4-bit setup with enough VRAM to avoid quality loss. The 8B model exists but its tool-calling reliability drops noticeably compared to 70B.

Hermes-3 / Firefunction-v2

These are fine-tunes built specifically for function calling rather than general chat. Hermes-3 (based on Llama 3.1) is tuned with a large synthetic dataset of tool-call examples and tends to be more consistent at emitting clean JSON with no extra prose — useful if you're parsing output with a strict regex or JSON.loads and don't want to deal with markdown fences or commentary. Firefunction-v2 (Fireworks) is optimized for latency and structured output and performs well in agent loops with many available tools.

Mistral / Mixtral function-calling checkpoints

Mistral's tool-use models work but are more prone to formatting drift under quantization, and tool selection accuracy drops faster as the number of available tools grows past 5–6. They're a reasonable choice if you're already standardized on Mistral for other reasons, but not the first pick if function calling is the primary workload.

A quick way to test on your own tools

Don't trust leaderboard numbers alone — run your actual tool schemas through candidate models with a small, hand-labeled test set (20–30 examples covering ambiguous cases, multi-tool scenarios, and cases where no tool should be called).

curl http://localhost:11434/api/chat \
  -d '{
    "model": "qwen2.5:14b",
    "messages": [{"role": "user", "content": "What is the weather in Berlin and convert 20 EUR to USD"}],
    "tools": [
      {"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}},
      {"type": "function", "function": {"name": "convert_currency", "parameters": {"type": "object", "properties": {"amount": {"type": "number"}, "from": {"type": "string"}, "to": {"type": "string"}}}}}
    ]
  }'

Check three things in the output: did it call both tools (parallel calling), are the argument types correct, and did it avoid inventing a third tool that doesn't exist.

Where local models still fall short

Even the best local options trail hosted frontier models on two things: tool selection at scale (15+ tools with overlapping purposes) and long tool-call chains where the model has to reason about results from a previous call before deciding the next one. If your product depends on reliable multi-step agent behavior — not just single tool calls — it's worth benchmarking against Claude, which has stronger native tool-use behavior at the reasoning layer.

If you want Claude-level function calling without managing infrastructure yourself, SubToAPI turns your existing Claude access into a standard HTTPS API — with structured tool use, streaming, and usage metadata over a normal sub_live_... key. It's not a local model, but it's a useful comparison point when deciding whether self-hosting is worth the engineering cost versus an API you can wire up in minutes. See the tool use docs and quickstart for the request format, or check pricing if you're evaluating cost against running your own GPU box.

const res = await fetch("https://api.subtoapi.app/v1/messages", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${process.env.SUBTOAPI_KEY}`,
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    model: "claude-sonnet-4",
    max_tokens: 1024,
    tools: [{
      name: "get_weather",
      description: "Get current weather for a city",
      input_schema: {
        type: "object",
        properties: { city: { type: "string" } },
        required: ["city"]
      }
    }],
    messages: [{ role: "user", content: "What's the weather in Berlin?" }]
  })
});

Bottom line

For most self-hosted setups, Qwen2.5-14B or 32B is the best balance of accuracy and hardware cost. If you have the GPUs, Llama 3.1/3.3 70B or a Hermes-3 fine-tune will be more reliable on complex, multi-tool workloads. Always validate against your own tool schemas — general benchmarks don't capture how a model behaves with your specific set of functions.

questions

Is a 7B model good enough for function calling? For simple, single-tool tasks with clear intent, yes — Qwen2.5-7B handles basic calls well. For 5+ overlapping tools or multi-step chains, step up to 14B or larger for meaningfully better accuracy.

Does quantization hurt function calling more than regular chat? Yes. JSON formatting and argument precision degrade faster under 4-bit and lower quantization than conversational fluency does. Stick to Q5/Q6 or higher if tool calling reliability matters.

Should I self-host or use a hosted API for function calling? Self-hosting makes sense if you have GPU capacity and predictable, narrow tool sets. For complex or high-stakes agent workflows, a hosted model with stronger reasoning (like Claude via a service such as SubToAPI) often outperforms local models without infrastructure overhead — see the messages docs for the request shape.