Best Function Calling Open Source LLM in 2025

If you're building an agent or tool-using pipeline and want to avoid vendor lock-in, the short answer is: Qwen2.5 (especially the 72B and 32B variants), Llama 3.3-70B, and Hermes 3 (Llama 3.1 finetune) are currently the strongest open source options for function calling, based on their performance on the Berkeley Function-Calling Leaderboard (BFCL) and real-world reliability in structured tool-use tasks. Mistral's Large and Mixtral models also support native tool calling but trail slightly on accuracy for nested or parallel calls.

None of these fully match Claude or GPT-4-class models on complex multi-step tool chains yet, but the gap has narrowed a lot over the past year. Which one is "best" for you depends less on a leaderboard rank and more on three practical things: how reliably the model emits valid JSON under your exact schema, whether it can chain multiple tool calls in one turn, and what you're willing to run locally versus pay for as a hosted API.

What "best" actually means for function calling

Benchmarks measure accuracy on curated tasks, but production function calling breaks in specific, boring ways. When evaluating an open source LLM for this, check:

• Schema adherence — does it reliably produce valid JSON matching your parameter types, or does it hallucinate extra fields?
• Parallel and nested calls — can it call two tools in one response, or chain a tool result into a second call without you re-prompting?
• Refusal behavior — does it call a tool when it shouldn't, or skip calling one when it should?
• Format stability under fine-tuning — some base models regress on tool calling after domain fine-tuning, which matters if you plan to customize the model.
• Context handling — how does accuracy degrade as you add more tool definitions to the system prompt?

A model that scores well on BFCL but falls apart with 15 tool definitions in context isn't "best" for an agent with a large toolbox.

Top open source models for function calling

Qwen2.5 (Alibaba)

Qwen2.5-72B-Instruct and the smaller 32B/14B variants have consistently ranked near the top of open source function-calling benchmarks. They support native tool-call formatting, handle parallel calls reasonably well, and the smaller variants are usable on a single high-memory GPU, which matters if you're self-hosting.

Llama 3.3-70B and Llama 3.1 (Meta)

Meta added structured tool-calling support starting with Llama 3.1, and Llama 3.3-70B improved on it further. It's a solid default if you want broad ecosystem support — most inference servers (vLLM, TGI, Ollama) have first-class support for Llama's tool-call format.

Hermes 3 (Nous Research)

A Llama 3.1 finetune specifically tuned for agentic and tool-use behavior. It tends to be more consistent at emitting clean JSON than the base Llama models, at the cost of slightly more aggressive tool-calling (it sometimes calls a tool when a direct answer would do).

Mistral Large / Mixtral 8x22B

Mistral's models support native function calling with a well-documented format, and Mixtral's mixture-of-experts architecture keeps inference costs lower than a dense 70B model. Accuracy on complex multi-tool tasks is a step behind Qwen2.5 and Llama 3.3, but it's a reasonable middle ground for cost-sensitive deployments.

DeepSeek-V3

DeepSeek's newer models have shown strong general reasoning and are gaining tool-calling support, but ecosystem tooling (inference server support, standardized formats) is less mature than for Llama or Qwen as of this writing.

A quick comparison

| Model | Parallel calls | Self-host friendly | Notes | |---|---|---|---| | Qwen2.5-72B | Strong | Needs 2x80GB GPU or quantization | Best overall accuracy | | Llama 3.3-70B | Good | Same as above | Best ecosystem support | | Hermes 3 (8B/70B) | Good | 8B runs on one GPU | Good for smaller deployments | | Mixtral 8x22B | Moderate | Cheaper inference (MoE) | Good cost/accuracy tradeoff | | DeepSeek-V3 | Improving | Large, needs serious infra | Watch for updates |

Testing a model's tool-calling format yourself

Most open source models served through vLLM or a similar OpenAI-compatible server expose tool calling in a familiar shape:

curl https://your-inference-server/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen2.5-72b-instruct",
    "messages": [{"role": "user", "content": "What is the weather in Berlin?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }]
  }'

Run the same schema through two or three candidate models with your real tool definitions — not toy examples — and check the raw output for malformed JSON, dropped required fields, or unnecessary tool calls. This will tell you more than any leaderboard.

When self-hosting isn't worth it

Open source models give you control and no per-token markup beyond your compute cost, but self-hosting a 70B-class model for reliable tool calling means GPU infrastructure, load balancing, and monitoring — real ongoing work. If your team's actual need is "reliable structured tool calling behind an API key" rather than "control over model weights," a hosted API of a closed model is often faster to ship and more consistent in production.

Claude, for example, handles multi-step and parallel tool calls with strong schema adherence out of the box, and SubToAPI turns an existing Claude subscription into a standard HTTPS API — application keys, streaming, and full tool-use support with no separate model hosting or GPU management. You define tools the same way, get structured JSON back, and skip the infrastructure question entirely. Check the tool use docs or quickstart if you want to compare it directly against a self-hosted setup.

For teams that need to fine-tune or run fully on-premises, an open source model is still the right call — you just have to budget for the infrastructure that comes with it.

questions

Is Llama or Qwen better for function calling? Qwen2.5 generally edges out Llama 3.3 on BFCL accuracy, especially for parallel and nested tool calls, but Llama has broader inference-server support and community tooling. Test both against your actual schemas before deciding.

Can small open source models (7B–14B) do reliable function calling? Yes, for simple single-tool tasks. Models like Qwen2.5-14B or Hermes 3-8B handle basic tool calls well, but accuracy drops noticeably with multiple tools, nested calls, or long context — expect more validation and retry logic in your pipeline.

Do I need to self-host to use these models for function calling? No — providers like Together, Fireworks, and Groq host Qwen, Llama, and Mixtral with OpenAI-compatible tool-calling APIs, so you get the open source model's behavior without managing GPUs yourself.