Ollama Best Function Calling Model: A Real Comparison
If you're running models locally with Ollama and need tool use to actually work — not just parse a JSON schema occasionally — the short answer is: Qwen2.5 (7B–32B) and Llama 3.1 8B/70B are currently the most reliable general-purpose choices, with Firefunction-v2 and Hermes-2-Pro as strong specialists when you need dedicated tool-calling behavior over broad reasoning.
This isn't a settled question the way it might be for hosted APIs. Ollama exposes a tools field in its /api/chat endpoint, but how well a model actually respects that schema, avoids hallucinating arguments, and knows when not to call a tool varies a lot between model families and even between quantizations of the same model. Below is what actually holds up in practice, plus how to test it yourself before you commit to a model.
What "function calling" means in Ollama
Ollama's chat API accepts a tools array (name, description, JSON schema for parameters) alongside your messages. The model is supposed to respond with a tool_calls field instead of plain text when it decides a function should run. This only works with models that were fine-tuned to understand that format — base models and older Llama 2-era models will mostly ignore it or hallucinate free-text pseudo-calls.
A basic request looks like this:
curl http://localhost:11434/api/chat -d '{
"model": "qwen2.5:14b",
"messages": [
{ "role": "user", "content": "What is the weather in Lisbon right now?" }
],
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city",
"parameters": {
"type": "object",
"properties": {
"city": { "type": "string" }
},
"required": ["city"]
}
}
}
],
"stream": false
}'A good model returns tool_calls with {"city": "Lisbon"} populated correctly. A weak one either answers in prose, invents a nonexistent function, or fills the schema with garbage.
The models that actually work
Qwen2.5 (7B, 14B, 32B)
Alibaba's Qwen2.5 line is currently the strongest open-weight family for structured tool use at reasonable local sizes. It sticks to schemas closely, handles multi-step tool chains (call → observe result → call again) better than most Llama-based models, and the 14B version in particular is a good balance of speed and reliability on a single consumer GPU.
Llama 3.1 8B and 70B
Meta explicitly trained Llama 3.1 for tool calling, and it shows — the 8B model is usable for simple single-tool tasks, while 70B is noticeably more reliable at picking the correct tool when multiple options are available, and at not calling a tool when a direct answer would do. If you have the VRAM, 70B is the safer bet for anything agentic.
Firefunction-v2
Built by Fireworks AI specifically for function calling, this model is narrower in general capability but very consistent at schema adherence. If your use case is almost entirely tool orchestration (routing requests to APIs, not open-ended chat), it's worth benchmarking against Qwen2.5 for your specific schemas.
Hermes-2-Pro-Llama-3
Nous Research's fine-tune adds a dedicated function-calling format on top of Llama 3, and it performs well on parallel tool calls (asking for two functions in one turn). It's a good fit if your agent needs to fan out multiple actions from a single user request.
Mistral Nemo / Mistral Small
Mistral's models support tool calling reasonably well and run fast, but in practice they're more prone to calling a tool when a plain-text answer was correct, which adds latency and cost if you're chaining calls. Usable, but I'd rank it behind Qwen2.5 and Llama 3.1 for anything with more than one tool defined.
Quick comparison
| Model | Best for | Weak point | |---|---|---| | Qwen2.5 14B/32B | General tool use, multi-step chains | Needs decent VRAM at 32B | | Llama 3.1 70B | High-stakes tool routing accuracy | Heavy to run locally | | Firefunction-v2 | Pure function-calling pipelines | Narrower general reasoning | | Hermes-2-Pro | Parallel tool calls | Less mature ecosystem support | | Mistral Nemo | Speed | Over-calls tools unnecessarily |
How to actually test this for your case
Benchmarks and leaderboards are a starting point, not a substitute for testing your own schemas. Run the same 10–15 realistic prompts against 2–3 candidate models with stream: false and check:
- Does it call the right tool, not just a tool?
- Are argument types correct (string vs number, required fields present)?
- Does it correctly abstain from calling a tool when the answer doesn't need one?
- What happens on a second turn after a tool result is fed back in?
That last point is where most local models fall apart — the first call often looks fine, but continuing a multi-turn tool loop reliably is much harder, and it's usually the deciding factor in production.
When local models aren't reliable enough
Local function calling is great for prototyping and for workloads where occasional schema drift is tolerable. For production agents — customer-facing tools, billing actions, anything where a wrong argument has a real cost — the reliability gap between local open models and Claude's tool use is still significant, particularly on longer multi-step tool loops.
If you're already prototyping locally with Ollama but need that production-grade reliability without building your own Claude integration from scratch, SubToAPI turns your existing Claude access into a standard HTTPS API with application keys, streaming, and full tool-use support — see /docs/tools for the request format, which is close enough to what you're already sending Ollama that migrating a prototype over is usually a small diff. Plans start at Solo €9, with a free trial at /signup.
questions
Is there one single "best" Ollama model for function calling? No — Qwen2.5 (14B/32B) and Llama 3.1 (8B/70B) are the strongest general picks today, but the right choice depends on your VRAM budget and whether you need single or parallel tool calls.
Do all Ollama models support the tools field? No. Only models specifically fine-tuned for it (Llama 3.1, Qwen2.5, Hermes-2-Pro, Firefunction-v2, Mistral's newer releases) reliably use it — older or base models will ignore the schema or respond in plain text.
Why does my local model call tools when it shouldn't? Smaller and less specialized models (notably some Mistral variants) tend to over-trigger tool calls because they weren't trained with strong enough negative examples of when not to call a function — testing with abstain-cases in your prompt set helps catch this before deployment.