Local LLM Tool Calling with Ollama and Python: Step-by-Step Implementation

Running Large Language Models locally has evolved from basic chat interfaces into sophisticated autonomous systems capable of executing deterministic software tools. With modern models like Llama 3.1, Qwen 2.5, and Mistral supporting native tool calling, developers can build private, zero-token-cost AI workflows that query internal databases, inspect system logs, and trigger automated tasks directly from Python.

In this deep guide, we implement production-grade Tool Calling with Ollama and Python. We examine tool schema definitions, multi-turn tool dispatching, handling structured responses, and building guardrails against model hallucinations.

1. Architectural Workflow: Local Tool Calling Loop

┌────────────────────────────────────────────────────────┐
│                   Inbound User Prompt                  │
│       "What is the CPU usage and free disk space?"     │
└───────────────────────────┬────────────────────────────┘
                            │
                            ▼
┌────────────────────────────────────────────────────────┐
│      Ollama Local Model (e.g. qwen2.5-coder / llama3.1) │
│       Evaluates Prompt + Tool Schemas (JSON Schema)    │
└───────────────────────────┬────────────────────────────┘
                            │ Model outputs tool_calls:
                            │ [{"name": "get_system_telemetry", "args": {}}]
                            ▼
┌────────────────────────────────────────────────────────┐
│          Python Dispatcher / Execution Engine          │
│          Executes registered Python function           │
└───────────────────────────┬────────────────────────────┘
                            │ Returns Tool Output Payload:
                            │ {"cpu_pct": 14.2, "disk_free_gb": 128}
                            ▼
┌────────────────────────────────────────────────────────┐
│    Final Inference Pass: Model synthesizes answer      │
│  "The current CPU load is 14.2% with 128GB disk free." │
└────────────────────────────────────────────────────────┘

2. Environment Setup

Ensure you have Ollama installed locally (ollama --version). Then install the official Python client:

pip install ollama>=0.3.0 pydantic>=2.7.0 psutil>=5.9.0

Make sure your target model is pulled and running:

ollama pull llama3.1:8b
# or for coding: ollama pull qwen2.5-coder:7b

3. Production Python Implementation: Tool Calling Engine

Create ollama_tools_engine.py:

import json
import psutil
from typing import Dict, Any, List, Callable
import ollama

# 1. Define Concrete Tools
def get_system_telemetry() -> Dict[str, Any]:
    # Returns real-time host CPU, RAM, and disk utilization.
    cpu = psutil.cpu_percent(interval=0.5)
    mem = psutil.virtual_memory()
    disk = psutil.disk_usage('/')
    return {
        "cpu_percent": cpu,
        "ram_used_percent": mem.percent,
        "ram_free_gb": round(mem.available / (1024**3), 2),
        "disk_free_gb": round(disk.free / (1024**3), 2)
    }

def calculate_compound_interest(principal: float, rate_pct: float, years: int) -> Dict[str, Any]:
    # Calculates compound interest over a specified duration.
    amount = principal * ((1 + (rate_pct / 100)) ** years)
    return {
        "principal": principal,
        "rate_pct": rate_pct,
        "years": years,
        "final_amount": round(amount, 2),
        "interest_earned": round(amount - principal, 2)
    }

# 2. Map Functions to Dispatch Table
TOOL_REGISTRY: Dict[str, Callable] = {
    "get_system_telemetry": get_system_telemetry,
    "calculate_compound_interest": calculate_compound_interest
}

# 3. Model Tool Declarations (Ollama JSON Schema)
TOOLS_SCHEMA = [
    {
        "type": "function",
        "function": {
            "name": "get_system_telemetry",
            "description": "Inspects local host CPU, RAM, and storage utilization metrics.",
            "parameters": {
                "type": "object",
                "properties": {},
                "required": []
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate_compound_interest",
            "description": "Calculates future investment value using annual compound interest formula.",
            "parameters": {
                "type": "object",
                "properties": {
                    "principal": {"type": "number", "description": "Initial deposit amount in USD"},
                    "rate_pct": {"type": "number", "description": "Annual interest rate percentage (e.g. 7.5)"},
                    "years": {"type": "integer", "description": "Investment timeframe in whole years"}
                },
                "required": ["principal", "rate_pct", "years"]
            }
        }
    }
]

# 4. Orchestration Loop
def run_agentic_turn(user_prompt: str, model_name: str = "llama3.1:8b"):
    messages = [{"role": "user", "content": user_prompt}]
    print(f"\n[User]: {user_prompt}")

    # Pass tool definitions into model inference
    response = ollama.chat(
        model=model_name,
        messages=messages,
        tools=TOOLS_SCHEMA
    )

    messages.append(response["message"])

    # Check if the model triggered any tool calls
    tool_calls = response["message"].get("tool_calls", [])
    if not tool_calls:
        print(f"[Model]: {response['message']['content']}")
        return

    for call in tool_calls:
        fn_name = call["function"]["name"]
        fn_args = call["function"].get("arguments", {})
        print(f"[*] Model requesting tool call: {fn_name}({fn_args})")

        if fn_name in TOOL_REGISTRY:
            try:
                fn_result = TOOL_REGISTRY[fn_name](**fn_args)
                # Feed tool execution output back to model
                messages.append({
                    "role": "tool",
                    "content": json.dumps(fn_result)
                })
            except Exception as e:
                messages.append({
                    "role": "tool",
                    "content": json.dumps({"error": str(e)})
                })

    # Final completion pass with tool context injected
    final_response = ollama.chat(model=model_name, messages=messages)
    print(f"[Model]: {final_response['message']['content']}")

if __name__ == "__main__":
    run_agentic_turn("Can you inspect my machine telemetry and tell me how much RAM and disk space are free?")

4. Troubleshooting & Best Practices

5. Frequently Asked Questions (FAQ)

Can I run Ollama tool calling offline without internet?

Yes! Ollama runs 100% locally on your machine. No data leaves your hardware, making it ideal for private enterprise databases and sensitive code inspection.

Which model is best for tool calling in 2026?

llama3.1:8b and qwen2.5-coder:7b provide the highest reliability and parameter adherence for local function execution.