Tool use and function calling

Overview

Tool use (also called function calling) is the capability that allows language models to interact with external systems, APIs, databases, and code execution environments by generating structured calls to predefined functions. Instead of merely generating text, the model can request actions in the real world.

The architecture: how tool use works

Step 1: Tool schema definition

You describe available tools to the model using a function schema—typically JSON Schema format:

{
  "name": "get_weather",
  "description": "Retrieves current weather for a location",
  "parameters": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "City name, e.g. 'San Francisco, CA'"
      },
      "units": {
        "type": "string",
        "enum": ["celsius", "fahrenheit"],
        "description": "Temperature unit"
      }
    },
    "required": ["location"]
  }
}

Why this format?

• The model needs to know: what the tool does (description), what arguments it takes (parameters), and what's required.
• JSON Schema is machine-readable and has a clear contract: types, enums, validation rules.

Step 2: Model generates function calls

When the user asks "What's the weather in Paris?", the model (given tool schemas) outputs:

{
  "function_call": {
    "name": "get_weather",
    "arguments": "{\"location\": \"Paris, France\", \"units\": \"celsius\"}"
  }
}

How does it learn to do this? During training/fine-tuning, models are shown examples of:

• User intent → correct function call
• Multi-step reasoning requiring tool use
• When to call a tool vs. answer directly

Modern models (GPT-4, Claude 3+, Gemini 1.5+) have this baked into their instruction-following.

Step 3: Your system executes the function

You parse the JSON, validate arguments, call the actual API:

def execute_tool(func_name, arguments):
    if func_name == "get_weather":
        args = json.loads(arguments)
        # Call real weather API
        return requests.get(f"https://api.weather.com?loc={args['location']}").json()

Critical safety point: The model does NOT execute code. You control what runs. This prevents arbitrary code execution.

Step 4: Inject result back into context

You return the tool result to the model:

{
  "role": "function",
  "name": "get_weather",
  "content": "{\"temp\": 18, \"condition\": \"cloudy\", \"units\": \"celsius\"}"
}

The model then generates the final user-facing response:

"It's currently 18°C and cloudy in Paris."

Why this matters: from chatbot to agent

Without tool use

• Model can only answer from training data (stale, no personal info)
• Hallucinations about facts it doesn't know
• Can't take action

With tool use

• Real-time data: current weather, stock prices, user's calendar
• Personalization: query user's database, access preferences
• Actions: book meetings, send emails, run code
• Verifiability: calculations done by code, not model's arithmetic

User: "I need to fly from NYC to Tokyo next Tuesday, what's the weather and my budget?"

Step 1: Model calls get_flight_info(origin="NYC", dest="Tokyo", date="2026-07-08") → Result: {"price": 850, "duration": "14h"}

Step 2: Model calls get_weather(location="Tokyo", date="2026-07-08") → Result: {"temp": 28, "condition": "sunny"}

Step 3: Model calls check_user_budget(category="travel") → Result: {"remaining": 1200}

Final response: "The flight costs $850 (14h), leaving you$350 in your travel budget. Tokyo will be sunny and28°C—pack light!"

Why each step?

• Flight info: Need actual pricing, can't guess
• Weather: Real-time forecast, not training data
• Budget: Private user data, must query database

Implementation patterns

Pattern 1: Single-turn function calling

Model decides in one pass whether to call a tool:

response = llm.chat(
    messages=[{"role": "user", "content": "What's 2^128?"}],
    tools=[calculator_tool],
    tool_choice="auto"  # Model decides
)
 
if response.tool_calls:
    result = execute(response.tool_calls[0])
    final = llm.chat(
        messages=[...previous, {"role": "function", "content": result}]
    )

Tradeoff: Fast, but can't do multi-step reasoning.

Pattern 2: Agentic loop (ReAct pattern)

Model repeatedly reasons and acts until task complete:

while not done:
    response = llm.chat(messages, tools=all_tools)
    if response.is_finalanswer:
        return response.content
    
    for tool_call in response.tool_calls:
        result = execute(tool_call)
        messages.append({"role": "function", "content": result})

Why this works: Model can chain tools (search → read docs → write code), similar to ReAct (Reason + Act) prompting.

$E[\text{tool calls}] = \frac{\text{task complexity}}{p(\text{direct answer})} \cdot p(\text{tool needed})$

For simple queries (weather): ~1 call. Complex (research + analysis): 5-10 calls.

Derivation: If $p_d$ is probability the model can answer directly, and $c$ is average complexity requiring tools, then the expected number of tool invocations before task completion is approximately $c \cdot (1 - p_d) / p_s$, where $p_s$ is success rate per tool call. In practice, models require multiple attempts for complex decomposition.

Common mistakes and fixes

❌ Bad:

{"function": "execute_python", "args": {"code": "import os; os.system('rm -rf /')"}}

Why it feels right: Model generated it, must be safe.

The fix:

1. Never expose raw code execution as a tool
2. Use allowlisted operations: {"function": "calculate", "args": {"expression": "2+2"}}
3. Sandbox any execution (containers, timeouts, resource limits)

Steel-man: The intuition that "the model wouldn't do that" fails because:

• Prompt injection can make it output malicious calls
• Models don't understand security implications
• One bad function schema = full compromise

❌ Bad: Model calls web_search, API times out, entire agent hangs.

Why it feels right: Tools should just work.

The fix:

def execute_tool_safe(call, timeout=10, retries=2):
    for attempt in range(retries):
        try:
            return func_map[call.name](**call.args, timeout=timeout)
        except TimeoutError:
            if attempt == retries - 1:
                return {"error": "Tool timeout"}

Then inject the error as a function result—let the model retry or give up gracefully.

Giving the model 50 tools at once.

Why it fails: Models have limited "tool selection accuracy." With 50 options, they pick wrong tools, hallucinate parameters, or give up.

The fix:

• Hierarchical tools: search_tools.google, search_tools.wiki
• Dynamic tool filtering: Only show relevant tools per task category
• Tool descriptions matter more than you think—the model relies on description field

Empirical data (OpenAI): Accuracy with 5 tools: ~95%. With 20 tools: ~75%. With 50: ~50%.

Variants and extensions

Parallel function calling

Model can call multiple independent tools simultaneously:

{
  "tool_calls": [
    {"id": "call_1", "function": {"name": "get_weather", "arguments": "..."}},
    {"id": "call_2", "function": {"name": "get_traffic", "arguments": "..."}}
  ]
}

Execute in parallel, return both results → faster agents.

Forced function calling

Set tool_choice: "required" to force the model to always call at least one tool. Useful for structured data extraction:

# Extract structured data
response = llm.chat(
    "Extract user intent from: 'Book me a flight to Paris'",
    tools=[extract_booking_tool],
    tool_choice="required"
)
# Guaranteed JSON output

"Okay, so you know how ChatGPT can write stories and answer questions? But it can't actually do stuff like check the weather or send an email, right? It just types words.

Tool use is like giving ChatGPT a phone with apps. You tell it 'Hey, I have a weather app, calculator app, and an email app.' Now when you ask 'What's the weather?', instead of guessing, ChatGPT says 'Okay, I'm going to use the weather app' and writes down:

CALL weather_app WITH location='London'

Then you actually open the app (ChatGPT can't), get the result (like '15°C and rainy'), and tell ChatGPT. Now it can say 'It's 15 degrees and rainy in London!'

The magic is ChatGPT knows when to use an app and what to tell it. That's what we trained it to do. But we always control which apps it can use—it can't just download TikTok and start posting!"

Connections

• 6.2.01-What-are-AI-agents: Tool use is the "hands" of an agent
• 6.2.02-Agent-architectures-and-planning: ReAct and other patterns use tool calls in reasoning loops
• 4.3.02-Prompt-engineering-techniques: Tool schemas are a form of prompt engineering
• 7.1.01-RAG-fundamentals: Tools often wrap RAG systems (search, retrieve, answer)
• 5.2.03-JSON-mode-and-structured-outputs: Function calling is structured output with execution semantics

#flashcards/ai-ml

What is tool use (function calling) in LMs? :: The capability allowing LMs to generate structured JSON function calls to interact with external APIs, databases, or code execution, transforming them from text generators into agents that can take actions.

Concept Map

Hinglish (regional understanding)