Tool use and function calling: Question bank — AI-ML deep dive

Before we start, two words we lean on constantly:

LM (language model) — the text-generating AI (e.g. GPT-4, Claude); throughout this page "LM" and "the model" mean the same thing.
Tool call — just a chunk of structured text (usually JSON) that the LM writes to say "please run this function with these arguments." It is a request, not an execution. Hold that distinction — half the traps below hinge on it.

And one loop we reference repeatedly:

ReAct (Reason + Act) — an agentic pattern where the LM alternates a reasoning step and an acting (tool-call) step, feeding each tool result back in before deciding the next move, until the task is done.

True or false — justify

The LM directly executes the tools it calls.

False — the LM only emits a JSON function call as text; your system parses, validates, and runs the real code. The LM never touches the API or the machine.

Function calling means the LM has been given internet access.

False — it has been given a description of tools. Whether any tool actually reaches the internet depends entirely on what code you wire behind the tool name.

If the LM outputs a tool call, the task must genuinely need that tool.

False — models sometimes call tools spuriously or hallucinate that a tool is needed; tool_choice="auto" lets it decide, and that decision can be wrong.

JSON Schema is used mainly because it looks tidy.

False — it is used because it is a machine-readable contract: types, enum values, and required fields let your system validate arguments before executing anything.

Once a tool returns a result, the conversation is over.

False — the result is injected back as a function/tool message and the LM must generate the final natural-language answer (or decide to call another tool) from it.

Giving the LM more tools always makes it more capable.

False — tool-selection accuracy tends to drop as the tool count grows, because picking the right tool becomes a harder multi-way classification problem. More options ≠ better choices.

Parallel function calling means the LM runs several tools at once.

False — the LM only proposes several independent calls in one turn; your executor decides to run them concurrently. The LM still does no execution.

A well-written description field is optional polish.

False — the description is the primary signal the LM uses to pick the right tool and fill parameters correctly. Weak descriptions are a leading cause of wrong tool selection.

Forcing tool_choice="required" guarantees correct structured output.

False — it only guarantees the LM calls a tool; the arguments can still be malformed or wrong. You must still validate against the schema (see 5.2.03-JSON-mode-and-structured-outputs).

The arguments field arrives as a ready-to-use dictionary.

False — it typically arrives as a JSON string that you must json.loads and then validate; treating it as pre-parsed is a common bug.

Any extra fields the LM adds to the arguments object are automatically ignored.

False — whether extras are ignored or rejected depends on your schema's additionalProperties setting; the LM may hallucinate stray fields, so you must decide the policy explicitly.

Spot the error

"We exposed an execute_python tool so the agent can do anything — the LM would never write harmful code."

The error is trusting the LM's intent. Prompt injection can make it emit rm -rf /, and the LM has no real security model. Fix: allowlist operations and sandbox execution.

"The API timed out, so we let the exception bubble up and crash the agent."

The error is having no timeout/retry handling. Instead, catch the failure and inject {"error": "..."} as a tool result so the LM can retry or degrade gracefully.

"To be safe we validate the tool name the LM returned, then run it."

Validating the name is not enough — you must also validate the arguments against the schema. A valid name with hallucinated or malicious parameters is still dangerous.

"Our agent hangs forever on complex tasks, but that's fine — it'll finish eventually."

The error is no loop bound. The agentic ReAct (Reason + Act) loop needs a max-iteration cap; models can loop indefinitely re-calling tools without converging.

"We fine-tuned the model on function calls, so it will never call a tool that doesn't exist."

Models can still hallucinate tool names or parameters, especially under many tools or ambiguous prompts. Always reject unknown names at the executor.

"Weather is dynamic, so we let the LM answer weather from its training data to save an API call."

Training data is stale — that is precisely the case where a tool is mandatory. Skipping the call reintroduces the hallucination problem tool use was meant to solve.

"The LM sent valid JSON last time, so we skip the parse-error check and go straight to execution."

The error is assuming JSON is always well-formed. A malformed arguments string will throw on json.loads; catch it and inject an error result instead of crashing.

"We put all 50 microservice endpoints in as flat tools; the LM has full power now."

This overloads tool selection and tanks accuracy. Fix: hierarchical grouping and dynamic filtering to show only task-relevant tools.

Why questions

Why does the LM write a call instead of executing the function itself?

Because the LM is a text generator with no runtime; separating proposal (LM) from execution (your system) is what gives you control and prevents arbitrary code execution.

Why is JSON Schema preferred over free-form text for tool definitions?

Because it gives an explicit, validatable contract — types, enums, required, additionalProperties, and defaults — so your code can reject or normalize bad arguments before running anything.

Why does the ReAct (Reason + Act) loop enable harder tasks than single-turn calling?

Because it lets the LM chain tools — observe a result, reason, then act again (search → read → compute) — so it can decompose problems it cannot solve in one pass. See 6.2.02-Agent-architectures-and-planning.

Why do tool results get injected back into the context instead of just being returned to the user?

Because the raw result (e.g. {"temp":18}) still needs the LM to interpret and phrase it; and later steps may depend on it, so it must live in the conversation history.

Why does "the model wouldn't do that" fail as a security argument?

Because prompt injection can steer outputs, and the LM has no understanding of security consequences — it optimizes for plausible completions, not safety.

Why do good tool descriptions matter more for large tool sets?

Because with many candidates the LM is effectively doing multi-way classification from descriptions; ambiguous or overlapping descriptions cause misrouting.

Why is retrieval (RAG) sometimes framed as "just another tool"?

Because a retrieval endpoint fits the same call → execute → inject-result loop; the LM calls search(query) and reasons over returned passages (see 7.1.01-RAG-fundamentals).

Why can clearer prompting reduce unnecessary tool calls?

Because instructions can tell the LM when a tool is warranted vs. when to answer directly, shrinking spurious calls — a 4.3.02-Prompt-engineering-techniques concern.

Edge cases

What should happen when the user query needs no tool at all?

With tool_choice="auto" the LM should answer directly and emit no function call; forcing a call here ("required") would produce an unnecessary or nonsensical invocation.

What if the LM calls a tool whose name isn't in your registry?

The executor should reject it and inject an error result, never attempt a fuzzy match — silently guessing which tool was meant invites wrong actions.

What if a required parameter is missing from the arguments?

Schema validation should fail before execution; inject a clear error back so the LM can re-emit a corrected call rather than running with a bad/default value.

What if the arguments string is malformed JSON (e.g. a trailing comma or unclosed brace)?

json.loads will raise; catch that, and inject a tool result like {"error": "invalid JSON in arguments"} so the LM can regenerate a well-formed call — never let the parse exception crash the loop.

What if the LM adds extra fields not declared in the schema?

It depends on additionalProperties: set to false, validation rejects the call (inject an error so it retries); left permissive, the extras are ignored but you should still log them, since they signal a confused model.

What if a parameter has a default defined in the schema and the LM omits it?

Prefer to inject the default rather than reject — the schema's default is the intended fallback, so fill it in and proceed; only reject if a required field (which has no default) is missing.

What if two parallel tool calls actually depend on each other?

They can't safely run in parallel — the LM mis-modeled the dependency. Your executor (or agent logic) must sequence dependent calls; parallelism is only valid for independent tools.

What if a tool streams its result or returns a partial payload?

Do not inject half-finished fragments as the observation — buffer the stream to completion (or inject a clearly-marked partial like {"partial": true, ...}) so the LM reasons over a coherent, complete result rather than a truncated one.

What happens at the very first turn with zero prior context?

The LM sees only the user query plus tool schemas; if schemas are missing it cannot call anything and falls back to plain text — no tools exposed means no tool use, full stop.

What if a tool returns an empty result or null?

That is a valid observation, not a crash — inject it as-is so the LM can reason ("no results found") and possibly try a different tool or ask the user.

What if the same tool is called repeatedly with identical arguments in a loop?

This signals non-convergence; a max-iteration cap plus optional call-deduplication should break the loop and return a graceful "unable to complete" answer.

What is the limiting behavior as the number of exposed tools → very large?

Tool-selection accuracy degrades toward chance-like guessing; the practical fix is to keep the active tool set small via filtering, regardless of how many tools exist in total.

Recall

Cover the answers and self-test. The LM executes tools. ::: False — it only writes the call; your system executes. More tools → better performance. ::: False — selection accuracy falls; keep active sets small. arguments is a parsed dict. ::: False — it's a JSON string you must parse and validate. Malformed JSON in arguments should crash the agent. ::: False — catch it and inject an error so the LM retries. A schema default means reject when omitted. ::: False — inject the default and proceed.