What is function calling (tool use) in LLMs? How models call your code, structured outputs, and building a reliable tool-using agent

What is function calling in an LLM?

Function calling, also called tool use, is a mechanism where you describe a set of functions to a model, name, description, and a JSON schema of its arguments, and the model, instead of only returning prose, can return a structured request to call one of those functions with specific argument values. Your application code receives that request, validates it, actually runs the function, and optionally feeds the result back to the model to continue. The model never executes code itself; it only ever proposes what to call and with what [1][2].

Also asked as: what is function calling · what is tool use in llm · function calling explained · how does function calling work · llm function calling meaning · what is tool calling · function calling vs tool calling · openai function calling explained

Every agentic system I have built, and every document-extraction pipeline, depends on this mechanism in one of its two disguises: as a way to let a model act, or as a way to force a model's output into a shape your code can trust.

A model that calls a function has not done anything yet. It has asked you to do something, in a format your code can check before it runs. Pranjul Rathour

Is tool calling the same thing as function calling?

Yes, in current usage they are the same mechanism under two names: OpenAI and Google call it function calling [1][3], Anthropic calls the equivalent capability tool use [2], and both describe the same shape, a schema-described capability the model can request to invoke. Some documentation reserves "tool" for a slightly broader idea that includes non-function capabilities such as web search or code execution provided by the platform itself, with "function calling" specifically meaning your own custom functions, but in practice the terms are used interchangeably almost everywhere.

Also asked as: is tool calling the same as function calling · function calling vs tool use · difference between tools and functions llm · openai tools vs functions

How does function calling actually work, step by step?

You send the model a prompt plus a list of function definitions, each with a name, a description, and a JSON Schema for its parameters. The model, based on the conversation, decides whether calling one of those functions would help, and if so, returns a structured object naming the function and its argument values instead of, or alongside, ordinary text. Your code parses that object, validates the arguments against your own rules, executes the real function, and sends the result back to the model as a new message so it can continue the conversation or produce a final answer.

Also asked as: how function calling works step by step · function calling flow · function calling architecture · how does an llm decide to call a function · function calling lifecycle

What is structured output, and how does it relate to function calling?

Structured output is function calling used for a single purpose: forcing the model's response into a schema you define, rather than letting it call an arbitrary tool. You define one "function" that represents the shape of the answer you want, a set of extracted fields, a classification, a form, and the model is constrained to fill it in rather than write free text [8]. This is the same JSON-Schema-and-validation mechanism as tool use; it is simply used to make model output reliably parseable rather than to trigger an action.

Also asked as: what is structured output llm · structured output vs function calling · json mode llm · how to force llm to return json · structured extraction with function calling · openai structured outputs

How does multi-step tool use turn a chatbot into an agent?

A single function call answers one question. An agent repeats the loop: call a tool, read the result, decide whether the goal is met or another tool call is needed, and continue until it is, or until a step limit is hit. This pattern, reason then act then observe, was formalised as ReAct [5] and is what separates a simple assistant from something that can, say, search documentation, run a calculation, and check its own answer before replying. The Model Context Protocol standardises how tools are described and connected across different agent implementations [6].

Also asked as: how do ai agents use tools · multi step function calling · react pattern llm · agent tool use loop · how do agents decide when to stop · agentic function calling

My longer explanation of agent architectures generally is on the AI agent page [9], and how tool descriptions get standardised across systems is on the MCP page [10].

How do I make function calling reliable in production?

Validate every proposed function call against a strict schema and your own business rules before running anything; never assume the model's arguments are safe just because they parsed. Set timeouts and retries around the actual function execution, not just the model call. Give the model a clear error message back when a call fails or arguments are invalid, so it can retry sensibly rather than looping blindly. Log every call, its arguments, and its result, so a bad outcome can be traced to exactly what was requested and what ran. Never let a model-proposed call touch anything destructive, a delete, a payment, a send, without an explicit confirmation boundary outside the model's control.

Also asked as: how to make function calling reliable · function calling best practices · validating llm function calls · function calling security · safe function calling production · llm tool use guardrails

My notes on project structure put the provider, the tool definitions, and the execution layer in separate modules for exactly this reason: validation and logging belong at the boundary, not scattered through the code [11].

What goes wrong with function calling in practice?

The model hallucinates a plausible-looking call with arguments that do not correspond to anything real, an order ID that does not exist, a date outside range; this is caught only by validation, never by the model itself. A tool description is ambiguous, and the model picks the wrong function among several similar ones. A multi-step loop runs without a step limit and burns cost on an unproductive path. Errors from a failed call are not surfaced back to the model in a useful form, so it repeats the same failing call. Every one of these is a process failure, not a fundamental limitation, and every one has a known fix.

Also asked as: function calling failures · llm hallucinating function arguments · agent tool use problems · function calling debugging · why does my agent call the wrong tool · function calling common bugs

Function calling interview questions

Explain what happens, step by step, when a model calls a function. Explain the difference between the model proposing a call and your code executing it. Describe how you would validate a proposed function call before running it. Explain how structured outputs reuse the same mechanism as tool use. Describe how a multi-step agent loop terminates. The strongest answer names a specific validation rule you added after something went wrong.

Also asked as: function calling interview questions · tool use interview questions llm · ai agent interview questions · structured output interview

Where should I start?

Define one simple function, a calculator or a lookup against a small local list, give it to a model with a strict schema, and make it call the function for a question that needs it. Log the full call and result. That one afternoon teaches the whole mechanism. For a hands-on session on building reliable tool-using agents, email pranjulrathour41@gmail.com or use pranjulrathour.scult.in/invite.

Pranjul Rathour

GenAI Engineer · Kanpur, Uttar Pradesh, India

GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).

Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.

PortfolioInvite me to your campusHire meLinkedInGitHubXInstagramThreadsFacebookBlueskyDev.toBloggerHashnodeSCULTEmail