Key takeaways
- Function calling lets a model return a structured request to call one of your functions, which your code then executes.
- You describe functions with names, descriptions and JSON schemas; the model chooses one and fills in the arguments.
- OpenAI introduced function calling in its API in June 2023; most major providers now support it.
- Function calling is not RAG, but retrieval is often exposed to the model as a function.
- Test it with cases that check the right function, correct arguments, and refusal when no function fits.
Function calling, also called tool use, lets a language model ask your application to run a function: instead of only replying with text, the model returns a structured request such as get_order_status(order_id="4521") with arguments in JSON, your code runs the function, and the result goes back to the model to finish its answer. It is how chatbots check orders, book slots, query databases and act as agents. This page explains what it is, how it works, OpenAI's version and its history, which models support it including Gemini and Mistral, how it relates to RAG, and how to test it.
What is function calling?
Function calling is a feature of LLM APIs where you describe functions your application can perform, and the model can choose to call one by returning its name and arguments in a structured format, rather than answering in free text.
The model never runs code itself. It decides that a function would help, produces the call with arguments that match your schema, and your application executes it, then sends back the result. This turns a model into an interface to real systems: databases, APIs, calculators, search and internal tools. It is the foundation of AI agents, which call functions repeatedly in a loop to complete multi-step tasks.
What is function calling in AI?
In AI, function calling is the mechanism that connects language models to external capabilities, letting them retrieve live data, perform calculations precisely and take actions, which models cannot do from their training alone.
For example, a college assistant can answer "Is lab 3 free at 2 pm?" only if it can call a check_room(room, time) function that queries the booking system. Without function calling, it would guess. With it, the answer is accurate and current. It also helps with structured extraction: asking the model to "call" a function whose arguments are the fields you want is a reliable way to get structured data.
How does function calling work in an LLM?
You send the model the user's message plus a list of function definitions. The model either replies normally or returns a function call with arguments. Your code runs the function, adds the result to the conversation, and calls the model again so it can respond using the result.
Also asked as: how does function calling work in llm
A function definition looks like this:
{"name": "check_room", "description": "Check if a campus room is free at a given time",
"parameters": {"type": "object",
"properties": {"room": {"type": "string"}, "time": {"type": "string", "description": "24h time, e.g. 14:00"}},
"required": ["room", "time"]}}
Good names and descriptions matter most: the model chooses functions by reading them. Validate arguments before executing, and never let a model call destructive functions without checks.
What is function calling in OpenAI?
In OpenAI's API, function calling lets you pass tool definitions with JSON schemas, and the model responds with tool calls containing the function name and JSON arguments. You execute them and return outputs; a strict mode makes arguments conform exactly to your schema.
OpenAI's models can call several functions in one turn and choose between your functions and built-in tools such as web search or file search, depending on the API. Use strict schemas for reliability, keep the number of functions manageable, and write descriptions that say when to use each one. The official function calling guide shows the request and response format for the current API.
When was function calling introduced?
OpenAI introduced function calling in its API in June 2023, which made tool use mainstream for developers. Other providers, including Anthropic, Google and Mistral, added their own tool use or function calling features soon after.
Earlier, developers used prompt tricks, asking the model to output JSON and parsing it, which was fragile. Native function calling made outputs reliable and models better at choosing tools, which led directly to the growth of AI agents. Research such as ReAct and Toolformer had already explored models using tools. Today, standards such as the Model Context Protocol build on function calling to connect models with many tools consistently.
Which LLMs support function calling?
Most current major models support it: OpenAI's GPT models, Anthropic's Claude, Google's Gemini, Mistral's models, and many open-weight models such as Llama, Qwen and others, often through serving tools like Ollama and vLLM.
Quality varies. Frontier hosted models are generally most reliable at choosing the right function and producing valid arguments, especially with many tools. Smaller open models work well for a few simple functions but make more mistakes as tasks get complex. Benchmarks such as the Berkeley Function Calling Leaderboard compare models; still, test with your own functions, because results depend heavily on your schemas and descriptions.
Does Gemini support function calling?
Yes. The Gemini API supports function calling: you declare functions with names, descriptions and parameter schemas, and Gemini returns function calls for your code to execute, including parallel calls and modes that control whether calling is automatic, required or disabled.
Gemini also offers built-in tools, such as grounding with Google Search and code execution, alongside your custom functions. The Python SDK can automatically call Python functions you pass in, which is convenient for prototypes. For production, handle calls explicitly so you can validate arguments, log actions and require confirmation for anything risky.
Does Mistral support function calling?
Yes. Mistral's API supports function calling with tool definitions in JSON schema form, and several of Mistral's models, including open-weight ones, are trained for tool use.
The workflow is similar to other providers: define tools, send them with the messages, handle tool calls in the response, return results, and let the model produce the final answer. Because Mistral offers both hosted and open models, you can prototype with the API and later self-host an open model with tool support. Check Mistral's documentation for which current models support function calling and any limits.
Is function calling RAG?
No, but they work well together. RAG, retrieval-augmented generation, retrieves relevant documents and adds them to the prompt. Function calling lets the model request actions or data from your code. Retrieval is often exposed to the model as a function, such as search_documents(query).
When retrieval is a function, the model decides when to search and what to search for, which is useful in agents and multi-turn conversations. Fixed RAG pipelines, which always retrieve before answering, are simpler and more predictable for document Q&A. RAG.NextUpgrad uses a fixed pipeline with hybrid retrieval, re-ranking and a confidence gate, because predictable retrieval made answers easier to test and trust.
How do you test function calling?
Build a test set of user requests with the expected function, arguments and behaviour, then check that the model picks the right function, fills arguments correctly, asks for missing information, and does not call functions when none fit.
Also asked as: what is function calling test
What to test:
- Selection: correct function chosen among similar ones.
- Arguments: valid types and values, correct formats such as dates.
- Missing information: the model asks instead of inventing.
- No-call cases: small talk or out-of-scope requests do not trigger calls.
- Multi-step: correct sequence of calls for complex tasks.
- Safety: prompt injection in tool results does not trigger unwanted actions.
Run tests after every change to prompts, functions or models, and log real usage to find new failure cases.
Sources
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.
