Gemini API Tool Retrieval: how tool_search and defer_loading work, what they cost, and how they compare to Claude
Gemini API Tool Retrieval, in preview since November 19, lets agents defer tool schemas with defer_loading and load them through a tool_search tool. Setup, client-side mode, the token math, common 400 errors, and a comparison with Claude's tool search tool.
Google put Tool Retrieval into preview in the Gemini API on November 19, 2026. It is meant for agents that carry a large tool library: instead of sending every function's full parameter schema on every request, you mark tools with defer_loading: true, add a tool_search tool, and Gemini loads the schemas it needs while it handles the request. The Gemini sections below are based on Google's announcement and the AI Studio documentation. The comparison section uses Anthropic's tool search documentation.
If you have used Anthropic's tool search tool on the Claude API, the design will look familiar, since it uses the same defer_loading flag. The details differ. This guide covers how the Gemini version works, what it costs, the errors it throws, and a side-by-side with Claude's version for teams that run both. The feature is a v1beta preview, so expect the API surface to change before general availability.
Quick facts
- Status: preview. It needs the v1beta API version, and Google says the API may change before general availability.
- Where: the Interactions API only. generateContent does not support it.
- Models listed in the docs: gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash and gemini-3.6-flash. The launch post says Gemini 3.6 and later, including all Gemini 4 models.
- Price: no separate charge. Normal input, cached-input and output rates apply, plus the extra tokens described under pricing below.
- Modes: server-side search (the default) or client-side search against your own registry. You can't mix the two in one request, and only one tool_search is allowed per request.
- MCP: set defer_loading: true on an mcp_server tool and every tool that server exposes is loaded on demand.
How server-side retrieval works
You add {"type": "tool_search"} to the tools array and set defer_loading: true on the functions you want to hold back. The model still sees the name and description of every deferred tool, but their parameter schemas stay out of context. When it needs one, it asks for it by name, the API loads the schema, and the model then emits a normal function_call, all in the same API call. This example is adapted from Google's launch post:
from google import genai
client = genai.Client()
tools = [
{"type": "tool_search"},
{
"type": "function",
"name": "issue_refund",
"description": "Refund an order to the original payment method.",
"defer_loading": True,
"parameters": {
"type": "object",
"properties": {
"order_id": {"type": "string"},
"amount": {"type": "number"},
},
"required": ["order_id", "amount"],
},
},
# ... 50+ other deferred functions (leave frequent tools undeferred)
]
interaction = client.interactions.create(
model="gemini-4-hydrogen",
input="Order A-1042 arrived damaged. Please refund 40 dollars.",
tools=tools,
)The response's steps array shows three entries: a tool_search_call naming issue_refund, a tool_search_result carrying its schema, and the function_call with the arguments. From there it is ordinary function calling: your code runs the refund and sends back a function_result. Google says to look for function_call entries in steps rather than relying on the interaction's status.
Two details are easy to miss. You still send the full tools array, deferred schemas included, on every request, because the API does not carry tool declarations between requests. And if you run stateless with store=false instead of previous_interaction_id, you must send the tool_search_call and tool_search_result steps back unchanged, signature values included, in their original order.
Client-side retrieval for your own registry
If you already keep a tool registry, an embedding index or per-user permission filters, declare tool_search with "execution": "client" and give it a name, a description and a parameters schema. The model then calls it like a function, your app runs the lookup, and you return matching function declarations in a function_result. That adds one round trip per search, which is why Google recommends server-side search for latency-sensitive apps. In exchange, tools don't have to be declared upfront, so you can expose new ones mid-conversation when a user connects an integration. defer_loading can't be used in client mode.
What it actually costs
There is no line item for Tool Retrieval, but there are several token effects, all from Google's documentation:
- Each deferred tool still adds about one line of input (its name and description), roughly 26 tokens per tool in Google's early testing. Parameter schemas aren't billed until they are loaded.
- Turning on server-side tool_search adds a small fixed block of instruction tokens.
- Each server-side search runs another model pass inside the request. It is billed in total_tool_use_tokens and costs about as much input as the prompt again, minus cached tokens.
- Loaded schemas stay in context and are billed on every later turn. They can't be unloaded.
That extra search pass is the main cost to weigh against the schemas you skip. A rough break-even check, inferred from Google's cost description rather than stated by Google: tokens saved per request ≈ (schema tokens you never load) − (deferred tools × about 26) − (searches per request × uncached prompt tokens) − the fixed tool_search block. Take 300 deferred tools as an example. Their name lines alone add about 7,800 tokens per request, so the schemas you skip must outweigh that plus every search pass. Plug in your own measured schema and prompt sizes. Google says retrieval can cost more with a small library or with a long uncached context that triggers frequent searches. It recommends plain static loading for fewer than about 20 stable tools and suggests measuring against your own traffic.
Google says Tool Retrieval plays well with prefix caching. Deferred schemas stay out of the start of the prompt, and loaded ones are appended later in the conversation, so the cached prefix doesn't change from turn to turn.
Errors and gotchas
- 400 if you set defer_loading without a tool_search tool, or set short_description (an optional compact summary for deferred functions) without defer_loading.
- 400 for duplicate or mixed tool_search definitions, a client-side search without name or description, or an unsupported model.
- 400 in stateless requests if retrieval steps or their signatures are missing.
- 400 when the model calls a deferred function that was never loaded. Google says this usually happens because the function's name appears in system instructions or in another tool's output, so don't defer tools you mention by name.
- A search that matches nothing returns a text response, not a structured error.
- The model may run several searches in one turn.
Google's other tips: include full parameter schemas for deferred tools (a schema sent with only a name comes back empty), tell the model in system instructions when it should search, prefix similar function names by service and put the service name first in short_description, and return declarations as objects, not JSON strings.
Gemini Tool Retrieval vs Claude's tool search tool
Anthropic's tool search tool on the Claude API also uses defer_loading on tool definitions and a search tool in the tools array, and it also lets you plug in your own search. Here is where the two differ, going by the documentation we checked on November 20, 2026 (Google's launch post and docs page, Anthropic's tool search page):
- Status: Claude's tool search is listed as stable on the Claude API, Amazon Bedrock (InvokeModel, not Converse), Google Cloud and Microsoft Foundry. Gemini's is a v1beta preview, Interactions API only.
- How the model searches: Claude offers two server-side variants. Regex (tool_search_tool_regex_20251119) takes patterns of up to 200 characters, and BM25 (tool_search_tool_bm25_20251119) takes natural-language queries of up to 500 characters. Google's docs describe the model requesting tools by name and don't name a regex or BM25 variant.
- Results: Claude returns tool_reference blocks, up to 5 matches by default, and the model can set a limit from 1 to 10,000. Gemini returns a tool_search_result step with the loaded schemas.
- Scale: Claude documents a cap of 10,000 deferred tools per request. We found no cap in the Google pages we checked.
- Required setup: on Claude, at least one tool, normally the tool search tool itself, must stay non-deferred. On Gemini, setting defer_loading without a tool_search tool is a 400 error.
- Custom search: on Claude, your own tool returns tool_reference blocks and every referenced tool must still be in the tools array. On Gemini, a client-side tool_search returns full function declarations, and tools don't need to be declared upfront.
- Caching: Claude excludes deferred tools from the system-prompt prefix and rejects a deferred tool that also carries cache_control (400), so the breakpoint goes on a non-deferred tool. Gemini keeps the cached prefix unchanged by appending loaded schemas later in the conversation.
- Why bother: Anthropic's page says a typical multi-server setup can use about 55k tokens of tool definitions before any work, that tool search typically cuts this by over 85 percent by loading 3–5 tools, and that tool selection degrades past 30–50 tools.
For a cross-provider agent, one option is to keep a single tool catalog with a defer flag per tool and translate it per provider. Claude tool definitions use input_schema where Gemini uses parameters, and the search-tool entries have different shapes. On the Gemini side, each server-side search is an extra model pass, so log total_tool_use_tokens as well as prompt size.
A prompt to audit your own tool list
Try it with your AI. Give this to your coding assistant in the repository where your agent's tool definitions live. It reads and proposes first, and changes nothing until you approve. Use redacted sample requests, and review any script before it runs.
I want to try Gemini API Tool Retrieval (preview, Interactions API, v1beta) on my agent.
1. First find where the tools array is built and tell me: how many tools there are,
which SDK version, API (generateContent or Interactions) and model we call, and
whether we use previous_interaction_id or store=false. If we use generateContent,
stop and tell me, because Tool Retrieval only works on the Interactions API. The
supported models listed are gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash and
gemini-3.6-flash.
2. Estimate how many tokens the full parameter schemas take. If there are fewer than
about 20 stable tools and no per-user or registry-based tools, tell me static loading
is probably better and stop.
3. Otherwise, propose which tools to keep loaded (the most frequently called ones and
any tool named in our system instructions or returned by another tool) and which to
mark defer_loading: true. Show the list and wait for my confirmation.
4. After I confirm, add {"type": "tool_search"} and the defer_loading flags in a
separate branch, without changing any tool's name, description or schema. If we use
store=false, make sure tool_search_call and tool_search_result steps are sent back
unchanged with their signatures, in order.
5. Write a small script that sends [3 redacted sample requests I will paste] before and
after the change, and prints input tokens, total_cached_tokens,
total_tool_use_tokens, the number of searches and the latency for each. Show me the
script before running it.
Do not touch API keys or credential files, and do not delete anything.FAQ
Does Gemini Tool Retrieval cost extra? There's no separate charge, but it isn't free. Tokens are billed at the model's standard input, cached-input and output rates: about 26 tokens per deferred tool, a fixed instruction block for server-side search, loaded schemas on later turns, and each search's extra model pass in total_tool_use_tokens.
Does it work with generateContent? No. Google's documentation says it's only available on the Interactions API, in preview, on the v1beta version.
Which Gemini models support Tool Retrieval? The documentation lists gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash and gemini-3.6-flash.
Can I defer a whole MCP server? Yes. Set defer_loading: true on an mcp_server tool and every tool that server exposes is loaded on demand.
Can loaded tools be unloaded later in the conversation? No. Retrieved schemas stay loaded for the rest of the conversation and are billed as input on later turns.
Related on claudefox
- Google's Gemini Decisions API scores up to 1,000 yes/no and multiple-choice questions in one call
- Gemini 4 Neon vs Claude Opus 5.5 and Sonnet 5.5: price, per-task cost and when to switch
- How to call Gemini 4 Hydrogen from Claude Code: gemini-api CLI, Docs MCP and skills
- Gemini 4 Carbon vs Claude Opus 5.5 and Fable 5.2: benchmarks and pricing compared
Source: Google: Introducing Tool Retrieval in the Gemini API, Gemini API docs: Tool retrieval, Claude API docs: Tool search tool
This story first appeared in the claudefox daily briefing for November 20, 2026.