← Files ElevenLabsARCHIVED FILE
skills/agents/references/client-tools.md
19.3 KB · Oct 5, 2026 · 18:06 UTC
# Client Tools
Extend your agent with custom capabilities. Tools let the agent take actions beyond just talking.
## Tool Types
| Type | Execution | Use Case |
|------|-----------|----------|
| **Webhook** | Server-side via HTTP | Database queries, API calls, secure operations |
| **Client** | Browser-side JavaScript | UI updates, local storage, navigation |
| **System** | Built-in ElevenLabs | End call, transfer, standard actions |
## Where Tools Live
Tools are defined inside `conversation_config.agent.prompt`. Webhook and client tools go in the `tools` array. System tools go in `built_in_tools`:
```python
conversation_config={
"agent": {
"prompt": {
"prompt": "You are helpful.",
"llm": "gemini-2.0-flash",
"tools": [...], # Webhook and client tools
"built_in_tools": {...} # System tools (end_call, transfer, etc.)
}
}
}
```
## Webhook Tools
Execute server-side logic when the agent needs external data or actions.
### Basic Webhook
```python
agent = client.conversational_ai.agents.create(
name="Weather Assistant",
conversation_config={
"agent": {
"prompt": {
"prompt": "You are a helpful assistant that can check the weather.",
"llm": "gemini-2.0-flash",
"tools": [{
"type": "webhook",
"name": "get_weather",
"description": "Get current weather for a city. Use when user asks about weather.",
"api_schema": {
"url": "https://api.example.com/weather",
"method": "POST",
"request_headers": {
"Authorization": "Bearer {{API_KEY}}"
},
"request_body_schema": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name, e.g., 'San Francisco'"
},
"units": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature units"
}
},
"required": ["city"]
}
}
}]
}
},
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
}
)
```
### Webhook Request Format
When the agent calls a webhook tool, ElevenLabs sends:
```json
{
"tool_call_id": "call_abc123",
"tool_name": "get_weather",
"parameters": {
"city": "San Francisco",
"units": "fahrenheit"
},
"conversation_id": "conv_xyz789"
}
```
### Webhook Response Format
Your server should respond with:
```json
{
"result": "The weather in San Francisco is 68°F and sunny."
}
```
Or for structured data:
```json
{
"result": {
"temperature": 68,
"condition": "sunny",
"humidity": 45
}
}
```
### Webhook with Authentication
```python
# Inside conversation_config.agent.prompt.tools:
{
"type": "webhook",
"name": "lookup_order",
"description": "Look up order status by order ID",
"response_timeout_secs": 10,
"api_schema": {
"url": "https://api.mystore.com/orders/lookup",
"method": "POST",
"request_headers": {
"Authorization": "Bearer {{ORDER_API_KEY}}",
"X-Store-ID": "store_123"
},
"request_body_schema": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "Order ID (e.g., ORD-12345)"
}
},
"required": ["order_id"]
}
}
}
```
Use workspace environment variables to keep a single server tool configuration working across
staging and production. `{{system_env__label}}` works in server tool URLs, secret environment
variables can populate `request_headers`, and auth-connection environment variables can populate
`api_schema.auth_connection`. The same environment-variable resolution model also applies to MCP
server connections.
```json
{
"api_schema": {
"url": "https://{{system_env__api_host}}.example.com/orders",
"method": "GET",
"request_headers": {
"X-Api-Key": { "env_var_label": "orders_api_key" }
},
"auth_connection": { "env_var_label": "orders_oauth" }
}
}
```
Workspace auth connections support OAuth2 client credentials, OAuth2 JWT, private key JWT,
basic auth, bearer auth, custom header auth, and mutual TLS (`mtls`).
System dynamic variables are also available in tool parameters and headers. Use
`{{system__conversation_history}}` when a webhook or sub-agent needs the full conversation
context as a lazily evaluated JSON history object with user, agent, and tool entries.
### Webhook Tool Options
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `response_timeout_secs` | int | `20` | Timeout in seconds (5-120) |
| `interruption_mode` | string | `"allow"` | Controls whether the user can interrupt around this tool call: `allow`, `disable_during_tool`, or `disable_during_tool_and_turn` |
| `execution_mode` | string | `"immediate"` | `immediate`, `post_tool_speech`, or `async` |
| `tool_call_sound` | string | - | Sound during execution: `typing`, `elevator1`-`elevator4` |
| `pre_tool_speech` | string | `"auto"` | Controls whether the agent speaks before execution: `auto`, `force`, or `off` |
| `tool_error_handling_mode` | string | `"auto"` | `auto`, `summarized`, `passthrough`, or `hide` |
| `api_schema.response_filter` | object | - | Filters JSON webhook responses before the LLM sees them. Use `mode: "allow"` with `filters` dot-paths to keep selected fields, or `mode: "hide_all"` to hide the response |
MCP server configuration supports the same `pre_tool_speech`, `interruption_mode`, `execution_mode`, and
`response_timeout_secs` controls at the server level, with per-tool overrides in
`tool_config_overrides`. Set a per-tool `tool_call_sound` override to `"off"` to silence that tool
while retaining the server default for other tools. MCP timeouts default to 30 seconds and must be
5-300 seconds.
Set `request_meta` on an MCP server configuration to send entries in the MCP `_meta` field of
`tools/call` requests. Values can be JSON scalars or references to workspace secrets, dynamic
variables, or environment variables that resolve for each call.
**Note:** The default `api_schema.method` is `GET`. Always set `"method": "POST"` explicitly for webhook tools that send request bodies.
### Server Implementation (Node.js)
```javascript
app.post("/webhook/get_weather", async (req, res) => {
const { parameters, conversation_id } = req.body;
const { city, units = "fahrenheit" } = parameters;
// Fetch weather from your data source
const weather = await weatherService.get(city, units);
res.json({
result: `It's ${weather.temp}°${units === "celsius" ? "C" : "F"} and ${weather.condition} in ${city}.`,
});
});
```
### Server Implementation (Python)
```python
@app.post("/webhook/get_weather")
async def get_weather(request: Request):
data = await request.json()
city = data["parameters"]["city"]
units = data["parameters"].get("units", "fahrenheit")
# Fetch weather from your data source
weather = weather_service.get(city, units)
return {
"result": f"It's {weather['temp']}°{'C' if units == 'celsius' else 'F'} and {weather['condition']} in {city}."
}
```
## Client Tools
Execute JavaScript in the user's browser. Useful for UI updates, navigation, or accessing browser APIs.
### Defining Client Tools
Client tools are registered when starting a conversation:
```javascript
import { Conversation } from "@elevenlabs/client";
const conversation = await Conversation.startSession({
agentId: "your-agent-id",
clientTools: {
show_product: async ({ productId }) => {
// Update UI to show product
const modal = document.getElementById("product-modal");
modal.innerHTML = await fetchProductCard(productId);
modal.showModal();
return { success: true, message: "Showing product" };
},
navigate_to: async ({ page }) => {
// Navigate to a page
window.location.href = `/${page}`;
return { success: true };
},
save_preference: async ({ key, value }) => {
// Store in localStorage
localStorage.setItem(key, value);
return { saved: true };
},
},
});
```
### React Registration with `useConversationClientTool`
When you use the React SDK, wrap your component tree in `ConversationProvider` and register
client tools from components with `useConversationClientTool`. Handlers are cleaned up
automatically on unmount and always use the latest closure value. Prefer granular hooks such as
`useConversationControls` and `useConversationStatus` for the session UI; `useConversation`
remains available when you want the full conversation object in one hook.
```typescript
import {
ConversationProvider,
useConversationClientTool,
useConversationControls,
useConversationStatus,
} from "@elevenlabs/react";
function Storefront() {
useConversationClientTool("show_product", async ({ productId }) => {
const modal = document.getElementById("product-modal");
modal.innerHTML = await fetchProductCard(productId);
modal.showModal();
return { success: true };
});
const { startSession, endSession } = useConversationControls();
const { status } = useConversationStatus();
if (status === "connected") {
return <button onClick={endSession}>End</button>;
}
return (
<button onClick={() => startSession({ agentId: "your-agent-id" })}>
Start
</button>
);
}
function App() {
return (
<ConversationProvider>
<Storefront />
</ConversationProvider>
);
}
```
### Registering Client Tools with Agent
Tell the agent about available client tools in `conversation_config.agent.prompt.tools`:
```python
agent = client.conversational_ai.agents.create(
name="Shopping Assistant",
conversation_config={
"agent": {
"prompt": {
"prompt": """You are a shopping assistant.
When users want to see a product, use show_product.
When users want to go somewhere, use navigate_to.""",
"llm": "gemini-2.0-flash",
"tools": [
{
"type": "client",
"name": "show_product",
"description": "Display a product card to the user",
"parameters": {
"type": "object",
"properties": {
"productId": {
"type": "string",
"description": "Product ID to display"
}
},
"required": ["productId"]
}
},
{
"type": "client",
"name": "navigate_to",
"description": "Navigate user to a different page",
"parameters": {
"type": "object",
"properties": {
"page": {
"type": "string",
"enum": ["cart", "checkout", "account", "home"],
"description": "Page to navigate to"
}
},
"required": ["page"]
}
}
]
}
},
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
}
)
```
### Client Tool Options
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `expects_response` | bool | `false` | Whether the tool returns data to the agent |
### Client Tool Return Values
Return data that the agent can use in conversation:
```javascript
clientTools: {
check_cart: async () => {
const cart = JSON.parse(localStorage.getItem("cart") || "[]");
return {
itemCount: cart.length,
total: cart.reduce((sum, item) => sum + item.price, 0),
items: cart.map((item) => item.name),
};
};
}
```
The agent receives this data and can say: "You have 3 items in your cart totaling $45.99."
## System Tools (built_in_tools)
Built-in tools provided by ElevenLabs. These are configured in `conversation_config.agent.prompt.built_in_tools` (not in the `tools` array):
```python
"built_in_tools": {
"end_call": {},
"transfer_to_number": {...},
"transfer_to_agent": {...},
"language_detection": {},
"skip_turn": {},
"voicemail_detection": {...},
"play_keypad_touch_tone": {}
}
```
Current API schemas also expose `agent_prompt_change`, `memory_entry_create`, `memory_entry_delete`, `memory_entry_search`, and `memory_entry_update` in `built_in_tools`.
Set `only_at_conversation_start: true` on the `language_detection` tool to constrain language
switching when no switch occurs in the first two user turns. Leave it `false` when the conversation
must remain able to switch languages later.
### end_call
Ends the current conversation:
```python
"built_in_tools": {
"end_call": {}
}
```
The agent can say "Goodbye!" and then end the call programmatically.
### transfer_to_number
Transfer to a phone number (requires telephony integration):
```python
"built_in_tools": {
"transfer_to_number": {
"transfers": [{
"transfer_destination": {"type": "phone", "phone_number": "+1234567890"},
"condition": "User asks to speak with a human agent"
}]
}
}
```
### transfer_to_agent
Transfer to another ElevenLabs agent or workflow node:
```python
"built_in_tools": {
"transfer_to_agent": {
"transfers": [{
"agent_id": "other-agent-id",
"node_id": "destination-workflow-node-id",
"preserve_client_tts_overrides": true,
"condition": "User asks about sales"
}]
}
}
```
Use `node_id` when the transfer should start at a specific workflow node. Omit
`agent_id` when the transfer stays within the current agent's workflow.
Set `preserve_client_tts_overrides` when client-side TTS overrides should continue
after the transfer.
## Best Practices
### Tool Descriptions
Write clear descriptions so the LLM knows when to use tools:
```python
# Good - specific and actionable
"description": "Look up order status. Use when customer asks about their order, delivery, or shipping."
# Bad - vague
"description": "Order tool"
```
### Parameter Descriptions
Help the LLM extract correct values:
```python
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "Order ID in format ORD-XXXXX (e.g., ORD-12345)"
},
"email": {
"type": "string",
"description": "Customer email address for verification"
}
}
}
```
For optional tool parameters that should never be sent in the request payload, set
`is_omitted: true` on the JSON schema property. Do not combine it with `description`,
`dynamic_variable`, `is_system_provided`, or `constant_value`.
### Error Handling
Configure how tool errors are shared with the agent using `tool_error_handling_mode`:
| Mode | Behavior |
|------|----------|
| `auto` | ElevenLabs automatically decides how to handle errors |
| `summarized` | Errors are summarized before being sent to the agent |
| `passthrough` | Full error details are passed to the agent |
| `hide` | Errors are hidden from the agent |
Return helpful error messages:
```javascript
// Server webhook
app.post("/webhook/lookup_order", async (req, res) => {
const { order_id } = req.body.parameters;
const order = await db.orders.find(order_id);
if (!order) {
return res.json({
result: {
error: true,
message: `Order ${order_id} not found. Please verify the order ID.`,
},
});
}
res.json({ result: order });
});
```
### Timeouts
Set reasonable timeouts for webhooks using `response_timeout_secs` (5-120 seconds, default 20). MCP server tool calls use the same field with a 30-second default and a 5-300 second range:
```python
{
"type": "webhook",
"name": "slow_operation",
"description": "Run a slow operation",
"response_timeout_secs": 30,
"api_schema": {
"url": "https://api.example.com/slow-operation",
"method": "POST"
}
}
```
## Complete Example
```python
agent = client.conversational_ai.agents.create(
name="E-commerce Assistant",
conversation_config={
"agent": {
"first_message": "Hi! How can I help you today?",
"language": "en",
"prompt": {
"prompt": """You are an e-commerce support assistant.
Available actions:
- lookup_order: Check order status
- show_product: Display products to customer
- end_call: End conversation politely
- transfer_to_number: Transfer to human support
Always verify order ID before lookup. Offer transfer for complex issues.""",
"llm": "gemini-2.0-flash",
"tools": [
# Webhook: Server-side order lookup
{
"type": "webhook",
"name": "lookup_order",
"description": "Look up order status by order ID or email",
"api_schema": {
"url": "https://api.mystore.com/orders/lookup",
"method": "POST",
"request_headers": {"Authorization": "Bearer {{API_KEY}}"},
"request_body_schema": {
"type": "object",
"properties": {
"order_id": {"type": "string"},
"email": {"type": "string"}
}
}
}
},
# Client: Browser-side product display
{
"type": "client",
"name": "show_product",
"description": "Display product details to the customer",
"parameters": {
"type": "object",
"properties": {
"product_id": {"type": "string"}
},
"required": ["product_id"]
}
}
],
"built_in_tools": {
"end_call": {},
"transfer_to_number": {
"transfers": [{
"transfer_destination": {"type": "phone", "phone_number": "+1234567890"},
"condition": "User asks for human support"
}]
}
}
}
},
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb", "model_id": "eleven_flash_v2_5"}
}
)
```
SHA-256: ac9b4afd7e0f880586d0f547b3dbea203a875bb1f5480c3c7a84f1af3c808e09