elevenlabs-agents

ElevenLabs Agent Builder

Build a production-ready conversational AI voice agent. Produces a configured agent with tools, knowledge base, and SDK integration.

Packages

npm install @elevenlabs/react           # React SDK
npm install @elevenlabs/client          # JavaScript SDK (browser + server)
npm install @elevenlabs/react-native    # React Native SDK
npm install @elevenlabs/elevenlabs-js   # Full API (server only)
npm install -g @elevenlabs/agents-cli   # CLI ("Agents as Code")

DEPRECATED: @11labs/react, @11labs/client -- uninstall if present.

Server-only warning: @elevenlabs/elevenlabs-js uses Node.js child_process and won't work in browsers. Use @elevenlabs/client for browser environments, or create a proxy server.

Workflow

Step 1: Create Agent via Dashboard or CLI

Dashboard: https://elevenlabs.io/app/conversational-ai -> Create Agent

CLI (Agents as Code):

elevenlabs agents init
elevenlabs agents add "Support Bot" --template customer-service
# Edit agent_configs/support-bot.json
elevenlabs agents push --env dev

Templates: default, minimal, voice-only, text-only, customer-service, assistant.

Configure:

• Voice -- Choose from 5000+ voices or clone
• LLM -- GPT, Claude, Gemini, or custom
• System prompt -- Use the 6-component framework below
• First message -- What the agent says when conversation starts

Step 2: Write the System Prompt

Use the 6-component framework for effective agent prompts:

1. Personality -- who the agent is:

You are [NAME], a [ROLE] at [COMPANY].
You have [EXPERIENCE]. Your traits: [LIST TRAITS].

2. Environment -- communication context:

You're communicating via [phone/chat/video].
Consider [environmental factors]. Adapt to [context].

3. Tone -- speech patterns and formality:

Tone: Professional yet warm. Use contractions for natural speech.
Avoid jargon. Keep responses to 2-3 sentences. Ask one question at a time.

4. Goal -- objectives and success criteria:

Primary Goal: Resolve customer issues on the first call.
Success: Customer verbally confirms issue is resolved.

5. Guardrails -- boundaries and ethics:

Never: provide medical/legal/financial advice, share confidential info.
Always: verify identity before account access, document interactions.
Escalation: customer requests manager, issue beyond knowledge base.

6. Tools -- available functions and when to use them:

1. lookup_order(order_id) -- Use when customer mentions an order.
2. transfer_to_supervisor() -- Use when issue requires manager approval.
Always explain what you're doing before calling a tool.

Step 3: Add Tools

Client-side tools (run in browser):

const clientTools = {
  updateCart: {
    description: "Add or remove items from the shopping cart",
    parameters: z.object({
      action: z.enum(['add', 'remove']),
      item: z.string(),
      quantity: z.number().min(1)
    }),
    handler: async ({ action, item, quantity }) => {
      const cart = getCart();
      action === 'add' ? cart.add(item, quantity) : cart.remove(item, quantity);
      return { success: true, total: cart.total, items: cart.items.length };
    }
  },
  navigate: {
    description: "Navigate user to a different page",
    parameters: z.object({ url: z.string().url() }),
    handler: async ({ url }) => { window.location.href = url; return { success: true }; }
  }
};

Server-side tools (webhooks):

{
  "name": "get_weather",
  "description": "Fetch current weather for a city",
  "url": "https://api.weather.com/v1/current",
  "method": "GET",
  "parameters": {
    "type": "object",
    "properties": {
      "city": { "type": "string", "description": "City name" }
    },
    "required": ["city"]
  },
  "headers": {
    "Authorization": "Bearer {{secret__weather_api_key}}"
  }
}

Use {{secret__key_name}} for API keys in webhook headers -- never hardcode.

MCP Tools -- CRITICAL COMPATIBILITY NOTE:

ElevenLabs labels their MCP integration as "Streamable HTTP" but does NOT support the actual MCP 2025-03-26 Streamable HTTP spec (SSE responses). ElevenLabs expects:

• Plain JSON responses (application/json), NOT SSE (text/event-stream)
• Protocol version 2024-11-05, NOT 2025-03-26
• Simple JSON-RPC over HTTP with direct JSON responses

What does NOT work:

• Official MCP SDK's createMcpHandler (returns SSE)
• Cloudflare Agents SDK McpServer.serve() (returns SSE)
• Any server returning Content-Type: text/event-stream

Working MCP server pattern for ElevenLabs:

import { Hono } from 'hono';
import { cors } from 'hono/cors';

const tools = [{
  name: "my_tool",
  description: "Tool description",
  inputSchema: {
    type: "object",
    properties: { param1: { type: "string", description: "Description" } },
    required: ["param1"]
  }
}];

async function handleMCPRequest(request, env) {
  const { id, method, params } = request;
  switch (method) {
    case 'initialize':
      return {
        jsonrpc: '2.0', id,
        result: {
          protocolVersion: '2024-11-05',  // MUST be 2024-11-05
          serverInfo: { name: 'my-mcp', version: '1.0.0' },
          capabilities: { tools: {} }
        }
      };
    case 'tools/list':
      return { jsonrpc: '2.0', id, result: { tools } };
    case 'tools/call':
      const result = await handleTool(params.name, params.arguments, env);
      return { jsonrpc: '2.0', id, result };
    default:
      return { jsonrpc: '2.0', id, error: { code: -32601, message: `Unknown: ${method}` } };
  }
}

const app = new Hono();
app.use('/*', cors({ origin: '*', allowMethods: ['GET', 'POST', 'OPTIONS'] }));
app.post('/mcp', async (c) => {
  const body = await c.req.json();
  return c.json(await handleMCPRequest(body, c.env));  // Plain JSON, NOT SSE
});
export default app;

Step 4: Add Knowledge Base (RAG)

Upload documents for the agent to reference:

• PDFs, text files, web URLs
• Configure via dashboard: Agent -> Knowledge Base -> Upload
• Or via API: POST /v1/convai/knowledge-base/upload (multipart/form-data)
• Agent automatically searches knowledge base during conversation

Step 5: Integrate SDK

React -- copy and customise assets/react-sdk-boilerplate.tsx:

import { useConversation } from '@elevenlabs/react';

const { startConversation, stopConversation, status } = useConversation({
  agentId: 'your-agent-id',
  signedUrl: '/api/elevenlabs/auth',
  clientTools,
  dynamicVariables: {
    user_name: 'John',
    account_type: 'premium',
  },
  onEvent: (event) => { /* transcript, agent_response, tool_call */ },
});

System prompt references dynamic variables as {{user_name}}.

React Native -- see assets/react-native-boilerplate.tsx Widget embed -- see assets/widget-embed-template.html Swift -- see assets/swift-sdk-boilerplate.swift

Step 6: Test

CLI testing:

# Run all tests for an agent
elevenlabs agents test "Support Agent"

# Add a test scenario
elevenlabs tests add "Refund Request" --template basic-llm

Test configuration:

{
  "name": "Refund Request Test",
  "scenario": "Customer requests refund for defective product",
  "user_input": "I want a refund for order #12345. The product arrived broken.",
  "success_criteria": [
    "Agent acknowledges the issue empathetically",
    "Agent asks for or uses provided order number",
    "Agent verifies order details",
    "Agent provides clear next steps or refund timeline"
  ],
  "evaluation_type": "llm"
}

Tool call testing:

{
  "name": "Order Lookup Test",
  "scenario": "Customer asks about order status",
  "user_input": "What's the status of order ORD-12345?",
  "expected_tool_call": {
    "tool_name": "lookup_order",
    "parameters": { "order_id": "ORD-12345" }
  }
}

API simulation:

const simulation = await client.agents.simulate({